1460059 - Intermittent /webdriver/tests/set_window_rect/resizing_and_positioning.py | test_window_resize - assert (400, 500) == (500, 600)

Reporter

Description

•

6 years ago

treeherder

Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=177546084&repo=autoland

https://queue.taskcluster.net/v1/task/MNpCKJfzSNK4vbRP-M5oyw/runs/0/artifacts/public/logs/live_backing.log

[task 2018-05-08T20:57:59.355Z] 20:57:59     INFO - STDOUT:     def test_window_resize(session):
[task 2018-05-08T20:57:59.355Z] 20:57:59     INFO - STDOUT:         if not window_size_supported(session):
[task 2018-05-08T20:57:59.356Z] 20:57:59     INFO - PID 16377 | 1525813079343	webdriver::server	DEBUG	<- 200 OK {"value": {}}
[task 2018-05-08T20:57:59.356Z] 20:57:59     INFO - STDOUT:             pytest.skip()
[task 2018-05-08T20:57:59.356Z] 20:57:59     INFO - STDOUT:     
[task 2018-05-08T20:57:59.356Z] 20:57:59     INFO - STDOUT:         # setting the window size by webdriver is synchronous
[task 2018-05-08T20:57:59.357Z] 20:57:59     INFO - STDOUT:         # so we should see the results immediately
[task 2018-05-08T20:57:59.357Z] 20:57:59     INFO - STDOUT:     
[task 2018-05-08T20:57:59.357Z] 20:57:59     INFO - STDOUT:         session.window.size = (400, 500)
[task 2018-05-08T20:57:59.358Z] 20:57:59     INFO - STDOUT:         assert session.window.size == (400, 500)
[task 2018-05-08T20:57:59.358Z] 20:57:59     INFO - STDOUT:     
[task 2018-05-08T20:57:59.358Z] 20:57:59     INFO - STDOUT:         session.window.size = (500, 600)
[task 2018-05-08T20:57:59.358Z] 20:57:59     INFO - STDOUT: >       assert session.window.size == (500, 600)
[task 2018-05-08T20:57:59.359Z] 20:57:59     INFO - STDOUT: E       assert (400, 500) == (500, 600)
[task 2018-05-08T20:57:59.359Z] 20:57:59     INFO - STDOUT: E         At index 0 diff: 400 != 500
[task 2018-05-08T20:57:59.359Z] 20:57:59     INFO - STDOUT: E         Full diff:
[task 2018-05-08T20:57:59.359Z] 20:57:59     INFO - STDOUT: E         - (400, 500)
[task 2018-05-08T20:57:59.360Z] 20:57:59     INFO - STDOUT: E         ?  ^    ^
[task 2018-05-08T20:57:59.360Z] 20:57:59     INFO - STDOUT: E         + (500, 600)
[task 2018-05-08T20:57:59.360Z] 20:57:59     INFO - STDOUT: E         ?  ^    ^
[task 2018-05-08T20:57:59.361Z] 20:57:59     INFO - STDOUT: session    = <Session 20f8272a-fff0-408e-8563-6f23121fd653>
[task 2018-05-08T20:57:59.361Z] 20:57:59     INFO - STDOUT: tests/web-platform/tests/webdriver/tests/set_window_rect/resizing_and_positioning.py
[task 2018-05-08T20:57:59.361Z] 20:57:59     INFO - STDOUT: :34: AssertionError
[task 2018-05-08T20:57:59.361Z] 20:57:59     INFO - STDOUT: =============================== warnings summary ===============================
[task 2018-05-08T20:57:59.362Z] 20:57:59     INFO - STDOUT: None
[task 2018-05-08T20:57:59.362Z] 20:57:59     INFO - STDOUT:   Module already imported so cannot be rewritten: mozlog
[task 2018-05-08T20:57:59.362Z] 20:57:59     INFO - STDOUT: -- Docs: http://doc.pytest.org/en/latest/warnings.html
[task 2018-05-08T20:57:59.362Z] 20:57:59     INFO - STDOUT: ===================== 1 failed, 1 warnings in 4.81 seconds =====================
[task 2018-05-08T20:57:59.391Z] 20:57:59     INFO - 
[task 2018-05-08T20:57:59.391Z] 20:57:59     INFO - TEST-UNEXPECTED-FAIL | /webdriver/tests/set_window_rect/resizing_and_positioning.py | test_window_resize - assert (400, 500) == (500, 600)
[task 2018-05-08T20:57:59.391Z] 20:57:59     INFO - session = <Session 20f8272a-fff0-408e-8563-6f23121fd653>
[task 2018-05-08T20:57:59.391Z] 20:57:59     INFO - 
[task 2018-05-08T20:57:59.392Z] 20:57:59     INFO -     def test_window_resize(session):
[task 2018-05-08T20:57:59.392Z] 20:57:59     INFO -         if not window_size_supported(session):
[task 2018-05-08T20:57:59.393Z] 20:57:59     INFO -             pytest.skip()
[task 2018-05-08T20:57:59.393Z] 20:57:59     INFO -     
[task 2018-05-08T20:57:59.393Z] 20:57:59     INFO -         # setting the window size by webdriver is synchronous
[task 2018-05-08T20:57:59.393Z] 20:57:59     INFO -         # so we should see the results immediately
[task 2018-05-08T20:57:59.394Z] 20:57:59     INFO -     
[task 2018-05-08T20:57:59.394Z] 20:57:59     INFO -         session.window.size = (400, 500)
[task 2018-05-08T20:57:59.394Z] 20:57:59     INFO -         assert session.window.size == (400, 500)
[task 2018-05-08T20:57:59.395Z] 20:57:59     INFO -     
[task 2018-05-08T20:57:59.395Z] 20:57:59     INFO -         session.window.size = (500, 600)
[task 2018-05-08T20:57:59.396Z] 20:57:59     INFO - >       assert session.window.size == (500, 600)
[task 2018-05-08T20:57:59.396Z] 20:57:59     INFO - E       assert (400, 500) == (500, 600)
[task 2018-05-08T20:57:59.397Z] 20:57:59     INFO - E         At index 0 diff: 400 != 500
[task 2018-05-08T20:57:59.397Z] 20:57:59     INFO - E         Full diff:
[task 2018-05-08T20:57:59.397Z] 20:57:59     INFO - E         - (400, 500)
[task 2018-05-08T20:57:59.398Z] 20:57:59     INFO - E         ?  ^    ^
[task 2018-05-08T20:57:59.398Z] 20:57:59     INFO - E         + (500, 600)
[task 2018-05-08T20:57:59.398Z] 20:57:59     INFO - E         ?  ^    ^
[task 2018-05-08T20:57:59.398Z] 20:57:59     INFO - 
[task 2018-05-08T20:57:59.399Z] 20:57:59     INFO - session    = <Session 20f8272a-fff0-408e-8563-6f23121fd653>
[task 2018-05-08T20:57:59.399Z] 20:57:59     INFO - 
[task 2018-05-08T20:57:59.399Z] 20:57:59     INFO - tests/web-platform/tests/webdriver/tests/set_window_rect/resizing_and_positioning.py:34: AssertionError
[task 2018-05-08T20:57:59.400Z] 20:57:59     INFO - TEST-OK | /webdriver/tests/set_window_rect/resizing_and_positioning.py | took 5053ms
[task 2018-05-08T20:57:59.422Z] 20:57:59  WARNING - u'runner_teardown': ()
[task 2018-05-08T20:57:59.424Z] 20:57:59     INFO - Starting runner
[task 2018-05-08T20:57:59.445Z] 20:57:59     INFO - PID 16542 | 1525813079442	geckodriver	INFO	geckodriver 0.20.1 ( '3b84be9ee9e6'  '2018-05-07 15:05 -0700')
[task 2018-05-08T20:57:59.445Z] 20:57:59     INFO - PID 16542 | 1525813079443	webdriver::httpapi	DEBUG	Creating routes
[task 2018-05-08T20:57:59.453Z] 20:57:59     INFO - PID 16542 | 1525813079446	geckodriver	INFO	Listening on 127.0.0.1:4446
[task 2018-05-08T20:57:59.945Z] 20:57:59     INFO - WebDriver HTTP server listening at http://127.0.0.1:4446/

Henrik Skupin [:whimboo][⌚️UTC+1]

Assignee

Comment 1

•

6 years ago

By the WebDriver specification Set Window Rect should return the current size but not the requested one:

> Return success with the JSON serialization of the current top-level browsing context’s window rect. 

But we return the requested size:

[task 2018-05-08T20:57:57.077Z] 20:57:57     INFO - PID 16377 | 1525813077068	geckodriver::marionette	TRACE	-> 76:[0,4,"WebDriver:SetWindowRect",{"height":600,"width":500,"x":null,"y":null}]
[task 2018-05-08T20:57:57.079Z] 20:57:57     INFO - PID 16377 | 1525813077071	Marionette	TRACE	0 -> [0,4,"WebDriver:SetWindowRect",{"height":600,"width":500,"x":null,"y":null}]
[task 2018-05-08T20:57:57.140Z] 20:57:57     INFO - PID 16377 | 1525813077132	Marionette	TRACE	0 <- [1,4,null,{"x":0,"y":0,"width":500,"height":600,"state":"normal"}]
[task 2018-05-08T20:57:57.142Z] 20:57:57     INFO - PID 16377 | 1525813077133	geckodriver::marionette	TRACE	<- [1,4,null,{"x":0,"y":0,"width":500,"height":600,"state":"normal"}]
[task 2018-05-08T20:57:57.143Z] 20:57:57     INFO - PID 16377 | 1525813077133	webdriver::server	DEBUG	<- 200 OK {"value": {"height":600,"width":500,"x":0,"y":0}}
[task 2018-05-08T20:57:57.144Z] 20:57:57     INFO - PID 16377 | 1525813077135	webdriver::server	DEBUG	-> GET /session/20f8272a-fff0-408e-8563-6f23121fd653/window/rect
[task 2018-05-08T20:57:57.145Z] 20:57:57     INFO - PID 16377 | 1525813077136	geckodriver::marionette	TRACE	-> 34:[0,5,"WebDriver:GetWindowRect",{}]
[task 2018-05-08T20:57:57.167Z] 20:57:57     INFO - PID 16377 | 1525813077161	Marionette	TRACE	0 -> [0,5,"WebDriver:GetWindowRect",{}]
[task 2018-05-08T20:57:57.168Z] 20:57:57     INFO - PID 16377 | 1525813077163	Marionette	TRACE	0 <- [1,5,null,{"x":0,"y":0,"width":400,"height":500,"state":"normal"}]
[task 2018-05-08T20:57:57.175Z] 20:57:57     INFO - PID 16377 | 1525813077165	geckodriver::marionette	TRACE	<- [1,5,null,{"x":0,"y":0,"width":400,"height":500,"state":"normal"}]
[task 2018-05-08T20:57:57.177Z] 20:57:57     INFO - PID 16377 | 1525813077165	webdriver::server	DEBUG	<- 200 OK {"value": {"height":500,"width":400,"x":0,"y":0}}

Andreas, can you fix that or mentor this bug?

Flags: needinfo?(ato)

Henrik Skupin [:whimboo][⌚️UTC+1]

Assignee

Comment 2

•

6 years ago

Maybe we need a new bug filed, if we cannot stabilize the test.

Andreas Tolfsen ❲:ato❳

Comment 3

•

6 years ago

We are returning the current size, but WebDriver:SetWindowRect ıs
occasionally racy and I haven’t been able to figure out how to fix
that.

Flags: needinfo?(ato)

Tiberius Oros[:tiberius_oros]

Comment 4

•

6 years ago

Starting retriggering this job for this failure: 'webdriver/tests/set_window_rect/resizing_and_positioning.py | test_window_resize - assert (400, 500) == (500, 600)'

https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-searchStr=linux32%2Fopt-web-platform-tests-wdspec-e10s&tochange=3b84be9ee9e6a19d8e49aa86b71c9d60b5eaec8b&fromchange=c5e2684bc485bbcd323f5ce65c61edbc91e492e3&selectedJob=177546084

on the following interval in treeherder:
https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-searchStr=linux32%2Fopt-web-platform-tests-wdspec-e10s&tochange=3b84be9ee9e6a19d8e49aa86b71c9d60b5eaec8b&fromchange=c5e2684bc485bbcd323f5ce65c61edbc91e492e3

and ended up with this push as the point of origin for the above failure:
https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=9f13b6c1bfd783e5d1aed14989b58fc5fcaa8502&filter-searchStr=linux32%2Fopt-web-platform-tests-wdspec-e10s

Flags: needinfo?(hskupin)

Whiteboard: [retriggered]

Henrik Skupin [:whimboo][⌚️UTC+1]

Assignee

Comment 5

•

6 years ago

Correct, because this patch refactored the folder structure of the tests. As such this specific file has been moved from the `contexts` sub-folder to `set_window_rect` sub-folder. At this point we might have a different pre-condition for the test, which means that the window size is different as from before the refactoring. Which could cause a problem like that.

Flags: needinfo?(hskupin)

Henrik Skupin [:whimboo][⌚️UTC+1]

Assignee

Comment 6

•

6 years ago

Hm, we have resizing covered completely under set.py, so not sure if we need this file at all anymore. The negative tests might be of interest. Also note that the test is using `session.window.size` to resize the window, and not a custom helper method as what we use in all the other tests.

Henrik Skupin [:whimboo][⌚️UTC+1]

Assignee

Updated

•

6 years ago

Assignee: nobody → hskupin

Status: NEW → ASSIGNED

status-firefox62: --- → affected

Priority: P5 → P2

Comment hidden (mozreview-request)

Review commit: https://reviewboard.mozilla.org/r/243372/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/243372/

Henrik Skupin [:whimboo][⌚️UTC+1]

Assignee

Updated

•

6 years ago

Attachment #8974983 - Flags: review?(ato)

Andreas Tolfsen ❲:ato❳

Comment 8

•

6 years ago

mozreview-review

Comment on attachment 8974983 [details]
Bug 1460059 - [wdspec] Remove superfluous set_window_rect/resizing_and_positioning.py.

https://reviewboard.mozilla.org/r/243372/#review249456

Attachment #8974983 - Flags: review?(ato) → review+

Pulsebot

Comment 9

•

6 years ago

Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/5cbb1c6750d1
[wdspec] Remove superfluous set_window_rect/resizing_and_positioning.py. r=ato

Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)

Comment 10

•

6 years ago

Created web-platform-tests PR https://github.com/w3c/web-platform-tests/pull/10974 for changes under testing/web-platform/tests

Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)

Updated

•

6 years ago

Whiteboard: [retriggered] → [retriggered][wptsync upstream]

Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)

Comment 11

•

6 years ago

Upstream web-platform-tests status checks passed, PR will merge once commit reaches central.

Andrei Ciure[:aciure]

Comment 12

•

6 years ago

bugherder

https://hg.mozilla.org/mozilla-central/rev/5cbb1c6750d1

Status: ASSIGNED → RESOLVED

Closed: 6 years ago

status-firefox62: affected → fixed

Resolution: --- → FIXED

Target Milestone: --- → mozilla62

Comment hidden (Intermittent Failures Robot)

34 failures in 660 pushes (0.052 failures/push) were associated with this bug in the last 7 days. 

This is the #41 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* try: 11
* mozilla-inbound: 10
* autoland: 10
* mozilla-central: 3

Platform breakdown:
* linux32: 16
* linux64: 12
* osx-10-10: 3
* linux64-nightly: 1
* linux64-ccov: 1
* lint: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1460059&startday=2018-05-07&endday=2018-05-13&tree=trunk

Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)

Comment 14

•

6 years ago

Upstream PR merged

BMO Automation

Comment 15

•

1 year ago

Moving bug to Remote Protocol::Marionette component per bug 1815831.

Component: geckodriver → Marionette

Product: Testing → Remote Protocol