Closed
Bug 1294075
Opened 8 years ago
Closed 5 years ago
Intermittent test_accessibility.py TestAccessibility.test_element_is_not_enabled_to_accessbility | AssertionError: ElementNotAccessibleException not raised
Categories
(Testing :: Marionette Client and Harness, defect, P3)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: intermittent-bug-filer, Assigned: bdahl)
References
Details
(Keywords: intermittent-failure, Whiteboard: [stockwell disabled])
Attachments
(1 file)
867 bytes,
patch
|
whimboo
:
review+
|
Details | Diff | Splinter Review |
Filed by: philringnalda [at] gmail.com https://treeherder.mozilla.org/logviewer.html#?job_id=11084015&repo=fx-team https://queue.taskcluster.net/v1/task/c2HTwMJdSNCDxW-0F-bLDg/runs/0/artifacts/public%2Flogs%2Flive_backing.log
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 3•7 years ago
|
||
According to https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1294075&entireHistory=true&tree=all, this appears to have spiked around 5 April.
Comment 4•7 years ago
|
||
I’m occasionally hitting this when running testing/marionette/harness/marionette_harness/tests/unit/test_accessibility.py on Linux opt locally. I’m working on bug 1321516 which has to make a few changes to the expectations of the test due to a new ‘element click intercepted’ error introduced by the WebDriver conforming element click implementation.
Comment 5•7 years ago
|
||
So it first appeared actually on April 4th (https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=1606323068fa116336bd6109f184b165d778f2c9&selectedJob=88546569). It might be helpful to have a pushlog of all the changes between the nightly builds when this failure appeared.
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 15•7 years ago
|
||
wow, this really has increased in frequency as of May 22nd: https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1294075&startday=2017-05-01&endday=2017-05-30&tree=all this is almost always occurring on on linux64 opt/pgo and on marionette-headless. here is a recent log from today: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=103174842&lineNumber=1354 the failures I see in the log do not appear to be useful: [task 2017-05-30T21:32:05.577507Z] 21:32:05 INFO - TEST-START | test_accessibility.py TestAccessibility.test_element_is_not_enabled_to_accessbility [task 2017-05-30T21:32:05.875538Z] 21:32:05 INFO - TEST-UNEXPECTED-FAIL | test_accessibility.py TestAccessibility.test_element_is_not_enabled_to_accessbility | AssertionError: ElementNotAccessibleException not raised [task 2017-05-30T21:32:05.876444Z] 21:32:05 INFO - Traceback (most recent call last): [task 2017-05-30T21:32:05.877607Z] 21:32:05 INFO - File "/home/worker/workspace/build/venv/local/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 166, in run [task 2017-05-30T21:32:05.878211Z] 21:32:05 INFO - testMethod() [task 2017-05-30T21:32:05.878838Z] 21:32:05 INFO - File "/home/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_accessibility.py", line 173, in test_element_is_not_enabled_to_accessbility [task 2017-05-30T21:32:05.879447Z] 21:32:05 INFO - lambda element: self.assertRaises(ElementNotAccessibleException, [task 2017-05-30T21:32:05.880105Z] 21:32:05 INFO - File "/home/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_accessibility.py", line 94, in run_element_test [task 2017-05-30T21:32:05.880181Z] 21:32:05 INFO - testFn(element) [task 2017-05-30T21:32:05.880731Z] 21:32:05 INFO - File "/home/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_accessibility.py", line 174, in <lambda> [task 2017-05-30T21:32:05.881271Z] 21:32:05 INFO - element.is_enabled)) [task 2017-05-30T21:32:05.881851Z] 21:32:05 INFO - TEST-INFO took 217ms :whimboo can we fix this issue this week or consider disabling this test? The failure rate is too high to provide value while running this test.
Flags: needinfo?(hskupin)
Whiteboard: [stockwell needswork]
Comment 16•7 years ago
|
||
It's not up to me. Brendan is working on some improvements for Marionette headless tests, so lets ask him.
Flags: needinfo?(hskupin) → needinfo?(bdahl)
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Assignee | ||
Comment 21•7 years ago
|
||
This appears to be the same issue as bug 1341493. I'll disable on headless for now, but the fix in https://bugzilla.mozilla.org/show_bug.cgi?id=1341493#c28 is the real solution. Also, as I mentioned in that bug, the issue appears to not really be headless but some of the accessibility code, so this won't be a high priority for me.
Flags: needinfo?(bdahl)
Assignee | ||
Comment 22•7 years ago
|
||
Attachment #8874513 -
Flags: review?(hskupin)
Comment hidden (Intermittent Failures Robot) |
Comment 24•7 years ago
|
||
Comment on attachment 8874513 [details] [diff] [review] Disable marionette accessibility test on headless. Review of attachment 8874513 [details] [diff] [review]: ----------------------------------------------------------------- A mozreview patch would have been nice so I could have pushed it immediately. So please do so yourself, once you are online. Thanks.
Attachment #8874513 -
Flags: review?(hskupin) → review+
Comment 25•7 years ago
|
||
Pushed by bdahl@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/1b21c086b953 Disable marionette accessibility test on headless.
Updated•7 years ago
|
Keywords: leave-open
Comment 26•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/1b21c086b953
Comment hidden (Intermittent Failures Robot) |
Updated•7 years ago
|
Whiteboard: [stockwell needswork] → [stockwell disabled]
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 32•7 years ago
|
||
Hopefully bug 1341493 fixed this one as well. Early OrangeFactor results are looking promising.
Comment hidden (Intermittent Failures Robot) |
Updated•6 years ago
|
Priority: -- → P3
Comment 34•5 years ago
|
||
The leave-open keyword is there and there is no activity for 6 months. :automatedtester, maybe it's time to close this bug?
Flags: needinfo?(dburns)
Comment 35•5 years ago
|
||
This appears to have been reenabled
Status: NEW → RESOLVED
Closed: 5 years ago
Flags: needinfo?(dburns)
Resolution: --- → FIXED
Updated•5 years ago
|
Assignee: nobody → bdahl
Updated•5 years ago
|
Keywords: leave-open
Updated•1 year ago
|
Product: Testing → Remote Protocol
Comment 37•1 year ago
|
||
Moving bug to Testing::Marionette Client and Harness component per bug 1815831.
Component: Marionette → Marionette Client and Harness
Product: Remote Protocol → Testing
You need to log in
before you can comment on or make changes to this bug.
Description
•