Closed
Bug 974471
Opened 10 years ago
Closed 6 years ago
Expected failures in setUp/tearDown are counted as errors
Categories
(Remote Protocol :: Marionette, defect, P3)
Tracking
(Not tracked)
RESOLVED
WONTFIX
People
(Reporter: ato, Unassigned)
Details
(Keywords: pi-marionette-runner, Whiteboard: [runner])
From bsilverberg's comment in bug 941158: I'm not sure that this has been resolved properly. Currently, if a test is marked as an expected failure and it errors during tearDown it gets reported as both an expected failure and an error. I believe we should not be getting the latter. Here is a sample stack trace from a job [1] on our CI: test_import_contacts_from_gmail (test_import_contacts_from_gmail.TestImportContactsFromGmail) ... expected failure ERROR ====================================================================== ERROR: None ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_client-0.7.3-py2.7.egg/marionette/marionette_test.py", line 173, in run self.tearDown() File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 1119, in tearDown MarionetteTestCase.tearDown(self) File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_client-0.7.3-py2.7.egg/marionette/marionette_test.py", line 318, in tearDown self.marionette.set_context("content") File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_client-0.7.3-py2.7.egg/marionette/marionette.py", line 847, in set_context return self._send_message('setContext', 'ok', value=context) File "/var/jenkins/workspace/b2g.hamachi.mozilla-central.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_client-0.7.3-py2.7.egg/marionette/marionette.py", line 581, in _send_message raise MarionetteException(message="Please start a session") MarionetteException: MarionetteException: Please start a session TEST-UNEXPECTED-FAIL | test_import_contacts_from_gmail.py test_import_contacts_from_gmail.TestImportContactsFromGmail.test_import_contacts_from_gmail | ---------------------------------------------------------------------- Ran 1 test in 418.978s FAILED (errors=1, expected failures=1) [1] http://qa-selenium.mv.mozilla.com:8080/job/b2g.hamachi.mozilla-central.ui.adhoc/199/
Updated•10 years ago
|
Whiteboard: [runner]
Updated•9 years ago
|
Keywords: ateam-marionette-runner
Reporter | ||
Comment 2•6 years ago
|
||
Unlikely to ever be fixed.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → WONTFIX
Updated•1 year ago
|
Product: Testing → Remote Protocol
You need to log in
before you can comment on or make changes to this bug.
Description
•