Closed Bug 1131070 Opened 9 years ago Closed 9 years ago

Investigate test_toggle_airplane_mode intermittent failure

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

ARM
Gonk (Firefox OS)
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: RobertC, Assigned: martijn.martijn)

References

Details

Attachments

(3 files)

test_toggle_airplane_mode is failing intermittently with the following stacktrace:
Traceback (most recent call last):
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_client-0.8.7-py2.7.egg/marionette/marionette_test.py", line 283, in run
testMethod()
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/tests/python/gaia-ui-tests/gaiatest/tests/functional/settings/test_settings_airplane_mode.py", line 35, in test_toggle_airplane_mode
system_app.wait_for_airplane_mode_icon_displayed()
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/tests/python/gaia-ui-tests/gaiatest/apps/system/app.py", line 91, in wait_for_airplane_mode_icon_displayed
Wait(self.marionette).until(expected.element_displayed(*self._airplane_mode_statusbar_locator))
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_client-0.8.7-py2.7.egg/marionette/wait.py", line 143, in until
cause=last_exc)
TimeoutException: TimeoutException: Timed out after 30.0 seconds

Build info:
Device firmware (base) 	L1TC100118D0
Device firmware (date) 	09 Feb 2015 01:57:06
Device firmware (incremental) 	eng.cltbld.20150209.045655
Device firmware (release) 	4.4.2
Device identifier 	flame
Gaia date 	08 Feb 2015 12:44:20
Gaia revision 	0d7b35f23402
Gecko build 	20150209010211
Gecko revision 	3436787a82d0
Gecko version 	38.0a1

http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/15/HTML_Report/

In the screenshot we can see the aiplane icon in the statusbat.
The issue is not reproduced manually.

Locally, with automation I got a reproduction rate of 5 out of 11.
QA Whiteboard: [fxosqa-auto-backlog+]
I can't reproduce with:
Gaia-Rev        0d7b35f23402c4cb29bca6b98280fec48a196dec
Gecko-Rev       https://hg.mozilla.org/mozilla-central/rev/3436787a82d0
Build-ID        20150209010211
Version         38.0a1
Device-Name     flame
FW-Release      4.4.2
FW-Incremental  eng.cltbld.20150204.192122
FW-Date         Wed Feb  4 19:21:34 EST 2015
Bootloader      L1TC100118D0

Although I've turned off the connect_to_cell_data() function, since I currently don't have a working sim card here. But I don't think that should matter for this failure to be seen.
Oh wait, I have been able to reproduce it actually, once out of 11 times.
And out of another 11 times, it didn't reproduce. It's difficult to fix for me, since it's so hard to reproduce for me.
Attached file mar04_airplane.html
Repro rate for this seemed to get a lot worse: the failure on Jenkins seemed to indicate a connecting to data failure, however in local automation I encountered the same failure as in comment # 0, where the test is unable to locate the 'airplane mode' icon in the status bar. Does not reproduce manually, as the the icon is indeed updated and within the bar.

gaiatest --testvars=gaiatest/testvars_6253.json --address=localhost:2828 --restart --repeat=9 --timeout=30000 --html-output=results/mar04_airplane.html gaiatest/tests/functional/settings/test_settings_airplane_mode.py

http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/64/testReport/junit/%28root%29/test_settings_airplane_mode_py%20TestAirplaneMode_test_toggle_airplane_mode/test_settings_airplane_mode_py_TestAirplaneMode_test_toggle_airplane_mode/

Jenkins:
Traceback (most recent call last):
  File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_client-0.8.7-py2.7.egg/marionette/marionette_test.py", line 283, in run
    testMethod()
  File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/tests/python/gaia-ui-tests/gaiatest/tests/functional/settings/test_settings_airplane_mode.py", line 35, in test_toggle_airplane_mode
    system_app.wait_for_airplane_mode_icon_displayed()
  File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/tests/python/gaia-ui-tests/gaiatest/apps/system/app.py", line 91, in wait_for_airplane_mode_icon_displayed
    Wait(self.marionette).until(expected.element_displayed(*self._airplane_mode_statusbar_locator))
  File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_client-0.8.7-py2.7.egg/marionette/wait.py", line 143, in until
    cause=last_exc)

TimeoutException: TimeoutException: Timed out after 30.1 seconds


Local (plus attached html):
Traceback (most recent call last):
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_client-0.9-py2.7.egg/marionette/marionette_test.py", line 290, in run
    testMethod()
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/tests/python/gaia-ui-tests/gaiatest/tests/functional/messages/test_inbox_to_settings.py", line 21, in test_settings_from_inbox
    activities = messages.tap_options()
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/tests/python/gaia-ui-tests/gaiatest/apps/messages/app.py", line 34, in tap_options
    self.marionette.find_element(*self._options_icon_locator).tap()
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/marionette.py", line 80, in tap
    return self.marionette._send_message('singleTap', 'ok', id=self.id, x=x, y=y)
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/decorators.py", line 36, in _
    return func(*args, **kwargs)
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/marionette.py", line 670, in _send_message
    self._handle_error(response)
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/marionette.py", line 715, in _handle_error
    raise errors.ElementNotVisibleException(message=message, status=status, stacktrace=stacktrace)
ElementNotVisibleException: ElementNotVisibleException: Element is not currently visible and may not be manipulated
Flags: needinfo?(pbylenga)
Flags: needinfo?(pbylenga)
Attachment #8573947 - Flags: review?(martijn.martijn)
Attachment #8573947 - Flags: review?(martijn.martijn) → review+
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
It seems like autolander doesn't support "leave-open".
Status: RESOLVED → REOPENED
Keywords: leave-open
Resolution: FIXED → ---
Assignee: nobody → martijn.martijn
Comment on attachment 8682136 [details] [review]
[gaia] mwargers:1131070 > mozilla-b2g:master

I repeated this test 10 times and they all passed. I think we should re-enable this test and see how it goes.
Attachment #8682136 - Flags: review?(npark)
Attachment #8682136 - Flags: review?(jlorenzo)
Attachment #8682136 - Flags: review?(npark) → review+
Comment on attachment 8682136 [details] [review]
[gaia] mwargers:1131070 > mozilla-b2g:master

Yeah, let's see how it goes under 512MB.
Attachment #8682136 - Flags: review?(jlorenzo) → review+
Merged: https://github.com/mozilla-b2g/gaia/commit/4f65f9c741ef068b48f73d8f2da4b7a8a50368cf
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Depends on: 1221630
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: