Closed Bug 1157316 Opened 9 years ago Closed 9 years ago

test_lockscreen_time_check.py: " JavascriptException: TypeError: document.getElementById(...) is null "

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

ARM
Gonk (Firefox OS)
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: onelson, Assigned: martijn.martijn)

References

()

Details

Attachments

(1 file)

Description:
test_lockscreen_time_check.py is failing consistently in mozilla-central.nightly non-smoke.2 report:

http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/174/HTML_Report/

Repro Steps:
1) Open Settings app
2) Navigate to 'Date and Time'
3) Observe current time
4) Change from 'Automatic' to 'Manual' time
5) Lock screen and observe phone time
6) Unlock phone and change from 'Manual' to 'Automatic'
7) Lock screen and observe phone time
8) Lock and unlock phone, observe phone time

Actual:
Phone time stays accurate with user modifications; test fails out unexpectedly

Expected: 
Phone time stays accurate with user modifications


Traceback (most recent call last):
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/.env/local/lib/python2.7/site-packages/marionette_client-0.11-py2.7.egg/marionette/marionette_test.py", line 296, in run
testMethod()
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/tests/python/gaia-ui-tests/gaiatest/tests/functional/lockscreen/test_lockscreen_time_check.py", line 66, in test_lockscreen_time_check
difference = lock_screen.time_in_datetime - old_time
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/tests/python/gaia-ui-tests/gaiatest/apps/lockscreen/app.py", line 57, in time_in_datetime
return datetime.strptime(self.time, '%I:%M %p')
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/tests/python/gaia-ui-tests/gaiatest/apps/lockscreen/app.py", line 52, in time
"return document.getElementById('%s').innerHTML" % self._time_locator[1])
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/.env/local/lib/python2.7/site-packages/marionette_driver-0.4-py2.7.egg/marionette_driver/marionette.py", line 1388, in execute_script
filename=os.path.basename(frame[0]))
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/.env/local/lib/python2.7/site-packages/marionette_driver-0.4-py2.7.egg/marionette_driver/decorators.py", line 36, in _
return func(*args, **kwargs)
File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/.env/local/lib/python2.7/site-packages/marionette_driver-0.4-py2.7.egg/marionette_driver/marionette.py", line 715, in _send_message
    self._handle_error(response)
  File "/var/jenkins/2/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/.env/local/lib/python2.7/site-packages/marionette_driver-0.4-py2.7.egg/marionette_driver/marionette.py", line 751, in _handle_error
    raise errors.lookup(status)(message, stacktrace=stacktrace)
JavascriptException: JavascriptException: TypeError: document.getElementById(...) is null
stacktrace:
execute_script @app.py, line 52
inline javascript, line 1246
src: "let __marionetteFunc = function(){return document.getElementById('lockscreen-clock-time').innerHTML};__marionetteFunc.apply(null, __marionetteParams);"


Environmental Variables:
Device firmware (base) 	L1TC100118D0
Device firmware (date) 	22 Apr 2015 08:50:37
Device firmware (incremental) 	eng.cltbld.20150422.045025
Device firmware (release) 	4.4.2
Device identifier 	flame
Device memory 	219772 kB
Device serial 	7f86cae1
Device uptime 	0 days 0 hours 2 minutes 14 seconds
Gaia date 	21 Apr 2015 19:52:45
Gaia revision 	15134b080b5f
Gecko build 	20150422010202
Gecko revision 	946ac85af8f4
Gecko version 	40.0a1


Reproducible manually: NO

Repro frequency: 0/5


The last line of the error traceback indicates that we're unable to find our time element, possible rename of the html/css.

document.getElementById(...) is null
...
src: "let __marionetteFunc = function(){return document.getElementById('lockscreen-clock-time').innerHTML};...

Either missing element, or the handle we have on the phone in between rapid lockings of the phone is interrupting the tests ability to find these elements.
QA Whiteboard: [QAnalyst-Triage?][fxosqa-auto-backlog?]
Flags: needinfo?(pbylenga)
QA Whiteboard: [QAnalyst-Triage?][fxosqa-auto-backlog?] → [QAnalyst-Triage+][fxosqa-auto-backlog?]
Flags: needinfo?(pbylenga)
Assignee: nobody → martijn.martijn
self.device.turn_screen_on() switches to displayed app, which apparently isn't the system app anymore:
http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/gaia_test.py#749
Comment on attachment 8609602 [details] [review]
[gaia] mwargers:1157316 > mozilla-b2g:master

This should make it work.
However, running this test with this pull request applied, I still get this failure then:
TEST-UNEXPECTED-FAIL | test_lockscreen_time_check.py TestLockScreen.test_lockscreen_time_check | AssertionError: 79260 not less than or equal to 240

Traceback (most recent call last):
  File "/Users/mwargers/.virtualenvs/gaia-py-latest/lib/python2.7/site-packages/marionette_client-0.13-py2.7.egg/marionette/marionette_test.py", line 296, in run
    testMethod()
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/tests/functional/lockscreen/test_lockscreen_time_check.py", line 69, in test_lockscreen_time_check
    self.assertLessEqual(difference.seconds, 240)
TEST-INFO took 105771ms
Attachment #8609602 - Flags: review?(onelson)
Attachment #8609602 - Flags: review?(npark)
Attachment #8609602 - Flags: review?(jlorenzo)
Comment on attachment 8609602 [details] [review]
[gaia] mwargers:1157316 > mozilla-b2g:master

I ran a local 'repeat=5' run and encountered a pass! The other 4 runs however encountered the same time error you observed:

Traceback (most recent call last):
  File "/home/flash/Desktop/oliverthor/git/gaia/tests/python/gaia-ui-tests/.env/local/lib/python2.7/site-packages/marionette_client-0.13-py2.7.egg/marionette/marionette_test.py", line 296, in run
    testMethod()
  File "/home/flash/Desktop/oliverthor/git/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/lockscreen/test_lockscreen_time_check.py", line 69, in test_lockscreen_time_check
    self.assertLessEqual(difference.seconds, 240)
TEST-INFO took 106524ms

Assuming we want to change the error for the reports as something more manageable to take care. This is the true error affecting the case now that marionette has switched frames to consider the lockscreen. Could consider xfailing this for now, time seems tricky to handle on the phone and we've been failing this test since April 21st:

* http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/173/

It's not a smoketest though so that may play a factor in this decision.
Either way, r+ for revealing the underlying issue.
Attachment #8609602 - Flags: review?(onelson) → review+
Comment on attachment 8609602 [details] [review]
[gaia] mwargers:1157316 > mozilla-b2g:master

I ran it locally for 10 times, and it all passed.  If they actually fail for the difference greater than 240, it means there is an actual bug with displaying the correct time probably.  There was bug 1140027 that exactly did this with the time change, but it was closed recently.  Its blocker bugs, Bug 1141258 is still open (and I can still repro it) and bug 1061797 is closed with being dup for 1110010, which is reopened, so I am not too surprised if you see the error.  If that happens on jenkins, we should disable this test, but your fix itself looks good to me.
Attachment #8609602 - Flags: review?(npark) → review+
I filed bug 1167848 to disable this test until bug 1141258 is fixed.
https://github.com/mozilla-b2g/gaia/commit/4bcd260ae62f2d40db8523f92cf7590b91d09465
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Attachment #8609602 - Flags: review?(jlorenzo)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: