Viorela Ioia [:viorela] Reporter
	Description • 10 years ago

Test test_sms_with_attachments.py is failing intermittently on b2g-inbound: jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.b2g-inbound.ui.functional.smoke/1534/HTML_Report/

I was able to reproduce this failure locally too, by running the automated test several times. It is not reproducible manually
This test is failing after selecting the picture to add to sms, while switching from camera to messages frame. 
Repro rate of the test failure while running locally: 2/5

Traceback (most recent call last):
File "/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.5-py2.7.egg/marionette/marionette_test.py", line 267, in run
testMethod()
File "/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/tests/functional/messages/test_sms_with_attachments.py", line 36, in test_sms_send
camera.tap_select_button()
File "/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/camera/app.py", line 88, in tap_select_button
select.tap()
File "/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.5-py2.7.egg/marionette/marionette.py", line 89, in tap
return self.marionette._send_message('singleTap', 'ok', id=self.id, x=x, y=y)
File "/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.5-py2.7.egg/marionette/decorators.py", line 36, in _
return func(*args, **kwargs)
File "/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.5-py2.7.egg/marionette/marionette.py", line 611, in _send_message
"Connection timed out", status=errors.ErrorCodes.TIMEOUT)
TimeoutException: TimeoutException: Connection timed out

Most recent errors/exceptions are:
12-09 07:14:43.038 E/adsprpc ( 201): vendor/qcom/proprietary/adsprpc/src/listener_android.c:128::error: -1: 0 == (nErr = __QAIC_HEADER(adsp_listener_next_invoke)( ctx, nErr, 0, 0, &ctx, &handle, &sc, bufs->inbufs, inBufsAllocated, bufs->inbufLenReqs, MAX_BUFS, bufs->outbufLenReqs, MAX_BUFS))12-09 07:14:43.048 E/adsprpc ( 201): vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:159:exit: closing /dev/adsprpc-smd, rpc errors are expected.12-09 07:16:50.148 W/GeckoConsole(27339): [JavaScript Error: "NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMessageSender.sendAsyncMessage]" {file: "chrome://specialpowers/content/SpecialPowersObserver.js" line: 103}]12-09 07:18:06.778 W/GeckoConsole(27339): [JavaScript Error: "NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMessageSender.sendAsyncMessage]" {file: "chrome://specialpowers/content/SpecialPowersObserver.js" line: 103}]12-09 07:18:09.378 W/GeckoConsole(27339): [JavaScript Error: "NS_ERROR_NOT_INITIALIZED: Component returned failure code: 0xc1f30001 (NS_ERROR_NOT_INITIALIZED) [nsIMessageSender.sendAsyncMessage]" {file: "chrome://specialpowers/content/SpecialPowersObserver.js" line: 103}]
The B2G process has restarted after crashing during the tests so Marionette can't respond due to either a Gecko, Gaia or Marionette error. Above, the 5 most recent errors are listed. Check logcat for all errors if these errors are not the cause of the failure.


Device firmware (base) 	L1TC10011880
Device firmware (date) 	08 Dec 2014 01:33:09
Device firmware (incremental) 	eng.cltbld.20141208.043252
Device firmware (release) 	4.4.2
Device identifier 	flame
Gaia date 	08 Dec 2014 00:49:22
Gaia revision 	dce6ee37d663
Gecko build 	20141208010735
Gecko revision 	178ab963d033
Gecko version 	37.0a1

This might be an IndexedDB issue, as in the logcat I saw:

W/Camera  (29485): [JavaScript Warning: "An IndexedDB transaction that was not yet complete has been aborted due to page navigation." {file: "app://camera.gaiamobile.org/js/main.js" line: 5}]
W/Camera  (29485): [JavaScript Warning: "An IndexedDB transaction that was not yet complete has been aborted due to page navigation." {file: "app://camera.gaiamobile.org/js/main.js" line: 5}]
I/Gecko   (28967): [Parent 28967] WARNING: waitpid failed pid:29485 errno:10: file ../../../gecko/ipc/chromium/src/base/process_util_posix.cc, line 267
I/Gecko   (28967): [Parent 28967] WARNING: pipe error (129): Connection reset by peer: file ../../../gecko/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 457
I/Gecko   (28967):
I/Gecko   (28967): ###!!! [Parent][MessageChannel] Error: Channel error: cannot send/recv

We need to investigate this further.

	Florin Strugariu [:Bebe]
	Comment 2 • 10 years ago

I ran some tests on this issue and can't reproduce it on:
Gaia-Rev        e074eac8f2cb4cfbfd43a23ee55c51a42857dbac
Gecko-Rev       https://hg.mozilla.org/integration/b2g-inbound/rev/e2d09a67c491
Build-ID        20141211024215
Version         37.0a1
Device-Name     flame
FW-Release      4.4.2
FW-Incremental  eng.cltbld.20141211.060526
FW-Date         Thu Dec 11 06:05:42 EST 2014
Bootloader      L1TC10011880

	Robert Chira [:RobertC]
	Comment 3 • 10 years ago

The issue described in this bug is also seen in Bug 1111981. The reproduction rate on Jenkins for both of the tests is quite big.

Wilson, do you know what could cause the IndexDB issues seen in the logcat?

	Robert Chira [:RobertC]
	Comment 4 • 10 years ago

STR from the test:
1. open Messages app
2. tap create new message
3. type phone number and message (I do not believe this is necessary to reproduce the failure)
4. tap attachment
5. tap camera
6. capture image
7. tap select

Step 7 is where the automated test fails.

	Robert Chira [:RobertC]
	Comment 5 • 10 years ago

After the discussion on IRC I tried adding a wait for the image to load before tapping on the select button, but that didn't work. I also tried waiting for an image to be saved on the device before tapping on select, unfortunately I got the same result.

I checked the device after the failure and the picture is visible in the gallery app.

	Robert Chira [:RobertC]
	Comment 6 • 10 years ago

I also tried adding a hard-coded 5 second sleep between capturing the picture and tapping on the select button and I still get the error.

Wilson, could this be related to a race condition when we tap select?

	Wilson Page [:wilsonpage]
	Comment 7 • 10 years ago

I'm confused because after the picture has been saved all indexeddb transactions should be complete. I don't know how else to advise from here :(

	Martijn Wargers (dead) Assignee
	Comment 8 • 10 years ago

Is this something that can be reproduced manually, also?

	Florin Strugariu [:Bebe]
	Comment 9 • 10 years ago

I was not able to reproduce this manually.

I managed to reproduce this locally using the automated test, but it has a very low reproduction rate locally 1/10 

On Jenkins this has a higher reproduction rate.
On flame-kk-319.b2g-inbound.ui.functional.smoke: 

  Failed 15 times in the last 30 runs. Stability: 50 %

	Oliver Nelson [:oliverthor]
	Comment 10 • 9 years ago

Saw this issue again today on Jenkins and reproed in local automation (1/10 similiar to comment # 9)
http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/42/

Jenkins reports that this case has failed 4 times in the last 20 runs. Stability: 80%

Traceback (most recent call last):
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.7-py2.7.egg/marionette/marionette_test.py", line 283, in run
    testMethod()
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/tests/functional/messages/test_sms_with_attachments.py", line 36, in test_sms_send
    camera.tap_select_button()
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/apps/camera/app.py", line 107, in tap_select_button
    select.tap()
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.7-py2.7.egg/marionette/marionette.py", line 89, in tap
    return self.marionette._send_message('singleTap', 'ok', id=self.id, x=x, y=y)
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.7-py2.7.egg/marionette/decorators.py", line 36, in _
    return func(*args, **kwargs)
  File "/var/jenkins/1/workspace/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.7-py2.7.egg/marionette/marionette.py", line 615, in _send_message
    "Connection timed out", status=errors.ErrorCodes.TIMEOUT)
TimeoutException: TimeoutException: Connection timed out

	Oliver Nelson [:oliverthor]
	Comment 11 • 9 years ago

This test failed today during the mozilla-central.nightly smoke, and when reproduced via Jenkins adhoc I received two different tracebacks: one produced with a screenshot in camera (1/5), and another with screenshot in messages (3/5).

http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/141/HTML_Report/
http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.adhoc/767/HTML_Report/


From Jenkins adhoc, test performed 5 times:

[1] Passed 1/5

[2] Camera Screenshot (1/5):
Traceback (most recent call last):
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_client-0.9-py2.7.egg/marionette/marionette_test.py", line 290, in run
    testMethod()
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/tests/functional/messages/test_sms_with_attachments.py", line 33, in test_sms_send
    camera = activities_list.tap_camera()
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/apps/system/regions/activities.py", line 70, in tap_camera
    camera.wait_for_loading_spinner_hidden()
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/apps/camera/app.py", line 47, in wait_for_loading_spinner_hidden
    loading_spinner = self.marionette.find_element(*self._loading_screen_locator)
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/marionette.py", line 1448, in find_element
    response = self._send_message('findElement', 'value', **kwargs)
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/decorators.py", line 36, in _
    return func(*args, **kwargs)
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/marionette.py", line 670, in _send_message
    self._handle_error(response)
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/marionette.py", line 709, in _handle_error
    raise errors.NoSuchElementException(message=message, status=status, stacktrace=stacktrace)
NoSuchElementException: NoSuchElementException: Unable to locate element: .loading-screen



[3] Messages Screenshot (3/5 + 1 from mozilla-central.nightly.smoke):
Traceback (most recent call last):
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_client-0.9-py2.7.egg/marionette/marionette_test.py", line 290, in run
    testMethod()
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/tests/functional/messages/test_sms_with_attachments.py", line 36, in test_sms_send
    camera.tap_select_button()
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/apps/camera/app.py", line 104, in tap_select_button
    Wait(self.marionette).until(lambda m: self.apps.displayed_app.name != self.name)
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/wait.py", line 122, in until
    rv = condition(self.marionette)
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/apps/camera/app.py", line 104, in <lambda>
    Wait(self.marionette).until(lambda m: self.apps.displayed_app.name != self.name)
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 78, in displayed_app
    self.marionette.switch_to_frame()
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/marionette.py", line 1139, in switch_to_frame
    response = self._send_message('switchToFrame', 'ok', id=frame, focus=focus)
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/decorators.py", line 36, in _
    return func(*args, **kwargs)
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/marionette.py", line 670, in _send_message
    self._handle_error(response)
  File "/var/jenkins/1/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.2-py2.7.egg/marionette_driver/marionette.py", line 747, in _handle_error
    raise errors.FrameSendNotInitializedError(message=message, status=status, stacktrace=stacktrace)
FrameSendNotInitializedError: FrameSendNotInitializedError: Error sending message to frame (NS_ERROR_NOT_INITIALIZED)

	Oliver Nelson [:oliverthor]
	Comment 12 • 9 years ago

Observing new consistent errors that result in a slightly different traceback for both:
* test_sms_with_attachments.py
* test_sms_with_picture_attached.py

http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.adhoc/767/HTML_Report/

Traceback (most recent call last):
File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_client-0.12-py2.7.egg/marionette/marionette_test.py", line 296, in run
testMethod()
File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/tests/functional/messages/test_sms_with_picture_attached.py", line 54, in test_sms_cropped_picture
crop_view.tap_crop_done()
File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/apps/gallery/regions/crop_view.py", line 38, in tap_crop_done
Wait(self.marionette).until(lambda m: self.apps.displayed_app.src != self._src)
File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.5-py2.7.egg/marionette_driver/wait.py", line 122, in until
rv = condition(self.marionette)
File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/apps/gallery/regions/crop_view.py", line 38, in <lambda>
Wait(self.marionette).until(lambda m: self.apps.displayed_app.src != self._src)
File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 106, in displayed_app
self.marionette.switch_to_frame()
File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.5-py2.7.egg/marionette_driver/marionette.py", line 1160, in switch_to_frame
return self._send_message("switchToFrame", "ok", **kwargs)
File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.5-py2.7.egg/marionette_driver/decorators.py", line 36, in _
return func(*args, **kwargs)
File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.5-py2.7.egg/marionette_driver/marionette.py", line 716, in _send_message
    self._handle_error(response)
  File "/var/jenkins/2/workspace/flame-kk.ui.adhoc/.env/local/lib/python2.7/site-packages/marionette_driver-0.5-py2.7.egg/marionette_driver/marionette.py", line 752, in _handle_error
    raise errors.lookup(status)(message, stacktrace=stacktrace)
NoSuchWindowException: NoSuchWindowException: None

	Martijn Wargers (dead) Assignee
	Comment 13 • 9 years ago

http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/functional/messages/test_sms_with_attachments.py#33
http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/functional/messages/test_sms_with_picture_attached.py#54

	Martijn Wargers (dead) Assignee
	Comment 15 • 9 years ago

Comment on attachment 8607782 [details] [review]
[gaia] mwargers:1109213 > mozilla-b2g:master

Sometimes it was failing on wait_for_loading_spinner_displayed, because apparently, the camera spinner was already hidden.
I think the wait_for_loading_spinner_displayed/wait_for_loading_spinner_hidden calls can be safely hidden, because camera.wait_for_capture_ready() does make sure that everything of the camera is setup already, before continuing.

After that, I get the NoSuchWindowException which is what this bug seems to be about. I can fix that which a similar workaround as I used in bug 1161088.

	Martijn Wargers (dead) Assignee
	Comment 16 • 9 years ago

I guess I should file a new bug about the fact this exception is happening, if this workaround gets approved and checked in.

	Johan Lorenzo [:jlorenzo]
	Comment 17 • 9 years ago

Comment on attachment 8607782 [details] [review]
[gaia] mwargers:1109213 > mozilla-b2g:master

Looks good to me!

	No-Jun Park [:njpark]
	Comment 18 • 9 years ago

Comment on attachment 8607782 [details] [review]
[gaia] mwargers:1109213 > mozilla-b2g:master

put a minor comment there, but looks good to me. yeah I think they recently (post 2.1) made some changes with how the spinner was being displayed on the Camera app so this is expected.

	Martijn Wargers (dead) Assignee
	Comment 19 • 9 years ago

Addressed that minor comment.
https://github.com/mozilla-b2g/gaia/commit/9304443f3ebdcb15528511a020882f02234093c9