Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Description • 8 years ago

10:17:13     INFO - Traceback (most recent call last):
 10:17:13     INFO -   File "/Users/mozauto/jenkins/workspace/mozilla-central_update/build/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 344, in run
 10:17:13     INFO -     testMethod()
 10:17:13     INFO -   File "/Users/mozauto/jenkins/workspace/mozilla-central_update/build/venv/lib/python2.7/site-packages/firefox_ui_tests/update/direct/test_direct_update.py", line 21, in test_update
 10:17:13     INFO -     self.check_update_applied()
 10:17:13     INFO -   File "/Users/mozauto/jenkins/workspace/mozilla-central_update/build/venv/lib/python2.7/site-packages/firefox_puppeteer/testcases/update.py", line 159, in check_update_applied
 10:17:13     INFO -     self.updates[self.current_update_index].type)

Marionette also has thrown an error in takeScreenshot. Maybe its related. I will file that as a separate bug and will mark it as dependency.

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 3 • 8 years ago

This is failing a lot on the lastet beta update tests for Firefox 48.0:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=d142c49033c015f67272562b37dbe2912cfc7f14&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest&selectedJob=1302898

I will check today what's causing that.

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 5 • 8 years ago

This problem seems to only occur for watershed releases when another update is in the queue. The following check will always fail because `type` is part of the `patch` sub-property:

https://dxr.mozilla.org/mozilla-central/source/testing/firefox-ui/harness/firefox_ui_harness/testcases.py?q=file%3Atestcases.py&redirect_type=single#167

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 6 • 8 years ago

My watershed suspicion was correct. In the following example we started with a 46.0 release and tried to upgrade to latest beta. But we ended up with 48.0b1:

https://aus5.mozilla.org/update/3/Firefox/46.0/20160411042519/WINNT_x86-msvc/en-US/beta-cdntest/Windows_NT%206.1/default/default/update.xml?force=1

Michelle, you should make sure not to include releases before the watershed release for testing on the given platform. Even with this bug fixed, the tests will fail - but then with a more helpful message.

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 7 • 8 years ago

Review commit: https://reviewboard.mozilla.org/r/63048/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/63048/

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 8 • 8 years ago

https://reviewboard.mozilla.org/r/63048/#review59976

::: testing/firefox-ui/harness/firefox_ui_harness/testcases.py:208
(Diff revision 1)
> +                                    'No further update of the same type gets offered: '
> +                                    '{0} != {1}'.format(
> +                                        self.software_update.active_update.type,
> +                                        update['patch']['type']
> +                                    ))
> +

I moved the whole block down here because it would be more important to do the version and buildid checks first before spitting out an update type failure, which is lesser understandable as why we upgraded to a specific version.

	Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)
	Comment 9 • 8 years ago

Comment on attachment 8769078 [details]
Bug 1247596 - Fix reference to the update type which is part of the patch.

https://reviewboard.mozilla.org/r/63048/#review59996

	Pulsebot
	Comment 10 • 8 years ago

Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/abb40d3751b5
Fix reference to the update type which is part of the patch. r=maja_zf

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 11 • 8 years ago

Maja, could you help getting this patch backported down to beta if all goes well by next week? It would just be adding the whiteboard entries. Thanks

	Phil Ringnalda (:philor)
	Comment 12 • 8 years ago
bugherder

https://hg.mozilla.org/mozilla-central/rev/abb40d3751b5

	Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)
	Comment 14 • 8 years ago

This seems to be behaving well. As per Henrik's request, adding checkin-needed tags

	Carsten Book [:Tomcat]
	Comment 16 • 8 years ago
bugherder uplift

https://hg.mozilla.org/releases/mozilla-aurora/rev/40ae0b4863fd

	Carsten Book [:Tomcat]
	Comment 17 • 8 years ago
bugherder uplift

https://hg.mozilla.org/releases/mozilla-beta/rev/7352ca17d1b7

	[SV Manager] Florin Mezei, QA (:FlorinMezei)
	Comment 19 • 8 years ago

(In reply to Henrik Skupin (:whimboo) from comment #6)
> Michelle, you should make sure not to include releases before the watershed
> release for testing on the given platform. Even with this bug fixed, the
> tests will fail - but then with a more helpful message.

Unfortunately we are mostly unaware when new watershed updates are setup. I've brought this up to Sylvestre so we can find a solution to get notified whenever new watersheds are setup, as it's not the first time that we find out about a new watershed after it was deployed.

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 20 • 8 years ago

Sounds great. Also the new failure message will clearly indicate now that Firefox has not been updated to the expected target build version but a former one.

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 22 • 8 years ago

(In reply to OrangeFactor Robot from comment #21)
> 53 automation job failures were associated with this bug yesterday.
> 
> Repository breakdown:
> * mozilla-release: 53

Those latest test failures as listed by Orange Factor should not have been starred against this bug, but a new one filed. Given that this hasn't been done yesterday I will do it now.