Closed Bug 1316567 Opened 8 years ago Closed 7 years ago

[tier-3] Intermittent test_direct_update.py TestDirectUpdate.test_update, test_fallback_update.py TestFallbackUpdate.test_update | AssertionError: Available update has been found

Categories

(Testing :: Firefox UI Tests, defect)

Product:
Testing
Component:
Firefox UI Tests
Version:
Version 3
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox51 affected)

Status:
VERIFIED INCOMPLETE
Tracking Flags:
Tracking Status
firefox51 --- affected

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: bulk-close-intermittents, intermittent-failure, Whiteboard: [stockwell unknown])

status-firefox51: --- → affected
Summary: Intermittent test_direct_update.py TestDirectUpdate.test_update | AssertionError: Available update has been found → Intermittent test_direct_update.py TestDirectUpdate.test_update, test_fallback_update.py TestFallbackUpdate.test_update | AssertionError: Available update has been found
Did we turn off updates for Nightly builds over the weekend? I can see lots of failures on Windows and Linux.
Flags: needinfo?(rail)
Flags: needinfo?(bugspam.Callek)
AFAIK, there was an issue in one of the nightly builds on Friday and recall that someone disabled the updates. I also have vague memory, that someone was going to send an announcement to one of the lists.
Flags: needinfo?(rail)
(In reply to Rail Aliiev [:rail] ⌚️ET from comment #10)
> AFAIK, there was an issue in one of the nightly builds on Friday and recall
> that someone disabled the updates. I also have vague memory, that someone
> was going to send an announcement to one of the lists.

Yes, RyanVM did the announce, but also said it was enabled, so I *think* its all set.

https://groups.google.com/d/topic/mozilla.dev.platform/2_9L32K32h8/discussion
Flags: needinfo?(bugspam.Callek)
can I assume that all these failures were related to a misconfigured nightly job this weekend?  the failures seem to have stopped!
Whiteboard: [stockwell fixed]
Thanks. That would explain it. And yes, we all all back to normal for nightly builds off mozilla-central.
With yesterdays tests we have a spike of this failure for mozilla-aurora. When I checked mozmill-ci I noticed that it was missed to change the channel from aurora to auroratest. I made this change for now.
I caught an interesting case for direct updates on mozilla-beta:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=a6edfa473c7d&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta(&selectedJob=101240456

Here we clearly get a valid snippet but the AUS service tells us that there is no update available:

https://treeherder.mozilla.org/logviewer.html#?job_id=101240456&repo=mozilla-beta&lineNumber=2697
https://treeherder.mozilla.org/logviewer.html#?job_id=101240456&repo=mozilla-beta&lineNumber=7096

08:16:00     INFO -  'build_pre': {'buildid': u'20170504103226',
08:16:00     INFO -                'channel': u'beta',
08:16:00     INFO -                'disabled_addons': None,
08:16:00     INFO -                'locale': u'pl',
08:16:00     INFO -                'mar_channels': set(['firefox-mozilla-beta',
08:16:00     INFO -                                     'firefox-mozilla-release']),
08:16:00     INFO -                'update_snippet': '<?xml version="1.0"?>\n<updates>\n    <update type="minor" displayVersion="54.0 Beta 10" appVersion="54.0" platformVersion="54.0" buildID="20170522172523" detailsURL="https://www.mozilla.org/pl/firefox/54.0/releasenotes/">\n        <patch type="complete" URL="http://download.mozilla.org/?product=firefox-54.0b10-complete&amp;os=win&amp;lang=pl&amp;force=1" hashFunction="sha512" hashValue="10b826b52bb3c8a698cee43c743189a4ea5ae16b626d68c3706c520fa2ca5706d3e34e63ddb7380fa9afb8779e2bb72e1137e0a5d62d2aaa1ebd9605ccb1a3b5" size="54414805"/>\n    </update>\n</updates>',
08:16:00     INFO -                'update_url': u'https://aus5.mozilla.org/update/6/Firefox/54.0/20170504103226/WINNT_x86-msvc-x86/pl/beta/Windows_NT%206.3.0.0%20(x86)(noBug1296630v1)(nowebsense)/SSE3/default/default/update.xml?force=1',
08:16:00     INFO -                'user_agent': u'Mozilla/5.0 (Windows NT 6.3; rv:54.0) Gecko/20100101 Firefox/54.0',
08:16:00     INFO -                'version': u'54.0'},
08:16:00     INFO -  'fallback': False,

The update available status gets retrieved by checking the currently selected deck of the about window:

https://dxr.mozilla.org/mozilla-central/source/testing/firefox-ui/harness/firefox_ui_harness/testcases.py#133

Checking the logs in more detail I can see the following failure of the application updater:

08:15:59     INFO -  *** AUS:SVC Checker:onError - request.status: 2152398890
08:15:59     INFO -  *** AUS:SVC getStatusTextFromCode - transfer error: Serwer proxy nie został odnaleziony (należy sprawdzić połączenie z Internetem), code: 2152398890

It looks like there was a problem with the network, and the proxy server has not been found.

So we should take failures into account for the above update available check. And if there is a failure we have to bail out.

I will file a new bug for this.
on mozilla-beta this test had 22 failures yesterday alone, this seems to be a new issue given the fact that we only had 3 failures in the previous week.

:whimboo- can you take a look at this?
Flags: needinfo?(hskupin)
Whiteboard: [stockwell fixed] → [stockwell needswork]
(In reply to Joel Maher ( :jmaher) from comment #26)
> on mozilla-beta this test had 22 failures yesterday alone, this seems to be
> a new issue given the fact that we only had 3 failures in the previous week.
> 
> :whimboo- can you take a look at this?

These failures were due to incorrect setup of the beta channel (55b3 was available instead of 55b4). They should be ignored (I've also informed people to mark such failure as "expected fail").
Note that I've also removed the associations to this bug and have reclassified all corresponding failing jobs as "expected fail" due to the incorrect setup of the channel.
Flags: needinfo?(hskupin)
Summary: Intermittent test_direct_update.py TestDirectUpdate.test_update, test_fallback_update.py TestFallbackUpdate.test_update | AssertionError: Available update has been found → [tier-3] Intermittent test_direct_update.py TestDirectUpdate.test_update, test_fallback_update.py TestFallbackUpdate.test_update | AssertionError: Available update has been found
this appears fixed, do we know why this isn't failing anymore?
Whiteboard: [stockwell needswork] → [stockwell unknown]
This is a kinda meta bug for every issue which could cause updates not to be found. If it's unclear people star it against this bug, without proper investigation. :/
Please also see bug 1367319 in the dependency list, which should give a better error. But given the unclear future of the update tests I don't change anything right now.
Status: NEW → RESOLVED
Closed: 7 years ago
Keywords: bulk-close-intermittents
Resolution: --- → INCOMPLETE
Failures from comment 38 are not associated with this bug - those showed up on ESR 52.4.1, but are due to the fact that the build was offered as update only on Mac OS X - so Win & Linux failures are expected (OS X failures were due to some misconfiguration by releng).

I've removed classification in treeherder and re-classified as "expected fail".
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.