Henrik Skupin [:whimboo][⌚️UTC+1] Reporter
	Description • 6 years ago

Since bug 1403923 has been landed the additional shutdown time of Firefox for each custom session can cause new timeout failures for tests, which haven't been seen before or happened rarely randomly because times were close to 25s already.

The question is if due to that change the default timeout should be increased so that tests won't need the meta tag for longer timeouts.

Right now the timeouts in use are 25s and 180s.

As it looks like more tests than expected are starting to fail. Until we have made a decision I will add all known bugs as dependencies.

Andreas, and James what do you think? What would be good default value?

	Andreas Tolfsen ❲:ato❳
	Comment 1 • 6 years ago

I think it probably makes sense to increase the default timeout,
considering there was a bug in geckodriver that it didn’t shut
down Firefox correctly.  Now that it does, tests that request a
lot of new sessions (new Firefox instances) are particularly badly
affected.

We set the default timeout under the expectation that geckodriver
behaved correctly, so if the teardown of a geckodriver session (and
consequently a Firefox instance) is now taking longer we should
adapt our world view.

	David Burns :automatedtester
	Comment 2 • 6 years ago

My first concern is if these extended timeouts are going to hide real issues. Yesterday in the marionette meeting, and on IRC to jmaher, you found a perf regression in shutdown because of these timeouts. We might miss this in the future.

	Henrik Skupin [:whimboo][⌚️UTC+1] Reporter
	Comment 3 • 6 years ago

I don't think that the wdspec tests are a good measure for possible perf regressions. At least not in the current state how we handle the timeouts. There are probably better harnesses like Talos which would have also covered this perf regression, but a bit later.

	James Graham [:jgraham]
	Comment 4 • 6 years ago

I don't have any way of answering this without just doing the work to see what makes the tests stable. Increasing the multiplier to 3x by default and using try to see if that fixes the problem seems like a good idea. Also splitting tests that start new sessions into their own file seems like a good idea.

	Henrik Skupin [:whimboo][⌚️UTC+1] Reporter
	Comment 5 • 6 years ago

As it looks like most of the tests which are causing a timeout are related to any kind of alert testing, and as such I have a suspicion that it might be influenced in some way by bug 1425559. Maybe best would be to wait for a fix for debug builds, and assume that opt builds also see an increased shutdown performance in such situations.

	Henrik Skupin [:whimboo][⌚️UTC+1] Reporter
	Comment 6 • 6 years ago

With bug 1425559 fixed we are still seeing timeout due to long shutdown times of Firefox opt builds. As I heard from :baku more bugs will be filed because there are potentially more infinite loops in spinning the event loop, while missing to handle the shutdown observer. I will keep an eye on those bugs.

That said we should post-pone any decision on this bug for now.

	Andreas Tolfsen ❲:ato❳
	Comment 7 • 6 years ago

This is still occasionally an issue, but we will file a new bug if
it reappears.