Armen [:armenzg] Reporter
	Description • 9 years ago

According to this log [1] we believe that t-xp32-ix-144 has its clock out of sync.

The master reported that the step started at this time:
> 2015-10-08 10:26:28.344254
yet the Mozharness timestamps indicate that the slave has a different time:
> MultiFileLogger online at 20151008 09:25:20

Could you please confirm this for me? Could you please also take a screenshot of the timezone settings and attach it to this bug?

It could be daylight savings but there is a difference of a minute over an hour.

jmaher, philor: If there were machines in our pool that have bad times, could this cause intermittent test failures or talos issues?

I want to make sure this has no effect in our testing.

[1] http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-beta-win32-pgo/1444309954/mozilla-beta_xp-ix_test-chromez-pgo-bm110-tests1-windows-build62.txt.gz

	Joel Maher ( :jmaher ) (UTC -8)
	Comment 1 • 9 years ago

I wouldn't think an incorrect time[zone] would cause talos results to be off, but I wouldn't be surprised by anything.  Also some SSL tests might have issues.  With everything being localhost, I am not sure timezones are of a concern as much, although some tests fail whenever Daylight Savings Time takes effect.

	Phil Ringnalda (:philor)
	Comment 2 • 9 years ago

Tests will fail if you get the year wrong enough that certs are either expired or not yet valid, and back before build slaves were consistently using ntp we would have suites fail because the test slave believed it was being expected to extract a build from the future, but in general if you have individual tests failing from a small difference, that's a sign of misbehavior on their part, because they shouldn't have any external way of knowing what the actual time is.

But, why assume from the mismatch that it is the slave which is wrong, rather than the master? http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/fx-team-win32-pgo/1444728615/fx-team_xp-ix_test_pgo-mochitest-browser-chrome-2-bm111-tests1-windows-build266.txt.gz is t-xp32-ix-103, also on bm111, showing the same 61 minute skew.

	Kyle Lahnakoski [:ekyle]
	Comment 3 • 9 years ago

(In reply to Phil Ringnalda (:philor) from comment #2)
> http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/fx-team-
> win32-pgo/1444728615/fx-team_xp-ix_test_pgo-mochitest-browser-chrome-2-bm111-
> tests1-windows-build266.txt.gz is t-xp32-ix-103, also on bm111, showing the
> same 61 minute skew.

In an effort to standardize the times from the machines, I convert to GMT, which means I need to guess the timezone.   A skew of over 60minutes would explain some the of the non-conventional timezones I have seen.

	Armen [:armenzg] Reporter
	Comment 4 • 9 years ago

Kyle, would you say this is a blocker for you? (bug 1212500)
What features of your modeling would not work?

	Kyle Lahnakoski [:ekyle]
	Comment 5 • 9 years ago

This is not a blocker.  The ETL has does a reasonable job of mitigating clock skew.  The problem will appear as over/under reporting of time in the last mozharness step.  

I will add `clock_skew` to the ETL so we can track them.

	Vlad Ciobancai [:vladC] [:vciobancai]
	Comment 6 • 9 years ago

(In reply to Armen Zambrano Gasparnian [:armenzg] from comment #0)
> According to this log [1] we believe that t-xp32-ix-144 has its clock out of
> sync.
> 
> The master reported that the step started at this time:
> > 2015-10-08 10:26:28.344254
> yet the Mozharness timestamps indicate that the slave has a different time:
> > MultiFileLogger online at 20151008 09:25:20
> 
> Could you please confirm this for me? Could you please also take a
> screenshot of the timezone settings and attach it to this bug?
> 
> It could be daylight savings but there is a difference of a minute over an
> hour.
> 
> jmaher, philor: If there were machines in our pool that have bad times,
> could this cause intermittent test failures or talos issues?
> 
> I want to make sure this has no effect in our testing.
> 
> [1]
> http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-beta-
> win32-pgo/1444309954/mozilla-beta_xp-ix_test-chromez-pgo-bm110-tests1-
> windows-build62.txt.gz

Attached you can find the date and time settings for this slave

	Chris Cooper [:coop] (he/him)
	Comment 7 • 9 years ago

(In reply to Vlad Ciobancai [:vladC] from comment #6)
> Attached you can find the date and time settings for this slave

Wow, is that really what the desktop looks like on XP slaves after/during a test? Don't/shouldn't we try to stick the generated files into a subfolder of the desktop?

	Joel Maher ( :jmaher ) (UTC -8)
	Comment 8 • 9 years ago

pretty ugly looking- I thought we had scripts to clean these machines up on reboot?

	Armen [:armenzg] Reporter
	Comment 9 • 9 years ago

It cleans other things.
IMHO it should be on the harnesses' duties to keep track of what artifacts it generates.

	Chris Cooper [:coop] (he/him)
	Comment 10 • 9 years ago

Let's move the cleanup issue to another bug if we're concerned about it. The time on this particular machine seems OK.