Closed Bug 967014 Opened 10 years ago Closed 10 years ago

The clock of some Fedora machines is out of sync

Categories

(Infrastructure & Operations Graveyard :: CIDuty, task, P2)

x86_64
Linux

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: armenzg, Assigned: coop)

References

Details

Rev3 Fedora 12 try debug test mochitest-browser-chrome job:
<jmaher> armenzg: I have noticed on a try run that a certificate is expired and it indicates the date on the machine is 2001: https://tbpl.mozilla.org/php/getParsedLog.php?id=34010820&tree=Try&full=1
<jmaher> search for 'expired'
<jmaher> armenzg: I see this on a lot of the fed-xxx machines :(

[cltbld@talos-r3-fed-079 ~]$ date 
Wed Mar 14 20:53:16 PST 2001

20:15:25     INFO -  TEST-INFO | chrome://mochitests/content/browser/browser/devtools/styleeditor/test/browser_styleeditor_enabled.js | Console message: [JavaScript Error: "example.com:443 uses an invalid security certificate.
20:15:25     INFO -  The certificate is not trusted because the issuer certificate has expired.
20:15:25     INFO -  The certificate will not be valid until 06/19/2012 08:26 AM. The current time is 03/14/2001 08:15 PM.
20:15:25     INFO -  (Error code: sec_error_expired_issuer_certificate)
20:15:25     INFO -  "]

TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/styleeditor/test/browser_styleeditor_enabled.js | Test timed out
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/styleeditor/test/browser_styleeditor_reload.js | Test timed out
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/styleeditor/test/browser_styleeditor_selectstylesheet.js | Test timed out
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/styleeditor/test/browser_styleeditor_sourcemaps.js | Test timed out
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/xpinstall/browser_httphash.js | 1 Add-on should have been successfully installed - Got 0, expected 1
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/xpinstall/browser_httphash2.js | Download should fail - Got -1, expected -2
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/xpinstall/browser_httphash3.js | 1 Add-on should have been successfully installed - Got 0, expected 1
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/xpinstall/browser_httphash5.js | 1 Add-on should have been successfully installed - Got 0, expected 1
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/xpinstall/browser_httphash6.js | Exception thrown - [Exception... "Failure"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/xpinstall/browser_httphash6.js :: setup_redirect :: line 12"  data: no]
Return code: 1
SSLError: [Errno 1] _ssl.c:480: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
dustin, if I want to update the clock of all Fedora machines, would this be the right way of doing so?
Using ntpdate pool.ntp.org

[root@talos-r3-fed-079 ~]# date
Wed Mar 14 21:09:22 PST 2001
[root@talos-r3-fed-079 ~]# ntpdate pool.ntp.org
 3 Feb 08:20:50 ntpdate[2297]: step time server 69.41.163.31 offset 406811483.828311 sec
You have mail in /var/spool/mail/root
[root@talos-r3-fed-079 ~]# date
Mon Feb  3 08:20:56 PST 2014
Flags: needinfo?(dustin)
Yes, that works.  It's not the best command on newer OS's, but Fedora 8 is old.
Flags: needinfo?(dustin)
Thank you dustin!
I updated all machines but I would not surprise if I missed any.

Please let us know if the issue is fixed or other machines have an old time.

jmaher, we can have this happen in the future as we don't have puppet to keep their clocks in sync.
Assignee: nobody → armenzg
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Yes, please fix it more permanently: everything in bug 968597 except c9 and c41 is actually because of this - Fedora slaves, including 079 that was your comment 0 example yet again, are running in 2001, and the SSL cert that a couple hundred browser-chrome tests depend on wasn't valid in 2001 so they fail. Slowly.
Blocks: 968597
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Disabled talos-r3-fed-069, talos-r3-fed-079, talos-r3-fed-080, talos-r3-fed-102 for being clearly currently in 2001.
For the sake of simplicity, could I just add |ntpdate pool.ntp.org| to /etc/rc.local on all the r3-fed machines? These machines still reboot frequently, so this would run on every boot.
Probably time.mozilla.org would be a better bet, but yes, that sounds like a good solution.
Assignee: armenzg → coop
Priority: -- → P2
All fed and fed64 slaves have /etc/rc.local updated now except for:

talos-r3-fed-024
talos-r3-fed-102
Dustin, is ntp.build.mozilla.org deprecated now ? I see that's the scl1 admin nodes.
Depends on: 982008, 981998
Ah, I had forgotten all about that.  Given that it's going away, yes, it shouldn't be used.  I'll add it to our migration list (we can just change the CNAME)
(In reply to Chris Cooper [:coop] [away until April 3] from comment #8)
> All fed and fed64 slaves have /etc/rc.local updated now except for:
> 
> talos-r3-fed-024
> talos-r3-fed-102

I got these two now they're available.
Status: REOPENED → RESOLVED
Closed: 10 years ago10 years ago
Resolution: --- → FIXED
Component: Platform Support → Buildduty
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.