Nick Thomas [:nthomas] (UTC+12) Assignee
	Description • 10 years ago

buildbot-master66.srv.releng.usw2.mozilla.com:File Age - /builds/b2g_bumper/b2g_bumper.stamp is CRITICAL: FILE_AGE CRITICAL: /builds/b2g_bumper/b2g_bumper.stamp is 35765 seconds old and 0 bytes

And fails in nightly devices builds:
18:59:55    FATAL - Giving up on gecko git revision for 9731b0b7fa78.

	Nick Thomas [:nthomas] (UTC+12) Assignee
	Comment 1 • 10 years ago

We rebooted cruncher today to add more disk, for the first time since the oh-my-gawd-I-don't-know-how-long.
The mapper app was running but apache wasn't set to start on boot, so requests on port 80 were getting dropped.

Started httpd, and set it to run on run levels 2345.

	Jordan Lund (:jlund)
	Comment 2 • 10 years ago

Nagios is complaining again:
/builds/gaia_bumper/gaia_bumper.stamp is CRITICAL: FILE_AGE CRITICAL
/builds/b2g_bumper/b2g_bumper.stamp is CRITICAL: FILE_AGE CRITICAL: /builds/b2g_bumper/b2g_bumper.stamp

	Jordan Lund (:jlund)
	Comment 3 • 10 years ago

grep'n for httpd, I get:

(aws-ve-2)[buildduty@cruncher.srv.releng.scl3 aws]$ ps -elfy | grep httpd
S root     12922     1  0  80   0  9700 90624 poll_s Feb03 ?        00:00:03 httpd
S apache   12923 12922  0  80   0  2872 55731 poll_s Feb03 ?        00:00:01 httpd
S apache   12924 12922  0  80   0 10224 92614 inet_c Feb03 ?        00:00:00 httpd
S apache   12925 12922  0  80   0 10220 92614 inet_c Feb03 ?        00:00:00 httpd
S apache   12926 12922  0  80   0 10224 92614 inet_c Feb03 ?        00:00:00 httpd
S apache   12927 12922  0  80   0 10188 92614 inet_c Feb03 ?        00:00:00 httpd
S apache   12928 12922  0  80   0 10220 92614 inet_c Feb03 ?        00:00:00 httpd
S apache   12929 12922  0  80   0 10224 92614 inet_c Feb03 ?        00:00:00 httpd
S apache   12930 12922  0  80   0 10224 92614 inet_c Feb03 ?        00:00:00 httpd
S apache   12931 12922  0  80   0 10224 92614 inet_c Feb03 ?        00:00:00 httpd
S apache   14886 12922  0  80   0 10220 92614 inet_c Feb03 ?        00:00:00 httpd
S apache   15488 12922  0  80   0 10224 92614 inet_c Feb03 ?        00:00:00 httpd
S apache   16620 12922  0  80   0 10220 92614 inet_c Feb03 ?        00:00:00 httpd
S 2325     28587 17694  0  80   0   844 25809 pipe_w 10:20 pts/2    00:00:00 grep httpd

do we use apachectl for managing httpd?

	Jordan Lund (:jlund)
	Comment 4 • 10 years ago

false alarm. Looks like nagios thinks this was solved a few min after the alert

	Nick Thomas [:nthomas] (UTC+12) Assignee
	Comment 5 • 10 years ago

There can be several causes of the alert in comment #2, have to dig in the bumper log to see what's up.