Nick Thomas [:nthomas] (UTC+12) Reporter
	Description • 9 years ago

Bug 1041763 did this originally, then bug 1130176 undid it (we failed to capture that anywhere in cloud-tools). Those masters are struggling a bit so lets upgrade again, it's only a couple of dollars per master per day.

	Nick Thomas [:nthomas] (UTC+12) Reporter
	Comment 1 • 9 years ago

Done:
bm117-tests1-linux64

To do:
bm51-tests1-linux64
bm52-tests1-linux64
bm53-tests1-linux64
bm54-tests1-linux64
bm67-tests1-linux64
bm68-tests1-linux64
bm113-tests1-linux64
bm114-tests1-linux64
bm115-tests1-linux64
bm116-tests1-linux64
bm118-tests1-linux64
bm120-tests1-linux64
bm121-tests1-linux64
bm122-tests1-linux64
bm123-tests1-linux64

	Amy Rich [:arr] [:arich]
	Comment 2 • 9 years ago

We should do bug 1135664 at the same time.

	Dustin J. Mitchell [:dustin] (he/him)
	Comment 3 • 9 years ago

Maybe we should think about a way to capture instance configurations somewhere.  CloudFormation can do this, for example.  This is the fourth? fifth? thing that cloud-tools didn't get right out of the box for the masters, or for the log-aggregation servers.  We're burning tens of man-hours fixing this stuff.

FWIW, I did check on the correct instance type:

(2015/02/09, US/Eastern)
[11:52:53] <dustin> rail: hey, looks like teh masters are a mix of m3.medium, m1.large, and a few m3.large
[11:52:57] <dustin> should I use any one of those in particular?
[11:55:00] <rail> yeah, would be great to use the same family
[11:55:15] <rail> not m1
[11:55:57] <dustin> ok
[11:56:00] <dustin> I'm doing m3.medium
[11:56:08] <dustin> easy enough to bump that to m3.large when necessary, right?
[11:56:28] <rail> yeah
[11:56:30] <dustin> kk

that was based on the majority, but not all, of the existing masters being m3.medium at that time.  So I suspect bug 1041763's effects were already wearing off at that time :)

	Dustin J. Mitchell [:dustin] (he/him)
	Comment 4 • 9 years ago

Nick, if you are gracefulling and stopping those hosts, please run the python script in bug 1135664 (with the code to reset the user-data uncommented) while they're stopped, and verify that they get their swap set up correctly on startup.

That bug is also waiting on a TCW, so if you want to merge the two and do the work in the TCW, that's cool too (it'd be a simple addition to the script)

	Nick Thomas [:nthomas] (UTC+12) Reporter
	Comment 5 • 9 years ago

I'm likely to leave this for buildduty or the TCW, but I'll try to remember if I change my mind on Sunday.

	Jordan Lund (:jlund)
	Comment 6 • 9 years ago

(In reply to Nick Thomas [:nthomas] from comment #1)

chewed away at a few of these today


done:
> bm51-tests1-linux64
> bm52-tests1-linux64
> bm53-tests1-linux64
> bm54-tests1-linux64
> bm67-tests1-linux64
> bm68-tests1-linux64
> bm113-tests1-linux64
> bm115-tests1-linux64


still todo:
> bm114-tests1-linux64
> bm116-tests1-linux64
> bm118-tests1-linux64
> bm120-tests1-linux64
> bm121-tests1-linux64
> bm122-tests1-linux64
> bm123-tests1-linux64

	Jordan Lund (:jlund)
	Comment 7 • 9 years ago

> still todo:
> > bm114-tests1-linux64
> > bm116-tests1-linux64
> > bm118-tests1-linux64
> > bm120-tests1-linux64
> > bm121-tests1-linux64
> > bm122-tests1-linux64
> > bm123-tests1-linux64

these were finished today

	Nick Thomas [:nthomas] (UTC+12) Reporter
	Comment 8 • 9 years ago

\o/

	Dustin J. Mitchell [:dustin] (he/him)
	Comment 9 • 9 years ago

As Jordan noticed with bm67, configs/buildbot-master still says "m3.medium"

https://github.com/mozilla/build-cloud-tools/pull/52