bhearsum@mozilla.com (:bhearsum) Assignee
	Description • 9 years ago

There's going to be quite a bit of buildbot bridge work upcoming, and having a dev environment is going to be pretty important to avoiding breaking production. Catlee and I talked about this a few months back, and the most useful way of doing it seems to be using a separate instance that's locked down to one branch, but still using production buildbot. We've been using a similar model for release runner, and it seems to be working well.

	bhearsum@mozilla.com (:bhearsum) Assignee
	Comment 1 • 9 years ago

This is modeled after the release runner dev work I did a few months ago, which has proven to work well. I gave it a try in --noop mode against an existing production deployment and the only diff is in allowed_builders, which is expected (we don't want prod to take alder jobs anymore, because that's where dev will be pointed).

This can't be deployed until the new database is set-up, and I insert all of dev secrets into Hiera (they're currently set to TODO).

	Dustin J. Mitchell [:dustin] (he/him)
	Comment 2 • 9 years ago

Comment on attachment 8653549 [details] [diff] [review]
support dev and prod in bbb puppet configs

Review of attachment 8653549 [details] [diff] [review]:
-----------------------------------------------------------------

::: manifests/moco-config.pp
@@ +339,5 @@
>      $buildbot_bridge_reflector_interval = 60
>  
> +    $buildbot_bridge_env_config = {
> +        "dev" => {
> +            # >= means that the latest version will get installed

Actually means that the latest version will be reinstalled on every puppet run (since puppet looks at the version installed and compares it to the foo==v.v specified in the packages list).  If you're OK with that for dev, then there's no great harm, but best to memorialize it in a comment all the same.

::: modules/buildbot_bridge/manifests/init.pp
@@ +8,5 @@
>      include packages::mozilla::python27
>      include packages::mysql_devel
>      include users::builder
>  
> +    $env_config = $config::buildbot_bridge_env_config[$buildbot_bridge_env]

Couldn't this be $::buildbot_bridge::settings::env_config?

	bhearsum@mozilla.com (:bhearsum) Assignee
	Comment 3 • 9 years ago

(In reply to Dustin J. Mitchell [:dustin] from comment #2)
> Comment on attachment 8653549 [details] [diff] [review]
> support dev and prod in bbb puppet configs
> 
> Review of attachment 8653549 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: manifests/moco-config.pp
> @@ +339,5 @@
> >      $buildbot_bridge_reflector_interval = 60
> >  
> > +    $buildbot_bridge_env_config = {
> > +        "dev" => {
> > +            # >= means that the latest version will get installed
> 
> Actually means that the latest version will be reinstalled on every puppet
> run (since puppet looks at the version installed and compares it to the
> foo==v.v specified in the packages list).  If you're OK with that for dev,
> then there's no great harm, but best to memorialize it in a comment all the
> same.

Ah, that's good to know! In that case, I'll switch this to an actual version number...I don't want to force reinstalls (and restarts of the services) every 30min. I was hoping to avoid needing to bump the version number to test dev versions, but it's not the end of the world.

> ::: modules/buildbot_bridge/manifests/init.pp
> @@ +8,5 @@
> >      include packages::mozilla::python27
> >      include packages::mysql_devel
> >      include users::builder
> >  
> > +    $env_config = $config::buildbot_bridge_env_config[$buildbot_bridge_env]
> 
> Couldn't this be $::buildbot_bridge::settings::env_config?

Good point, I'll fix this up too.

	bhearsum@mozilla.com (:bhearsum) Assignee
	Comment 4 • 9 years ago

This should fix your comments, and tested fine against a prod node.

	bhearsum@mozilla.com (:bhearsum) Assignee
	Comment 5 • 9 years ago

I landed this and it's mostly working. The only thing that's having trouble is the Pulse connection, because of bug 1199385. I'm trying to get that sorted out.

	bhearsum@mozilla.com (:bhearsum) Assignee
	Comment 6 • 9 years ago

Turns out that my allowed builders strategy doesn't work, because the Bridge cancels tasks with disallowed Builders....which means the production instance still picks up Alder jobs, but cancels them (screwing up the dev instance trying to claim them)...

	bhearsum@mozilla.com (:bhearsum) Assignee
	Comment 7 • 9 years ago

(In reply to Ben Hearsum (:bhearsum) from comment #6)
> Turns out that my allowed builders strategy doesn't work, because the Bridge
> cancels tasks with disallowed Builders....which means the production
> instance still picks up Alder jobs, but cancels them (screwing up the dev
> instance trying to claim them)...

Fixed this in bug 1201861, it seems to be working well now!