James Lal [:lightsofapollo] Reporter
	Description • 9 years ago

Last week we (armen/jlund/jlal) where discussing mozharness changes the possibility of moving it into the tree has come up a few times... I think for _gecko_ there are very clear wins here this makes it way easier to hack on mozharness scripts which effect builds/tests and now that we have begun the pinning process I see this only as an improvement... The "but" here is mozharness does much more then build/test (vcs sync / bumper bot / etc...) which don't cleanly integrate into gecko here the goal of this bug is to figure out a path forward for all of the above (gecko work / vcs work).

There are some technical details which should be addressed as well:

Q: "how do I get mozharness if it gets the tree form me"
A: We can fetch subfolders of mozharness as well as cache specific version of mozharness in TC decision tasks (and in other infra as well)

Q: How do I use this for my server side (bumper bot?) infra if we now live in tree (mozharness is no longer easily clonable)
A: Distribute change via pypi
A: Use the caches we generate per commit(ish)
A: Extract the "core" of mozharness into a standalone package which only contains the core framework pieces so both the server side/gecko side can exist happily

(more)

	Justin Wood (:Callek)
	Comment 1 • 9 years ago

The fact that we use mozharness to clone the tree in many cases, and that it has independant testing, and is ease-of-use for a small self-contained "I know what I'm touching" repo, and contributor interest with things like github, make me think this is a horrid idea.

I'm happy to debate pros/cons though. And possibly attend a session at whistler.

	James Lal [:lightsofapollo] Reporter
	Comment 2 • 9 years ago

Justin- Can you outline what would be harder for you here? The ease of being able to change mozharness -> push to try -> see results -> repeat seems a very clear win to me for anything related to gecko or it's related trees. Running this purely locally will certainly be different but not sure how much impact that will have on overall usage.

	Gregory Szorc [:gps]
	Comment 3 • 9 years ago

+1 on the proposal.

I frequently find myself wanting to change stuff inside the Firefox build and test systems *and* in automation and having separate repos where changes need to be coordinated adds a lot of overhead to that process. It is frequently enough overhead that I don't make the change because I don't want to deal with it. Having mozharness in tree also increases discovery and visibility of mozharness. It will make it much easier for mach commands to run mozharness and for mozharness to run mach commands. This will result in convergence of behavior between "local client builds" and "what automation does." This is a very good thing.

	Chris AtLee [:catlee]
	Comment 4 • 9 years ago

+1 as well.

For "external" tools that deal with 0 or more than one repositories (e.g. b2g bumper, vcs sync, etc.), we can take snapshots from in-tree mozharness as required. These tools are the minority of uses of mozharness.

Most of the mozharness tools are for dealing with the tree itself, e.g. build & test automation scripts.

So what's left to solve is the bootstrapping issue - how do we make buildbot get mozharness from the tree efficiently?

	Armen [:armenzg]
	Comment 5 • 9 years ago

For build job seems easy since non-mozharness code paths checkout the gecko tree.

For test jobs, there was the chance of grabbing a tar archive directly from hgweb, however, gps threatened our lives so we put that idea down.

I had the idea of a script that fetches the test.zip, unzips only the mozharness pieces contained inside of it and run the mozharness scripts (e.g. python scripts/desktop.py original parameters + --test-installer /path/to/test.zip or similar).

I thought of this a while ago so my thoughts might be blurred.

Once we can do this:
* add mozharness from in-tree into the tests.zip
* create a script that fetches the test.zip
* extract partially
* run mozharness with local test.zip

I believe we would have most needed pieces.

	Armen [:armenzg]
	Comment 6 • 9 years ago

FYI, we have to remember that we would loose the ability of testing mozharness changes without having to push again.

Right now we can simply re-trigger after having push to a mozharness _user_ repo.

Anyways, just stating it for the record.

	Jordan Lund (:jlund)
	Comment 7 • 9 years ago

I had a meeting with Armen and gps about some things I could try.

I hope to start tackling this in March.

	Hal Wine [:hwine] use NI!
	Comment 8 • 9 years ago

(In reply to Chris AtLee [:catlee] from comment #4)
> For "external" tools that deal with 0 or more than one repositories (e.g.
> b2g bumper, vcs sync, etc.), we can take snapshots from in-tree mozharness
> as required. These tools are the minority of uses of mozharness.

I'm not sure the snapshot approach would work well - at least not for vcs-sync. vcs-sync relies both on the code of mozharness (stable - snapshots would work fine), but also the configs (volitile) kept in the current repo. Making commits to m-c to reflect changes in bugzilla repo mirroring needs seems "messy", and there's a question of where the updated pinned version is kept.

My initial thought for vcs-sync is to simply fork (or inherit) the existing mozharness repository.

	Pete Moore [:pmoore][:pete]
	Comment 9 • 9 years ago

Perhaps radical: use mozilla-central in-tree mozharness version (no snapshots) for b2g bumper / vcs sync, *including* having the configs in-tree at the hands of devs, but:

  * report-on-change - i.e. increase transparency of config changes and/or lock down access to configs in hg (if hg can manage commit permissions per subfolder)
  * have a reliable suite of unit tests for both vcs sync and b2g bumper that not only test code, but validate sanity of configs

With this:
  * Developers are empowered to make changes, also to repo mirroring
  * Sheriffs are empowered to roll back changes
  * Changes do not need to by sync'd across to a snapshot / change process is simple
  * Transparency of changes for all
  * Bad changes automatically detected and highlighted in tests

We'd need to make it clear in the code comments that the production version is taken from mozilla-central, and that the version in other trees is irrelevant.

	Jordan Lund (:jlund)
	Comment 10 • 9 years ago

(In reply to Pete Moore [:pmoore][:pete] from comment #9)
> Perhaps radical: use mozilla-central in-tree mozharness version (no
> snapshots) for b2g bumper / vcs sync, *including* having the configs in-tree
> at the hands of devs
> 

I like this approach. are there major objections? I would like to have one copy where the source comes from. I think in the future, we will break mozharness into components that are installable via pip and this is what our services will use but until then, maybe we can continue to have one source of truth...

	Justin Wood (:Callek)
	Comment 11 • 8 years ago

jordan, this bug is resolvable, correct?

Any of its deps resolveable?

	Jordan Lund (:jlund)
	Comment 12 • 8 years ago

cleaned up a bunch of deps. this is basically done