bhearsum@mozilla.com (:bhearsum) Assignee
	Description • 9 years ago

Partner repacks are one of the last things we don't support in post_upload.py, and we need to support them to unblock the S3 migration. We should support both current release automation and release promotion.

For the former, partner builds are uploaded to "partner-repacks/$partner/$platform/$locale/$file" (relative to the candidates dir).

For the latter, it looks like they go to tinderbox-builds/$branch-$platform-partner_repack/$partner/$platform/$locale/$file.

	bhearsum@mozilla.com (:bhearsum) Assignee
	Comment 1 • 9 years ago

OK, so it turns out that the existing post_upload.py will work fine for the release case:
[ffxbld@dev-stage01 tmp]$ cd foo
[ffxbld@dev-stage01 foo]$ ls
[ffxbld@dev-stage01 foo]$ mkdir -p 38.0.6/build1/partner-repacks/yandex-ua/linux-i686/ru
[ffxbld@dev-stage01 foo]$ touch 38.0.6/build1/partner-repacks/yandex-ua/linux-i686/ru/firefox-38.0.6.tar.bz2
ffxbld@dev-stage01 stage]$ python post_upload.py -p firefox -n 1 -v 38.0.6 --release-to-candidates-dir --signed /tmp/foo/38.0.6/build1/ /tmp/foo/38.0.6/build1/partner-repacks/yandex-ua/linux-i686/ru/firefox-38.0.6.tar.bz2 
sys.argv: ['post_upload.py', '-p', 'firefox', '-n', '1', '-v', '38.0.6', '--release-to-candidates-dir', '--signed', '/tmp/foo/38.0.6/build1/', '/tmp/foo/38.0.6/build1/partner-repacks/yandex-ua/linux-i686/ru/firefox-38.0.6.tar.bz2']
mkdir /home/ftp/pub/firefox/candidates/38.0.6-candidates
ln -s /home/ftp/pub/firefox/candidates/38.0.6-candidates /home/ftp/pub/firefox/nightly/38.0.6-candidates
http://dev-stage01.srv.releng.scl3.mozilla.com/pub/mozilla.org/firefox/nightly/38.0.6-candidates/build1/partner-repacks/yandex-ua/linux-i686/ru/firefox-38.0.6.tar.bz2


But the --release-to-tinderbox-builds action doesn't seem to preserve the directory structure:
[ffxbld@dev-stage01 stage]$ python post_upload.py -p firefox --tinderbox-builds-dir mozilla-release-linux-i686-partner_repack --release-to-tinderbox-builds /tmp/foo/38.0.6/build1/ /tmp/foo/38.0.6/build1/partner-repacks/yandex-ua/linux-i686/ru/firefox-38.0.6.tar.bz2 
sys.argv: ['post_upload.py', '-p', 'firefox', '--tinderbox-builds-dir', 'mozilla-release-linux-i686-partner_repack', '--release-to-tinderbox-builds', '/tmp/foo/38.0.6/build1/', '/tmp/foo/38.0.6/build1/partner-repacks/yandex-ua/linux-i686/ru/firefox-38.0.6.tar.bz2']
http://dev-stage01.srv.releng.scl3.mozilla.com/pub/mozilla.org/firefox/tinderbox-builds/mozilla-release-linux-i686-partner_repack/firefox-38.0.6.tar.bz2


Not sure how much effort should be put into making this nice, but this seems like a simple fix:
@@ -220,17 +220,18 @@ def ReleaseToBuildDir(builds_dir, builds_url, options, upload_dir, files, dated)
         tinderboxBuildsPath = os.path.join(
             tinderboxBuildsPath, options.builddir)
         tinderboxUrl = os.path.join(tinderboxUrl, options.builddir)
 
     for f in files:
         # Reject MAR files. They don't belong here.
         if f.endswith('.mar'):
             continue
-        if options.tinderbox_builds_dir.endswith('l10n') and f.endswith('.xpi'):
+        if (options.tinderbox_builds_dir.endswith('l10n') and f.endswith('.xpi') or \
+            options.tinderbox_builds_dirs.endswith("partner_repacks"):
             CopyFileToDir(


And ends up looking like this when run - note that unlike the candidates dir command, I'm keeping the "partner-repacks" part of the path in the first positional arg, because that shouldn't be part of the path beneath the tinderbox builds dir:
[ffxbld@dev-stage01 stage]$ python post_upload.py -p firefox --tinderbox-builds-dir mozilla-release-linux-i686-partner_repack --release-to-tinderbox-builds /tmp/foo/38.0.6/build1/partner-repacks /tmp/foo/38.0.6/build1/partner-repacks/yandex-ua/linux-i686/ru/firefox-38.0.6.tar.bz2 
sys.argv: ['post_upload.py', '-p', 'firefox', '--tinderbox-builds-dir', 'mozilla-release-linux-i686-partner_repack', '--release-to-tinderbox-builds', '/tmp/foo/38.0.6/build1/partner-repacks', '/tmp/foo/38.0.6/build1/partner-repacks/yandex-ua/linux-i686/ru/firefox-38.0.6.tar.bz2']
http://dev-stage01.srv.releng.scl3.mozilla.com/pub/mozilla.org/firefox/tinderbox-builds/mozilla-release-linux-i686-partner_repack/yandex-ua/linux-i686/ru/firefox-38.0.6.tar.bz2

	Nick Thomas [:nthomas] (UTC+12)
	Comment 2 • 9 years ago

The --release-to-tinderbox-builds case sounds like release promotion. I think we're going to end up with tc artifacts + beetmover in the end, so I wouldn't worry too much about it now.

BTW, if we do end up with any post_upload.py changes please let oremj know, so he can integrate them into his S3 version.

	bhearsum@mozilla.com (:bhearsum) Assignee
	Comment 3 • 9 years ago

(In reply to Nick Thomas [:nthomas] from comment #2)
> The --release-to-tinderbox-builds case sounds like release promotion. I
> think we're going to end up with tc artifacts + beetmover in the end, so I
> wouldn't worry too much about it now.
> 
> BTW, if we do end up with any post_upload.py changes please let oremj know,
> so he can integrate them into his S3 version.

Ah, OK. In that case we can probably do without any post_upload.py changes. On the off chance that we end up needing these builds to upload correctly to tinderbox-builds, we can come back to this and address it then.