Closed Bug 1206106 Opened 9 years ago Closed 9 years ago

Create a java tarball that works on CentOS 6

Categories

(Release Engineering :: General, defect)

Product:
Release Engineering
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox44 fixed)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox44 --- fixed

People

(Reporter: dustin, Assigned: dustin)

References

Details

Attachments

(1 file)

150922_1206106_java_centos_manifest_change.patch
1.09 KB, patch
dustin
: review+
Details | Diff | Splinter Review 
testing/taskcluster/scripts/misc/repackage-jdk.sh is repackaging an Ubuntu-compatible Java.  Now that we're building on CentOS 6, we need a CentOS-6 compatible Java, probably repackaged from an RPM.
Assignee: nobody → dustin
Blocks: 1163084
Blocks: 1199720
so reading through bug 1161075, it seems like you built it by hand and uploaded it to tooltool here: https://bugzilla.mozilla.org/show_bug.cgi?id=1161075#c31

I guess you went back to using the ubuntu one when our ubuntu machines didn't like the centos based one so you put a check in the build sys[1] to say 'if buildbot: ignore the tooltool downloaded java_home dir; if taskcluster(then ubuntu): use java_home'

what's interesting is that your check looks for centos 6.2 but our buildbot machines seem to use 6.5 so I don't understand how this[1] block is being skipped in the buildbot case.

anyway, I'll try your tooltool centos based java. if it works, I guess we could probably use that for both buildbot and tascluster?

[1] https://hg.mozilla.org/mozilla-central/rev/7b4b396ba9cd#l2.16
The mock environments are CentOS 6.2, hence the check.

Since we're using CentOS 6.7 in TC, I suspect there will be issues trying to run the same tooltool download in BB and TC.  But if not, then yes, doing the same on both platforms would be ideal.
(In reply to Dustin J. Mitchell [:dustin] from comment #2)
> The mock environments are CentOS 6.2, hence the check.

ah, right, the mock env is different os.


> Since we're using CentOS 6.7 in TC, I suspect there will be issues trying to
> run the same tooltool download in BB and TC.

I was close but I think it's not a perfect fit:
7:46:06     INFO -  /home/worker/workspace/build/src/java_home/bin/jar cMf gecko-thirdparty.jar -C gecko-thirdparty-classes .
07:46:09     INFO -  Note: Some input files use or override a deprecated API.
07:46:09     INFO -  Note: Recompile with -Xlint:deprecation for details.
07:46:09     INFO -  Note: Some input files use unchecked or unsafe operations.
07:46:09     INFO -  Note: Recompile with -Xlint:unchecked for details.
07:46:10     INFO -  /home/worker/workspace/build/src/java_home/bin/jar cMf sync-thirdparty.jar -C sync-thirdparty-classes .
07:46:11     INFO -  gmake[5]: Leaving directory `/home/worker/workspace/build/src/obj-firefox/mobile/android/base'
07:46:11     INFO -  gmake[4]: *** [mobile/android/base/libs] Error 2
07:46:11     INFO -  gmake[4]: Leaving directory `/home/worker/workspace/build/src/obj-firefox'
07:46:11     INFO -  gmake[3]: *** [libs] Error 2
07:46:11     INFO -  gmake[3]: Leaving directory `/home/worker/workspace/build/src/obj-firefox'
07:46:11     INFO -  gmake[2]: *** [default] Error 2
07:46:11     INFO -  gmake[2]: Leaving directory `/home/worker/workspace/build/src/obj-firefox'
07:46:11     INFO -  gmake[1]: *** [realbuild] Error 2
07:46:11     INFO -  gmake[1]: Leaving directory `/home/worker/workspace/build/src'
07:46:11     INFO -  gmake: *** [build] Error 2

I'll try one based on newer centos
So I tried doing this based on centos 6.7 and still getting errors:

push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=8ba3ba4eb1e3&exclusion_profile=false
log: https://s3-us-west-2.amazonaws.com/taskcluster-public-artifacts/XhkAP901QgSp0xLf5Satgw/0/public/logs/live_backing.log

snippet:
02:38:09     INFO -  /home/worker/workspace/build/src/android-sdk-linux/build-tools/22.0.1/aapt: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory
02:38:09     INFO -  gmake[5]: *** [.aapt.deps] Error 127

02:38:09     INFO -  /home/worker/workspace/build/src/android-sdk-linux/build-tools/22.0.1/aapt: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory


what I did:

# repackage openjdk via centos 6.7 machine
[
root@taskcluster-worker open_jdk]# wget http://mirror.centos.org/centos/6.7/updates/x86_64/Packages/java-1.7.0-openjdk-1.7.0.85-2.6.1.3.el6_7.x86_64.rpm
[root@taskcluster-worker open_jdk]# wget http://mirror.centos.org/centos/6.7/updates/x86_64/Packages/java-1.7.0-openjdk-devel-1.7.0.85-2.6.1.3.el6_7.x86_64.rpm
[root@taskcluster-worker open_jdk]# rpm2cpio java-1.7.0-openjdk-1.7.0.85-2.6.1.3.el6_7.x86_64.rpm | cpio -ivd
[root@taskcluster-worker open_jdk]# rpm2cpio java-1.7.0-openjdk-devel-1.7.0.85-2.6.1.3.el6_7.x86_64.rpm | cpio -ivd
[root@taskcluster-worker jvm]# cat > java-1.7.0-openjdk-1.7.0.85.x86_64/VERSION
Built from http://mirror.centos.org/centos/6.7/updates/x86_64/Packages/
    java-1.7.0-openjdk-devel-1.7.0.85-2.6.1.3.el6_7.x86_64.rpm
    java-1.7.0-openjdk-1.7.0.85-2.6.1.3.el6_7.x86_64.rpm

Run through rpm2cpio | cpio, and /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.85.x86_64 renamed to 'java_home'.
[root@taskcluster-worker jvm]# mv java-1.7.0-openjdk-1.7.0.85.x86_64 java_home
[root@taskcluster-worker jvm]# tar -Jcvf java_home-1.7.0-openjdk-1.7.0.85.x86_64.tar.xz java_home


#upload to tooltool
[root@taskcluster-worker ~]# python build-tooltool/tooltool.py add --visibility public java_home-1.7.0-openjdk-1.7.0.85.x86_64.tar.xz
[root@taskcluster-worker ~]# python build-tooltool/tooltool.py upload --authentication-file=~/auth.txt --message='Bug 1206106 - Create a java tarball that works on CentOS 6'

# added the following to the in-tree android manifest
[root@taskcluster-worker ~]# cat manifest.tt
[
{
"size": 31013068,
"visibility": "public",
"digest": "e30a26f98a3448064857491aee1a7a26f98494f86a89113de9be17c37c8181ed60250706fed881ec1f035002fcdaf8b9b4a7d9ae70ce40acff2f1acfbb40f8d9",
"algorithm": "sha512",
"filename": "java_home-1.7.0-openjdk-1.7.0.85.x86_64.tar.xz"
}
]


dustin: seems like you were also having issues with centos based rpm in TC: https://bugzilla.mozilla.org/show_bug.cgi?id=1161075#c36 any thoughts here?
Huh, that doesn't look like a java issue -- there are lots of successful java invocations above.  Rather, it's a failure of aapt, which is in the SDK.

[worker@taskcluster-worker ~]$ ldd ./android-sdk-linux/build-tools/22.0.1/aapt
        linux-gate.so.1 =>  (0xf774c000)
        librt.so.1 => /lib/librt.so.1 (0xf762f000)
        libdl.so.2 => /lib/libdl.so.2 (0xf762a000)
        libpthread.so.0 => /lib/libpthread.so.0 (0xf760f000)
        libz.so.1 => not found
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xf7523000)
        libm.so.6 => /lib/libm.so.6 (0xf74f9000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xf74db000)
        libc.so.6 => /lib/libc.so.6 (0xf7344000)
        /lib/ld-linux.so.2 (0xf774d000)

That file is provided by zlib, but the Android SDK is a 32-bit toolchain (!@#%!) so we need zlib.i686.

[root@taskcluster-worker ~]# yum install -y zlib.i686
...
[worker@taskcluster-worker ~]$ ldd ./android-sdk-linux/build-tools/22.0.1/aapt
        linux-gate.so.1 =>  (0xf7708000)
        librt.so.1 => /lib/librt.so.1 (0xf75eb000)
        libdl.so.2 => /lib/libdl.so.2 (0xf75e6000)
        libpthread.so.0 => /lib/libpthread.so.0 (0xf75cb000)
        libz.so.1 => /lib/libz.so.1 (0xf75b7000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xf74cb000)
        libm.so.6 => /lib/libm.so.6 (0xf74a1000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xf7483000)
        libc.so.6 => /lib/libc.so.6 (0xf72ec000)
        /lib/ld-linux.so.2 (0xf7709000)

so!  Please go ahead and land your changes for java.  I'll roll zlib.i686 into the patch I'm working on for bug 1189892.
Oh, and it'd be super-sweet if you could encapsulate the "what I did" from comment 4 into the script under testing/taskcluster/scripts/misc.
(In reply to Dustin J. Mitchell [:dustin] from comment #6)
> Oh, and it'd be super-sweet if you could encapsulate the "what I did" from
> comment 4 into the script under testing/taskcluster/scripts/misc.

my very rough translation of cmds: comment 7
I'll land once it's reviewed
Attachment #8664390 - Flags: review?(dustin)
Comment on attachment 8664390 [details] [diff] [review]
150922_1206106_java_centos_manifest_change.patch

Review of attachment 8664390 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks for the script.  You can just delete the old ubuntu repacker -- we don't use that anymore.
Attachment #8664390 - Flags: review?(dustin) → review+
That's an unrelated infra issue -- re-landing this patch should be fine.
Flags: needinfo?(dustin)
Blocks: 1118394
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: