Chris AtLee [:catlee] Reporter
	Description • 12 years ago

test archives are quite large, and are currently compressed using zip. as of this writing:

58099173 firefox-13.0a1.en-US.linux-i686.tests.zip
35245341 firefox-13.0a1.en-US.linux-i686.tests.tar.bz2
25022044 firefox-13.0a1.en-US.linux-i686.tests.tar.xz

Using .xz files would save more than 50% on disk space, disk io and network bandwidth, three resources we're always regularly hitting upper limits on.

We need all of our build and test machines to support creating and extracting .tar.xz files.

	Carsten Book [:Tomcat]
	Comment 2 • 11 years ago

seems linux should be very easy via the integrated updater. 

For mac there is http://sourceforge.net/projects/macpkg/.

Windows might be a little more work, i found:

 If you have xz-5.x.x.tar.gz in C:\devel, you should be able to build
    it now with the following commands:

        cd /c/devel
        tar xzf xz-5.x.x.tar.gz
        cd xz-5.x.x
        bash windows/build.bash

    If you used some other directory than C:\devel\tools for the build
    tools, edit the variables near the beginning of build.bash first.

    If you want to build manually, read the buildit() function in
    build.bash. Look especially at the latter configure invocation.

	Andreas Tolfsen ❲:ato❳
	Comment 3 • 11 years ago

When I did some performance measurements on lzma archives for my previous employer a while back, they consumed less disk space but took longer to extract.  I haven't checked this again recently, but balancing download time of test packages vs. extraction time could be as important since some systems may have other limitations.

	Chris Cooper [:coop] (he/him)
	Comment 4 • 10 years ago

All of our linux platforms (including ec2 and spot) have xz installed by default now.

$ xz --version
xz (XZ Utils) 4.999.9beta
liblzma 4.999.9beta

Need someone for build or find packages for Mac and Windows.

	maillist-mozilla
	Comment 5 • 10 years ago

ping. this is getting more and more pressing as each firefox release grows by several megabytes.

the only difference between creating .bz2 and .xz is to use tar cJf instead of tar cjf (upper case J).
to make it even more effective create uncompressed .tar via tar cf and then compress it by invoking xz utility with extreme setting, i.e. xz -e -9.

	Chris AtLee [:catlee] Reporter
	Comment 6 • 10 years ago

(In reply to maillist-mozilla from comment #5)
> ping. this is getting more and more pressing as each firefox release grows
> by several megabytes.
> 
> the only difference between creating .bz2 and .xz is to use tar cJf instead
> of tar cjf (upper case J).
> to make it even more effective create uncompressed .tar via tar cf and then
> compress it by invoking xz utility with extreme setting, i.e. xz -e -9.

Unfortunately it's not quite that easy. We support many different platforms in our build/test infrastructure, including older platforms like Windows XP and OSX 10.7. We need to make sure xz archives are supported there as well.

	Chris AtLee [:catlee] Reporter
	Comment 7 • 10 years ago

That being said, it may be possible do this on Linux only to start with.

	maillist-mozilla
	Comment 8 • 10 years ago

oh, i thought this discussion is mainly about the source code tarballs.
aren't the windows/mac binaries shipped in the native format (sfx for windows, dmg for mac) anyways ?

	Chris AtLee [:catlee] Reporter
	Comment 9 • 10 years ago

release-specific tasks like source tarballs are being tracked in bug 807637. this bug is about general support for .xz in our infra.

	Andreas Tolfsen ❲:ato❳
	Comment 10 • 10 years ago

Please note that the increase in extraction time might make CI systems, and consequently the feedback loop to the developer, take longer due to greater extraction times with lzma.

	maillist-mozilla
	Comment 11 • 10 years ago

(In reply to Andreas Tolfsen (:ato) from comment #10)
> Please note that the increase in extraction time might make CI systems, and
> consequently the feedback loop to the developer, take longer due to greater
> extraction times with lzma.

i think you misunderstood something. xz/lzma decompression is 4x *faster* than bzip2 decompression.

	sjw
	Comment 12 • 7 years ago

Just a tiny update for macOS: xz compressed tarballs are supported in current versions, even in the GUI. xz compressed files can be uncompressed using the CLI.

Meanwhile some other projects like the kernel and Debian (dpkg) switched to xz.