Binary from optimised Linux build is 44M
Categories
(Testing :: geckodriver, defect, P3)
Tracking
(firefox70 fixed)
Tracking | Status | |
---|---|---|
firefox70 | --- | fixed |
People
(Reporter: ato, Assigned: nalexander)
References
(Blocks 1 open bug)
Details
Attachments
(2 files)
nalexander noted in https://bugzil.la/1441656#c3 that the geckodriver binary produced by an optimised Firefox build in central is 44M. I confirmed that this is also the case on Linux: > % du -shL obj-x86_64-pc-linux-gnu/dist/bin/ > 44M obj-x86_64-pc-linux-gnu/dist/bin/geckodriver If we compare this to a "cargo build --release" build, we can see that the expected size is somewhere close to ~8.7M: > % du -sh testing/geckodriver/target/release/geckodriver > 8.7M testing/geckodriver/target/release/geckodriver This bug is not related to https://bugzil.la/1392313, which aim is to _further_ reduce the binary size to a more managable level beyond 8.7M.
Reporter | ||
Updated•6 years ago
|
Reporter | ||
Updated•6 years ago
|
Reporter | ||
Comment 1•6 years ago
|
||
This bug is obviously preventing us from distributing release binaries produced by TaskCluster. Do you have any ideas or pointers where we should look as to why the build produced by "./mach build testing/geckodriver" (ensure you have ac_add_options --enable-geckodriver in your mozconfig first) is so vastly larger in size than those of "cd testing/geckodriver; cargo build"?
Comment 2•6 years ago
|
||
(In reply to Andreas Tolfsen 「:ato」 from comment #1) > This bug is obviously preventing us from distributing release > binaries produced by TaskCluster. Do you have any ideas or pointers > where we should look as to why the build produced by "./mach build > testing/geckodriver" (ensure you have ac_add_options --enable-geckodriver > in your mozconfig first) is so vastly larger in size than those of > "cd testing/geckodriver; cargo build"? Does your mozconfig have --enable-debug-symbols or the like turned on? That would probably account for a large chunk of it, since I believe `cargo build --release` wouldn't include debug symbols by default. Regardless of whether that helps, looking at `readelf -SW` output for each binary would be interesting.
Comment 3•6 years ago
|
||
Also, the default rustc opt level for `./mach build` is probably somewhat different than `cargo build --release`, and the latter probably includes LTO? Sticking `ac_add_options --enable-release` in your mozconfig might help`.
Comment 4•6 years ago
|
||
Oh, and if you're doing this locally, incremental compilation is probably turned on (?), so you would want to ensure that's turned off as well, as incremental compilation trades size of binaries for better compilation times. (Incremental compilation should be turned off on automation builds.)
Comment 5•5 years ago
|
||
I noticed the same yesterday while working on bug 1493948 and comparing the binary sizes between different platforms. As it looks like this issue only exists on Linux 32/64 but not on MacOS nor Windows:
Just click on any of the Fetch-geckodriver
jobs, and select the geckodriver
artifact.
Comment 6•5 years ago
|
||
$ ls -l target/release/geckodriver
-rwxrwxr-x 2 henrik henrik 6141392 Feb 22 11:12 target/release/geckodriver
Comment 7•5 years ago
|
||
$ ls -l obj-x86_64-pc-linux-gnu/testing/geckodriver/x86_64-unknown-linux-gnu/release/geckodriver
-rwxrwxr-x 2 henrik henrik 43927704 Feb 22 16:17 obj-x86_64-pc-linux-gnu/testing/geckodriver/x86_64-unknown-linux-gnu/release/geckodriver
Updated•5 years ago
|
Comment 8•5 years ago
|
||
It looks like that binary has all the debug info in it, which is ~1/3-1/2 of the binary.
Updated•5 years ago
|
Comment 9•5 years ago
|
||
Note that using ac_add_options --enable-release
doesn't make a difference. Also when I compile with cargo incremental compilation is turned off.
(In reply to Nathan Froyd [:froydnj] from comment #8)
It looks like that binary has all the debug info in it, which is ~1/3-1/2 of the binary.
I wonder why that is not stripped off. But even then the binary size would be still 5x that large as with cargo build --release
.
Comment 10•5 years ago
|
||
Also why does that only happen for Linux build jobs but not Mac and Windows? Maybe we missed to add something to the build configuration for this platform?
Updated•5 years ago
|
Comment 11•5 years ago
|
||
(In reply to Nathan Froyd [:froydnj] from comment #8)
It looks like that binary has all the debug info in it, which is ~1/3-1/2 of the binary.
This comment applied to the readelf output for the cargo --release
binary. You can see the debug info starting:
[28] .bss NOBITS 000000000055fd20 35fd20 0002a8 00 WA 0 0 8
[29] .comment PROGBITS 0000000000000000 35fd20 000057 01 MS 0 0 1
[30] .debug_pubnames PROGBITS 0000000000000000 35fd77 027658 00 0 0 1
...and continuing on from there. That 0x35fd77 is the start of .debug_pubnames; basically everything prior to that is what your binary size is. So about 3.5MB (plus a little extra for the .{sym,str}tab sections later on).
Whereas the mozilla-central geckodriver has:
[30] .bss NOBITS 000000000056fd10 36fd10 0002a8 00 WA 0 0 8
[31] .comment PROGBITS 0000000000000000 36fd10 0000a9 01 MS 0 0 1
[32] .debug_pubnames PROGBITS 0000000000000000 36fdb9 21ec85 00 0 0 1
Notice that .debug_pubnames starts at 0x36fdb9, so about the same place as the cargo --release
version. But the debug info sections are much bigger. Compare the cargo --release
bits:
...
[39] .debug_ranges PROGBITS 0000000000000000 51ce01 0497a0 00 0 0 1
[40] .symtab SYMTAB 0000000000000000 5665a8 01bd08 18 41 2728 8
where .symtab starts at 0x5665a8, to the mozilla-central version:
...
[41] .debug_ranges PROGBITS 0000000000000000 2579e20 3f5f60 00 0 0 1
[42] .symtab SYMTAB 0000000000000000 296fd80 01bc18 18 43 2741 8
where .symtab starts at 0x296fd80, which is a significant difference in the amount of debug info. We compile with more debug information in m-c versus cargo --release
, so that's probably where the size difference is coming from on Linux.
On Mac, at least, debug information can be stored in separate files, so if you hunted around for those, you'd probably notice similar size differences. Same thing on Windows.
Comment 12•5 years ago
|
||
I see. Do you know of a way to instruct the build to not include the debug info but store it in a separate file on Linux? Or are we forced to strip the binaries afterward?
Maybe we should simply disable debug info at all for the geckodriver opt builds in mc, only leave them enabled for debug builds?
Comment 13•5 years ago
|
||
(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #12)
I see. Do you know of a way to instruct the build to not include the debug info but store it in a separate file on Linux? Or are we forced to strip the binaries afterward?
There's a GCC option (clang too?) called...-gsplit-dwarf, I think? I have no idea whether there's anything comparable for Rust.
I know we strip target binaries (see ENABLE_STRIP uses in config/rules.mk
), but it looks like we don't do anything similar for host binaries.
Maybe we should simply disable debug info at all for the geckodriver opt builds in mc, only leave them enabled for debug builds?
We could do that via separate Rust flags for target and host builds (and maybe even programs vs. libraries, depending).
Comment 14•5 years ago
|
||
Ok, so I think it will be better to wait for bug 1534533 then which will move building geckodriver to its own toolchain build task.
Comment 15•5 years ago
|
||
I triggered a new geckodriver toolchain job, which we will make use of for releasing geckodriver in the future:
And the geckodriver binary for Linux is 6.6 MB only now!
Description
•