User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.53 Safari/537.36

Steps to reproduce:

Building Firefox master with GCC with LTO and PGO.

Actual results:

99:55.55 JARLOG_FILE=jarlog/en-US.log /home/marxin/Programming/gecko-dev/obj-x86_64-pc-linux-gnu/_virtualenvs/init/bin/python /home/marxin/Programming/gecko-dev/build/pgo/profileserver.py
99:55.76 Traceback (most recent call last):
99:55.76 File "/home/marxin/Programming/gecko-dev/build/pgo/profileserver.py", line 36, in <module>
99:55.76 binary = build.get_binary_path(where="staged-package")
99:55.76 File "/home/marxin/Programming/gecko-dev/python/mozbuild/mozbuild/base.py", line 478, in get_binary_path
99:55.76 raise Exception('Binary expected at %s does not exist.' % path)
99:55.76 Exception: Binary expected at /home/marxin/Programming/gecko-dev/obj-x86_64-pc-linux-gnu/dist/firefox/firefox does not exist.

The binary is here:

$ obj-x86_64-pc-linux-gnu/dist/bin/firefox

My .mozconfig looks as follows:
cat .mozconfig

mk_add_options MOZ_MAKE_FLAGS="-j8"

ac_add_options --enable-application=browser
ac_add_options --disable-tests
ac_add_options --disable-debug
ac_add_options --enable-debug-symbols
ac_add_options --enable-system-icu

# valgrind purpose
ac_add_options --disable-jemalloc
ac_add_options --disable-valgrind

# ac_add_options --disable-ogg
# ac_add_options --with-system-libvpx
# ac_add_options --disable-necko-wifi
ac_add_options --disable-elf-hack

# current ld.gold 2.29.1 is ICEing -> fixed on trunk
ac_add_options --enable-linker=bfd

ac_add_options --enable-lto
ac_add_options MOZ_PGO=1

export CC=gcc
export CXX=g++

ac_add_options --disable-profiling
ac_add_options --disable-vtune

Couldn't reproduce, as this issue is too technical for me. I'm assigning a component to get a start on this, if considered wrong, please feel free to change it to a more appropriate one.

Did we break PGO'ing without packaging when we made the transition to a three-stage PGO automation build?

That sounds plausible - I'll take a look.

I tried to reproduce this but I wasn't able to. The packaging step called from the Makefile in [1] should be creating the obj-x86_64-pc-linux-gnu/dist/firefox/firefox directory.

Can you capture the output when you run the build and attach it to the bug? That might offer some clues.

[1] https://dxr.mozilla.org/mozilla-central/rev/c2593a3058afdfeaac5c990e18794ee8257afe99/Makefile.in#186

Unassigning until we get more info.

Works for me.