Richard Marti (:Paenglab) Reporter
	Description • 7 years ago

0:56.06 error: failed to run custom build command for `miniz-sys v0.1.9`
 0:56.09 process didn't exit successfully: `z:\Mozilla\TB\testing\geckodriver\.\release\build\miniz-sys-0d42c5494ba5bc47\build-script-build` (exit code: 101)
 0:56.10 --- stdout
 0:56.10 TARGET = Some("i686-pc-windows-msvc")
 0:56.10 OPT_LEVEL = Some("3")
 0:56.10 TARGET = Some("i686-pc-windows-msvc")
 0:56.10 HOST = Some("x86_64-pc-windows-msvc")
 0:56.10 TARGET = Some("i686-pc-windows-msvc")
 0:56.10 TARGET = Some("i686-pc-windows-msvc")
 0:56.10 HOST = Some("x86_64-pc-windows-msvc")
 0:56.11 CC_i686-pc-windows-msvc = None
 0:56.11 CC_i686_pc_windows_msvc = None
 0:56.11 TARGET_CC = None
 0:56.11 CC = None
 0:56.12 TARGET = Some("i686-pc-windows-msvc")
 0:56.12 HOST = Some("x86_64-pc-windows-msvc")
 0:56.12 CFLAGS_i686-pc-windows-msvc = None
 0:56.12 CFLAGS_i686_pc_windows_msvc = None
 0:56.12 TARGET_CFLAGS = None
 0:56.12 CFLAGS = None
 0:56.13 PROFILE = Some("release")
 0:56.13 running: "cl.exe" "/nologo" "/MD" "/O2" "/Foz:\\Mozilla\\TB\\testing\\geckodriver\\.\\i686-pc-windows-msvc\\release\\build\\miniz-sys-18005000ddedadf4\\out\\miniz.o" "/c" "miniz.c"
 0:56.13
 0:56.13
 0:56.13 failed to execute command: Das System kann die angegebene Datei nicht finden. (os error 2)
 0:56.13 Is `cl.exe` not installed? (see https://github.com/alexcrichton/gcc-rs#compile-time-requirements for help)

I tried to set the path explained in https://github.com/alexcrichton/gcc-rs#compile-time-requirements, but didn't success.

What do I need to do? I'm not a compiler specialist.

	Frank-Rainer Grahl (:frg)
	Comment 1 • 7 years ago

Me too. Unable to compile suite. Is probably caused by Bug 1340637. From the patch it looks like 
it doesn't take the comm-central dir into accound and tries to find the source in the topmost sourcedir:

/Foz:\\Mozilla\\TB\\testing\\

Should probably be /Foz:\\Mozilla\\TB\\mozilla\\testing\\ in case of comm-central builds. Best locally would probably to have an option to disable it. I see not much if any use for it locally. That still would still ned a fix for the offcial TB and SM builds if it is needed there.

	Jorg K (CEST = GMT+2)
	Comment 2 • 7 years ago

Andreas and Ted, it seems that changes in bug 1340637 busted local builds, at least for Thunderbird for some people.

I tried local Firefox and Thunderbird 32bit debug builds and they seem to work.

Richard, can you please attach a longer log of the first 56 seconds to see where the error happens. My build is happily compiling ;-)

	Richard Marti (:Paenglab) Reporter
	Comment 3 • 7 years ago

I don't have the log anymore. But before where no errors. If needed I have a older log (the initial build which failed) with output before the error.

	Jorg K (CEST = GMT+2)
	Comment 4 • 7 years ago

I spoke too early, it fails for me, too, after eight(!!) minutes.

If it's a "top source dir" issue, then FF local build won't be affected. I'm just wondering why server builds on treeherder still work for C-C. I've just retriggered a Mac opt
https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=abc0bcd3caec68343fce1ff4fcb700edd6c72076&selectedJob=102237328
and that worked.

	Frank-Rainer Grahl (:frg)
	Comment 5 • 7 years ago

>> /Foz:\\Mozilla\\TB\\testing\\geckodriver

This is the object dir so I take it back that it might be a toplevel source problem. If the script tries to run the x86 cl.exe for cross compile it might miss some files from the x64 version. The server uses the tooltool package and might not be affected. Anyway just speculation.

	Andreas Tolfsen ❲:ato❳ Assignee
	Comment 6 • 7 years ago

A quick search indicates cl.exe is not exposed in the Windows command prompt by default unless you enable the ‘SDK command line’ or install the C++ requirements in Visual Studio.  Possibly you need to start a Visual Studio command prompt that sets all the correct path variables for it to pick up cl.exe correctly.

We need cl.exe to compile some C libraries as part of a Rust dependency used by testing/geckodriver.

Note that with https://bugzilla.mozilla.org/show_bug.cgi?id=1368035 we are making the compilation of testing/geckodriver opt-in rather than mandatory on local developer machines.  Compiling it in CI will still be required as it is used to run the WPT wdspec tests.

	Jorg K (CEST = GMT+2)
	Comment 7 • 7 years ago

I local Firefox build fails as well. So basically you've broken all local compiles on Windows for FF, TB and SM.

So can you please come up with a fix or workaround real quick.

	Andreas Tolfsen ❲:ato❳ Assignee
	Comment 8 • 7 years ago

I am temporarily disabling compilation of testing/geckodriver in https://bugzilla.mozilla.org/show_bug.cgi?id=1368084 until https://bugzilla.mozilla.org/show_bug.cgi?id=1368035, to make it opt-in for local developers, is fixed.

	Xidorn Quan [:xidorn] UTC+11
	Comment 10 • 7 years ago

(In reply to Andreas Tolfsen ‹:ato› from comment #6)
> A quick search indicates cl.exe is not exposed in the Windows command prompt
> by default unless you enable the ‘SDK command line’ or install the C++
> requirements in Visual Studio.  Possibly you need to start a Visual Studio
> command prompt that sets all the correct path variables for it to pick up
> cl.exe correctly.

The problem is not this. As I mentioned in bug 1367935 comment 4, I'm able to build geckodriver via cargo directly from the source code in the same environment, which means that it is mach which hides the necessary environment variables.

It reminds me of bug 1350001 which removed almost all environment variables for Windows when calling into cargo, which then broke stylo build (see bug 1355464). They could be related.

	Xidorn Quan [:xidorn] UTC+11
	Comment 12 • 7 years ago

(In reply to Jorg K (GMT+2) from comment #11)
> Created attachment 8871793 [details] [diff] [review]
> Patch by FRG to get the compile going again ;-) - final solution may differ.

That's the same workaround I came up with in bug 1367935 comment 2 :)

	Andreas Tolfsen ❲:ato❳ Assignee
	Comment 13 • 7 years ago

(In reply to Xidorn Quan [:xidorn] UTC+10 from comment #10)
> (In reply to Andreas Tolfsen ‹:ato› from comment #6)
> > A quick search indicates cl.exe is not exposed in the Windows command prompt
> > by default unless you enable the ‘SDK command line’ or install the C++
> > requirements in Visual Studio.  Possibly you need to start a Visual Studio
> > command prompt that sets all the correct path variables for it to pick up
> > cl.exe correctly.
> 
> The problem is not this. As I mentioned in bug 1367935 comment 4, I'm able
> to build geckodriver via cargo directly from the source code in the same
> environment, which means that it is mach which hides the necessary
> environment variables.
> 
> It reminds me of bug 1350001 which removed almost all environment variables
> for Windows when calling into cargo, which then broke stylo build (see bug
> 1355464). They could be related.

This is very interesting information.  I will follow up on this.

In the meantime I’m resolving this bug since the build problems on m-c are fixed by https://bugzilla.mozilla.org/show_bug.cgi?id=1368084.

	Frank-Rainer Grahl (:frg)
	Comment 14 • 7 years ago

> This is very interesting information.  I will follow up on this.

If you need to call cl.exe directly also take into account the the x86 cross compiler needs the x64 libs and that VS2017 also has a different directory structure. 

https://bugzilla.mozilla.org/show_bug.cgi?id=1318143#c34

	(not currently active) Ted Mielczarek
	Comment 15 • 7 years ago

xidorn (or someone else who can reproduce this): can you `cargo run` this test app from the same shell where you're building Firefox? It's just using the same version of the gcc crate that we have vendored, and printing the MSVC paths it returns.

On my Windows machine, which has MSVC 2015 and 2017 installed, in a MozillaBuild start-shell.bat shell it prints:
CC_x86_64-pc-windows-msvc = None
CC_x86_64_pc_windows_msvc = None
HOST_CC = None
CC = None
CFLAGS_x86_64-pc-windows-msvc = None
CFLAGS_x86_64_pc_windows_msvc = None
HOST_CFLAGS = None
CFLAGS = None
64-bit MSVC path: "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64\\cl.exe"
CC_i686-pc-windows-msvc = None
CC_i686_pc_windows_msvc = None
TARGET_CC = None
CC = None
CFLAGS_i686-pc-windows-msvc = None
CFLAGS_i686_pc_windows_msvc = None
TARGET_CFLAGS = None
CFLAGS = None
32-bit MSVC path: "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64_x86\\cl.exe"

	(not currently active) Ted Mielczarek
	Comment 16 • 7 years ago

Oops, the test app is here: https://github.com/luser/rust-msvc-test

	Masatoshi Kimura [:emk]
	Comment 17 • 7 years ago

$ cargo run
    Updating registry `https://github.com/rust-lang/crates.io-index`
 Downloading gcc v0.3.42
   Compiling gcc v0.3.42
   Compiling msvc-test v0.1.0 (file:///J:/temp/rust-msvc-test-master/rust-msvc-test-master)
    Finished dev [unoptimized + debuginfo] target(s) in 12.78 secs
     Running `target\debug\msvc-test.exe`
CC_x86_64-pc-windows-msvc = None
CC_x86_64_pc_windows_msvc = None
HOST_CC = None
CC = None
CFLAGS_x86_64-pc-windows-msvc = None
CFLAGS_x86_64_pc_windows_msvc = None
HOST_CFLAGS = None
CFLAGS = None
64-bit MSVC path: "d:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\cl.exe"
CC_i686-pc-windows-msvc = None
CC_i686_pc_windows_msvc = None
TARGET_CC = None
CC = None
CFLAGS_i686-pc-windows-msvc = None
CFLAGS_i686_pc_windows_msvc = None
TARGET_CFLAGS = None
CFLAGS = None
32-bit MSVC path: "d:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\cl.exe"

(I installed MSVC 2015 on D: because my C: is a SSD)

	(not currently active) Ted Mielczarek
	Comment 18 • 7 years ago

This seems wrong:
32-bit MSVC path: "d:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\cl.exe"

You don't have any MSVC environment variables set in that shell, do you?

	Masatoshi Kimura [:emk]
	Comment 19 • 7 years ago

What are "MSVC environment variables"? I'm just running start-shell-msvc2015-x64.bat.

	Masatoshi Kimura [:emk]
	Comment 20 • 7 years ago

If I run start-shell-msvc2015.bat, I got:

$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.4 secs
     Running `target\debug\msvc-test.exe`
CC_x86_64-pc-windows-msvc = None
CC_x86_64_pc_windows_msvc = None
HOST_CC = None
CC = None
CFLAGS_x86_64-pc-windows-msvc = None
CFLAGS_x86_64_pc_windows_msvc = None
HOST_CFLAGS = None
CFLAGS = None
64-bit MSVC path: "d:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64_x86\\cl.exe"
CC_i686-pc-windows-msvc = None
CC_i686_pc_windows_msvc = None
TARGET_CC = None
CC = None
CFLAGS_i686-pc-windows-msvc = None
CFLAGS_i686_pc_windows_msvc = None
TARGET_CFLAGS = None
CFLAGS = None
32-bit MSVC path: "d:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64_x86\\cl.exe"

	Masatoshi Kimura [:emk]
	Comment 21 • 7 years ago

By the way, I backed out bug 1368084 and I got the following error:
 1:58.69 failed to execute command: 隰厄ｿｽ・ｮ螢ｹ・・ｹｧ蠕娯螺郢晁ｼ斐＜郢ｧ・､郢晢ｽｫ邵ｺ迹夲ｽｦ荵昶命邵ｺ荵晢ｽ顔ｸｺ・ｾ邵ｺ蟶呻ｽ鍋ｸｲ・ｽ(os error 2)
Looks like a wrong encoding is used.

	Xidorn Quan [:xidorn] UTC+11
	Comment 22 • 7 years ago

start-shell-msvc2015-x64.bat gives me:

$ cargo run
   Compiling gcc v0.3.42
   Compiling msvc-test v0.1.0 (file:///C:/mozilla-source/rust-msvc-test)
    Finished dev [unoptimized + debuginfo] target(s) in 3.21 secs
     Running `target\debug\msvc-test.exe`
CC_x86_64-pc-windows-msvc = None
CC_x86_64_pc_windows_msvc = None
HOST_CC = None
CC = None
CFLAGS_x86_64-pc-windows-msvc = None
CFLAGS_x86_64_pc_windows_msvc = None
HOST_CFLAGS = None
CFLAGS = None
64-bit MSVC path: "c:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\cl.exe"
CC_i686-pc-windows-msvc = None
CC_i686_pc_windows_msvc = None
TARGET_CC = None
CC = None
CFLAGS_i686-pc-windows-msvc = None
CFLAGS_i686_pc_windows_msvc = None
TARGET_CFLAGS = None
CFLAGS = None
32-bit MSVC path: "c:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\cl.exe"


Seems to be similar to emk's.

	Masatoshi Kimura [:emk]
	Comment 23 • 7 years ago

(In reply to Andreas Tolfsen ‹:ato› from comment #13)
> In the meantime I’m resolving this bug since the build problems on m-c are
> fixed by https://bugzilla.mozilla.org/show_bug.cgi?id=1368084.

Is there a bug that tracks the underlying issue so that we can re-enable geckodriver building on Windows?
I don't think it's a good idea to continue works on a closed bug.

	Jorg K (CEST = GMT+2)
	Comment 24 • 7 years ago

Bug 1368035?

	Masatoshi Kimura [:emk]
	Comment 25 • 7 years ago

Apparently it does not take care of failures in local builds.

	Andreas Tolfsen ❲:ato❳ Assignee
	Comment 26 • 7 years ago

(In reply to Masatoshi Kimura [:emk] from comment #23)
> (In reply to Andreas Tolfsen ‹:ato› from comment #13)
> > In the meantime I’m resolving this bug since the build problems on m-c are
> > fixed by https://bugzilla.mozilla.org/show_bug.cgi?id=1368084.
> 
> Is there a bug that tracks the underlying issue so that we can re-enable
> geckodriver building on Windows?

We can use this bug to track that, certainly.

	Masatoshi Kimura [:emk]
	Comment 27 • 7 years ago

Oh, I got the correct result if I run start-shell.bat:
$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.4 secs
     Running `target\debug\msvc-test.exe`
CC_x86_64-pc-windows-msvc = None
CC_x86_64_pc_windows_msvc = None
HOST_CC = None
CC = None
CFLAGS_x86_64-pc-windows-msvc = None
CFLAGS_x86_64_pc_windows_msvc = None
HOST_CFLAGS = None
CFLAGS = None
64-bit MSVC path: "D:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64\\cl.exe"
CC_i686-pc-windows-msvc = None
CC_i686_pc_windows_msvc = None
TARGET_CC = None
CC = None
CFLAGS_i686-pc-windows-msvc = None
CFLAGS_i686_pc_windows_msvc = None
TARGET_CFLAGS = None
CFLAGS = None
32-bit MSVC path: "D:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64_x86\\cl.exe"

But the current latest release of MozillaBuild does not support building from start-shell.bat yet. I have to use start-shell-msvc2015(-x64).bat.

	Masatoshi Kimura [:emk]
	Comment 28 • 7 years ago

(In reply to Masatoshi Kimura [:emk] from comment #27)
> But the current latest release of MozillaBuild does not support building
> from start-shell.bat yet. I have to use start-shell-msvc2015(-x64).bat.

Hm, I was wrong. I could build Firefox with geckodriver enabled.

	Masatoshi Kimura [:emk]
	Comment 29 • 7 years ago

start-shell-msvc2015(-x64).bat will set $VCINSTALLDIR, so the tool detection will not run:
https://github.com/alexcrichton/gcc-rs/blob/0d87a9b739a9b0086015d5356de729f1dc1fb010/src/windows_registry.rs#L70-L74

	Andreas Tolfsen ❲:ato❳ Assignee
	Comment 30 • 6 years ago

I think this problem has been resolved.  At least we have had no
complaints that enabling geckodriver in the default local build has
caused any problems for Windows developers.