Eric Tsai (no more review request accepted) Reporter
	Description • 7 years ago

Original issue from https://github.com/mozilla/geckodriver/issues/733

Using Python 3, Selenium 3.4.1, geckodriver 0.16.1 64 bits, run Firefox 55 Nightly, build after 2017-05-02-03-02-11, like 2017-05-04-03-03-20

Step to reproduce:

1. Download and extract latest Nightly
2. Launch Firefox Nightly with 

    binary = FirefoxBinary("./firefox/firefox.exe")
    firefox = webdriver.Firefox(firefox_binary=binary)
    firefox.get("http://webcompat-ua-dummy.schub.io/")

Expect result: See a Firefox image
Actual result: See "Please use Chrome to visit this website". Open about:support, no any "Nightly Features", seem no System Addon installed, including "Web Compat". No Graphics information. Attach geckodriver log see the following error message:

>> 1494583876191	addons.manager	ERROR	startup failed: [Exception... "Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIFile.create]"  nsresult: "0x80070057 (NS_ERROR_ILLEGAL_VALUE)"  location: "JS frame :: resource://gre/modules/FileUtils.jsm :: FileUtils_getDir :: line 70"  data: no] Stack trace: FileUtils_getDir()@resource://gre/modules/FileUtils.jsm:70 < FileUtils_getFile()@resource://gre/modules/FileUtils.jsm:42 < validateBlocklist()@resource://gre/modules/AddonManager.jsm:653 < startup()@resource://gre/modules/AddonManager.jsm:813 < startup()@resource://gre/modules/AddonManager.jsm:3081 < observe()@resource://gre/components/addonManager.js:65
>> JavaScript error: resource://gre/modules/AddonManager.jsm, line 1593: NS_ERROR_NOT_INITIALIZED: AddonManager is not initialized
```

Here is the test result:
geckodriver 0.15.0 + Firefox 55 2017-05-02-03-02-11 works
geckodriver 0.15.0 + Firefox 55 2017-05-04-03-03-20 with Nightly Feature but can't get URL
geckodriver 0.16.0 + Firefox 55 2017-05-02-03-02-11 no Nightly Feature but can get URL
geckodriver 0.16.0 + Firefox 55 2017-05-04-03-03-20 no Nightly Feature but can get URL
geckodriver 0.16.1 + Firefox 55 2017-05-04-03-02-11 no Nightly Feature but can get URL
geckodriver 0.16.1 + Firefox 55 2017-05-04-03-03-20 no Nightly Feature but can get URL

	David Burns :automatedtester
	Comment 1 • 7 years ago

This appears to be related to Nightly Features and has nothing to Marionette. It will need to be triaged to the correct team but I dont know who that might be

	:shell escalante
	Comment 2 • 7 years ago

awan looking for dupe

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 4 • 7 years ago

Andrew, I don't see that the reporter is starting Firefox from a read-only file system. Are you sure this is the right bug to dupe?

	Andrew Swan [:aswan]
	Comment 5 • 7 years ago

Oh sorry, its a similar stack trace but a different exception.
I know very little about Selenium, is this something I have any hope of reproducing locally?

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 6 • 7 years ago

I thought that I was able to see it when running Marionette tests directly. But with a recent Nightly those failures do not appear anymore.

Eric, can you please check with a Nightly from today? If it still appears it might be a Windows only issue.

	Eric Tsai (no more review request accepted) Reporter
	Comment 7 • 7 years ago

Henrik, sorry for late reply. But the build is still not work for me today. So I think it's a Windows only issue, or Windows 7 issue? Because the same code works on ubuntu.

	Andrew Swan [:aswan]
	Comment 8 • 7 years ago

Eric, can you answer the question from comment 5?

	Eric Tsai (no more review request accepted) Reporter
	Comment 9 • 7 years ago

It's easy to reproduce locally, here are what you need
Windows 7 pro machine
Python 3, install selenium package
Download latest Nightly, assign Firefox binary to the download Nightly and run the code in initial report.
I will see what break this between 2017-05-02 and 2017-05-04 and if this also break on Windows 10.

	Jason Juang (:juangj)
	Comment 10 • 7 years ago

I can reproduce this as well. I'm using (32-bit) Windows 7 and the Firefox 55 Nightly from 2017-05-24. I cannot reproduce this on Linux (Ubuntu, 64-bit).

To reproduce without Selenium:
1. Download geckodriver from https://github.com/mozilla/geckodriver/releases
2. Run: geckodriver.exe --port=4444 -b "C:\Program Files\Nightly\firefox.exe"
3. Send a POST request with your tool of choice, e.g.,
   c:\bin\wget.exe http://localhost:4444/session --post-data="{\"alwaysMatch\": {\"browserName\": \"firefox\"}}"

That should launch an instance of Firefox. In the Browser Console, I see the same error described by the original report (ending in "NS_ERROR_NOT_INITIALIZED: AddonManager is not initialized").

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 11 • 7 years ago

So this is basically not a regression which recently started in Firefox. But I assume this happens because some logging settings have been changed in the 0.16 release of Geckodriver, which now cause debug/trace level output of the Add-on Manager to appear. 

I'm not sure yet why this only affects Windows, and also which preference it is. I will dig into those changes.

Until I'm clear what the problem is I will not move this bug around.

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 12 • 7 years ago

Ok, so the problem here is that the blocklist file cannot be created because the `XCurProcD` reports an invalid path which has a correlation to what geckodriver prints out when starting the application:

1496315305763	geckodriver	INFO	Listening on 127.0.0.1:52450
1496315308072	geckodriver::marionette	INFO	Starting browser \\?\C:\mozilla\bin\nightly\firefox.exe with args ["-marionette"]

This results in a failure because of the following invalid path passed into `dir.create()`: \\?\C:\mozilla\bin\nightly\browser

So this is clearly geckodriver related. Moving to Testing/Marionette for now until we have the new component setup.

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 13 • 7 years ago

The bug is only visible when the Firefox binary gets passed via the `moz:firefoxOptions` capability on Windows. And as such regressed by the landing of the patch to support the new session command (https://github.com/mozilla/geckodriver/pull/568)

The exact problem lays here:
https://github.com/mozilla/geckodriver/commit/6f1e3c192463342a0a49f5f3f0af914ad0e1ae7a#diff-881ab48c5bc67f4e8c9d3ce5d7221613R45

Calling `canonicalize` on the path actually gives it this strange looking format. It's interesting that Windows is able to start the application, but Firefox itself is not able to handle it internally.

https://doc.rust-lang.org/std/path/struct.Path.html#method.canonicalize

Removing this call makes it working again.

So if Firefox cannot handle it we might have to avoid using this method.

James was there any reason why you have added this? Was there something broken before?

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 14 • 7 years ago

Benjamin, do you know if there is a bug about the problem in Firefox in not being able to handle a normalized path like "\\?\C:\mozilla\bin\nightly\firefox.exe"? Please see comment 12 for details. Thanks.

	James Graham [:jgraham]
	Comment 15 • 7 years ago

12:36 < jgraham> whimboo: https://doc.rust-lang.org/std/fs/fn.canonicalize.html all the normaization and symlink stripping part is useful. It seems unfortunate if Fx can't deal with a normalized windows path
12:37 < jgraham> In particular resolving symlinks makes application.ini accessible more often so we can inspect the Fx version more easily
12:38 < jgraham> whimboo: Can you make a patch that will strip the unrecognised prefix on Windows, if that's sufficient?

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 16 • 7 years ago

Could this issue actually be related to bug 393182?

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 17 • 7 years ago

So we are going to fix this in geckodriver now by not using the canonicalized path to start Firefox, but add that to mozversion so we can ensure to successfully retrieve the version information for path with symlinks.

The patch for mozversion is up here: https://github.com/jgraham/mozversion/pull/3

Once that got merged and the new crate published I will update geckodriver.

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 18 • 7 years ago

A new mozversion crate got published with the fix. I will now work on the geckodriver patch.

Please note that every component in Firefox which relies on XCurProcD is broken:
https://dxr.mozilla.org/mozilla-central/search?q=XCurProcD&=mozilla-central

So once this is fixed I would propose that we get a new release of geckodriver out.

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 20 • 7 years ago

Andreas, is there a way to test those changes yet via try? It would make me happier for landing.

	Andreas Tolfsen ❲:ato❳
	Comment 21 • 7 years ago
mozreview-review

Comment on attachment 8873805 [details]
Bug 1364385 - Do not use canonicalized path to start Firefox.

https://reviewboard.mozilla.org/r/145236/#review149226

This is fine, but please do a try run to ensure it compiles correctly.

Also, please make a separate commit for the dependency updates next time to make it easier for me to know what to look at.

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 22 • 7 years ago

(In reply to Andreas Tolfsen ‹:ato› from comment #21)
> This is fine, but please do a try run to ensure it compiles correctly.

That was my question in comment 20. So we clarified and I triggered a try. All looks fine.

	Pulsebot
	Comment 23 • 7 years ago

Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/bff272abb590
Do not use canonicalized path to start Firefox. r=ato

	Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)
	Comment 24 • 7 years ago
bugherder

https://hg.mozilla.org/mozilla-central/rev/bff272abb590

	Eric Tsai (no more review request accepted) Reporter
	Comment 25 • 7 years ago

Hi :whimboo, this patch is already landed, do I need a new geckodriver to work with Firefox Nightly?

	Henrik Skupin [:whimboo][⌚️UTC+1] Assignee
	Comment 26 • 7 years ago

Eric, you have to wait for the next release (see bug 1369709 for tracking it) or build it yourself from within `testing/geckodriver`.