François Marier [:francois] Reporter
	Description • 7 years ago

In order to get better debugging output when tests fail, we would like to define extra environment variables in the Safe Browsing tests.

We could do it globally, but then it would pollute the log of tests that don't need it.

Henrik suggested passing them via self.marionette.instance.env:
https://bugzilla.mozilla.org/show_bug.cgi?id=1386013#c8

	Henrik Skupin [:whimboo][⌚️UTC+1]
	Comment 1 • 7 years ago

The env variables in use which are hard-coded right now can be found here:

https://dxr.mozilla.org/mozilla-central/rev/52285ea5e54c73d3ed824544cef2ee3f195f05e6/testing/marionette/client/marionette_driver/geckoinstance.py#232-243

What we might need is an env property on the GeckoInstance class which would allow us to add custom variables.

	Andreas Tolfsen ❲:ato❳
	Comment 2 • 6 years ago

Can you explain what solution you’re looking for, and for what tests?  I have a hard time understanding the scope and impact of this bug.

	François Marier [:francois] Reporter
	Comment 3 • 6 years ago

A specific example is in https://bugzilla.mozilla.org/show_bug.cgi?id=1386013#c0: setting MOZ_LOG="UrlClassifierStreamUpdater:1" before running testing/firefox-ui/tests/functional/safebrowsing/test_initial_download.py

That would give us to see more debug information in the test log when the test fails.

	Andreas Tolfsen ❲:ato❳
	Comment 4 • 6 years ago

So it just seems we need to make the harness inherit the existing
environment.  That seems like a simple enough solution.

	Andreas Tolfsen ❲:ato❳
	Comment 5 • 6 years ago

It looks like the existing environment is already used:

    https://searchfox.org/mozilla-central/source/testing/marionette/client/marionette_driver/geckoinstance.py#301

	Andreas Tolfsen ❲:ato❳
	Comment 6 • 6 years ago

I can confirm that passing environment variables to Firefox works:

> % MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 ./mach marionette test --gecko-log -
>  0:00.00 INFO Using workspace for temporary data: "/home/ato/src/gecko"
>  0:00.00 mozversion INFO application_buildid: 20180226131555
>  0:00.00 mozversion INFO application_display_name: Nightly
>  0:00.00 mozversion INFO application_id: {ec8030f7-c20a-464f-9b0e-13a3a9e97384}
>  0:00.00 mozversion INFO application_name: Firefox
>  0:00.00 mozversion INFO application_remotingname: firefox
>  0:00.00 mozversion INFO application_vendor: Mozilla
>  0:00.00 mozversion INFO application_version: 60.0a1
>  0:00.00 mozversion INFO platform_buildid: 20180226131555
>  0:00.00 mozversion INFO platform_version: 60.0a1
>  0:00.01 INFO Application command: /home/ato/src/gecko/obj-x86_64-pc-linux-gnu/dist/bin/firefox -no-remote -marionette -profile /tmp/tmpPKYKe9.mozrunner
> libGL error: No matching fbConfigs or visuals found
> libGL error: failed to load driver: swrast
> FATAL ERROR: Non-local network connections are disabled and a connection attempt to tiles.services.mozilla.com (34.211.99.53) was made.
> You should only access hostnames available via the test networking proxy (if running mochitests) or from a test-specific httpd.js server (if running xpcshell tests). Browser services should be disabled or redirected to a local server.
> ExceptionHandler::GenerateDump cloned child 11414
> ExceptionHandler::SendContinueSignalToChild sent continue signal to child
> ExceptionHandler::WaitForContinueSignal waiting for continue signal...
> Sandbox: Unexpected EOF, op 2 flags 00 path /home/ato/src/gecko/dom/locales/en-US/chrome/layout/xbl.properties
> ^?mach interrupted by signal or user action. Stopping.

	Henrik Skupin [:whimboo][⌚️UTC+1]
	Comment 7 • 6 years ago

Andreas, keep in mind that we don't want that for all tests in the suite but only to specific ones from a given component like subtests. So it has to work from the TestCase class.

	Andreas Tolfsen ❲:ato❳
	Comment 8 • 6 years ago

Oh I see.  This would be super-easy with a pytest fixture, but alas we can’t use those.