User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0) Gecko/20100101 Firefox/106.0

Steps to reproduce:

Run mach with any command outside MozillaBuild on a non-Unicode Windows environment. I was able to reproduce this on CPython 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] on win32, running on a standard MSYS2 instance.

Actual results:

Fails with error:

Error running mach:

    ['bootstrap']

The error occurred in mach itself. This is likely a bug in mach itself or a
fundamental problem with a loaded module.

You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file general| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

UnicodeDecodeError: 'cp949' codec can't decode byte 0xef in position 8840: illegal multibyte sequence

  File "C:\mozilla-source\mozilla-unified\python\mach\mach\main.py", line 399, in run
    return self._run(argv)
  File "C:\mozilla-source\mozilla-unified\python\mach\mach\main.py", line 488, in _run
    report_invocation_metrics(context.telemetry, handler.name)
  File "C:\mozilla-source\mozilla-unified\python\mach\mach\telemetry.py", line 89, in report_invocation_metrics
    filter_args(command, sys.argv, instance.topsrcdir, instance.topobjdir)
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\base.py", line 259, in topobjdir
    self._topobjdir = self.resolve_mozconfig_topobjdir(
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\base.py", line 201, in resolve_mozconfig_topobjdir
    topobjdir = self.mozconfig["topobjdir"] or default
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\base.py", line 356, in mozconfig
    return self.base_mozconfig_info["mozconfig"]
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\base.py", line 346, in base_mozconfig_info
    return self.get_base_mozconfig_info(
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\util.py", line 1052, in __call__
    self[args] = self.func(*args)
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\base.py", line 331, in get_base_mozconfig_info
    sandbox._value_for(sandbox["mozconfig_options"])
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\configure\__init__.py", line 615, in _value_for
    return self._value_for_depends(self._depends[obj])
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\util.py", line 1061, in method_call
    cache[args] = self.func(instance, *args)
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\configure\__init__.py", line 627, in _value_for_depends
    value = obj.result()
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\util.py", line 1061, in method_call
    cache[args] = self.func(instance, *args)
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\configure\__init__.py", line 162, in result
    resolved_args = [self.sandbox._value_for(d) for d in self.dependencies]
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\configure\__init__.py", line 162, in <listcomp>
    resolved_args = [self.sandbox._value_for(d) for d in self.dependencies]
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\configure\__init__.py", line 618, in _value_for
    return self._value_for_depends(obj)
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\util.py", line 1061, in method_call
    cache[args] = self.func(instance, *args)
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\configure\__init__.py", line 627, in _value_for_depends
    value = obj.result()
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\util.py", line 1061, in method_call
    cache[args] = self.func(instance, *args)
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\configure\__init__.py", line 163, in result
    return self._func(*resolved_args)
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\configure\__init__.py", line 1310, in wrapped
    return new_func(*args, **kwargs)
  File "C:/mozilla-source/mozilla-unified/build/moz.configure/init.configure", line 86, in mozconfig
    mozconfig = loader.read_mozconfig(mozconfig)
  File "c:\mozilla-source\mozilla-unified\python\mozbuild\mozbuild\mozconfig.py", line 161, in read_mozconfig
    subprocess.check_output(
  File "C:\Users\basix\scoop\apps\python\current\lib\subprocess.py", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "C:\Users\basix\scoop\apps\python\current\lib\subprocess.py", line 503, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "c:\mozilla-source\mozilla-unified\third_party\python\sentry_sdk\sentry_sdk\integrations\stdlib.py", line 223, in sentry_patched_popen_communicate
    return old_popen_communicate(self, *a, **kw)
  File "C:\Users\basix\scoop\apps\python\current\lib\subprocess.py", line 1139, in communicate
    stdout = self.stdout.read()

Expected results:

It should successfully read the mozconfig and run normally.

I

I tried using Python 3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)] on win32. The workaround I could find was:

• Enable Python UTF8 Mode
• Edit python/mozbuild/mozbuild/mozconfig.py manually

If mozconfig is expected to be always UTF-8 I'd like to work on the patch of this issue.

Looks like we do always expect utf-8.

In general, I highly recommend going down the default path and using MozillaBuild with MSYS2. Do you have a specific need for doing what you're doing?

I have an MSYS2 environment tailored to suit my needs, and I noticed there's an experimental mode for running outside MozillaBuild. I'm happy with using MozillaBuild for now though.

This prevents certain Windows Python installations from using the locale encoding.

https://hg.mozilla.org/mozilla-central/rev/60e99446dac9