mach doc is busted with python 3.10
Categories
(Developer Infrastructure :: Source Documentation, defect)
Tracking
(firefox110 fixed)
Tracking | Status | |
---|---|---|
firefox110 | --- | fixed |
People
(Reporter: emilio, Assigned: Sylvestre)
References
(Blocks 2 open bugs)
Details
Attachments
(5 files)
Traceback (most recent call last):
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/registry.py", line 429, in load_extension
mod = import_module(extname)
File "/usr/lib64/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx_js/__init__.py", line 1, in <module>
from .directives import (auto_class_directive_bound_to_app,
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx_js/directives.py", line 13, in <module>
from .renderers import AutoFunctionRenderer, AutoClassRenderer, AutoAttributeRenderer
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx_js/renderers.py", line 11, in <module>
from .parsers import PathVisitor
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx_js/parsers.py", line 3, in <module>
from parsimonious import Grammar, NodeVisitor
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/parsimonious/__init__.py", line 8, in <module>
from parsimonious.grammar import Grammar, TokenGrammar
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/parsimonious/grammar.py", line 8, in <module>
from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib64/python3.10/collections/__init__.py)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/cmd/build.py", line 276, in build_main
app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/application.py", line 237, in __init__
self.setup_extension(extension)
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/application.py", line 394, in setup_extension
self.registry.load_extension(self, extname)
File "/home/emilio/src/moz/gecko-3/obj-debug/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/registry.py", line 432, in load_extension
raise ExtensionError(__('Could not import extension %s') % extname,
sphinx.errors.ExtensionError: Could not import extension sphinx_js (exception: cannot import name 'Mapping' from 'collections' (/usr/lib64/python3.10/collections/__init__.py))
Extension error:
Could not import extension sphinx_js (exception: cannot import name 'Mapping' from 'collections' (/usr/lib64/python3.10/collections/__init__.py))
Reporter | ||
Comment 1•2 years ago
|
||
Updating parsimonious
should do, but I think we also need to update https://github.com/mozilla/sphinx-js to use the newer version of parsimonious
.
Comment 2•2 years ago
|
||
The severity field is not set for this bug.
:Sylvestre, could you have a look please?
For more information, please visit auto_nag documentation.
Assignee | ||
Comment 3•2 years ago
|
||
yeah, we should fix it.
I noticed that lonnen has been involved \o/
Updated•2 years ago
|
Comment 4•1 year ago
|
||
It appears rather complex to update sphinx-js to work with modern python
versions (3.10 and more recent). In the meantime, and as a work around,
developers that wish to run ./mach docs
to preview regular (non-js) docs can
apply this patch.
Assignee | ||
Updated•1 year ago
|
Assignee | ||
Comment 5•1 year ago
•
|
||
I tried to update sphinx-js with this patch: ``` Traceback (most recent call last): File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/cmd/build.py", line 280, in build_main app.build(args.force_all, filenames) File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/application.py", line 344, in build self.builder.build_update() File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 294, in build_update self.build(to_build, File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 358, in build self.write(docnames, list(updated_docnames), method) File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 529, in write self._write_parallel(sorted(docnames), File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 566, in _write_parallel doctree = self.env.get_and_resolve_doctree(docname, self) File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/environment/__init__.py", line 535, in get_and_resolve_doctree self.apply_post_transforms(doctree, docname) File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/environment/__init__.py", line 581, in apply_post_transforms transformer.apply_transforms() File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/transforms/__init__.py", line 86, in apply_transforms super().apply_transforms() File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/docutils/transforms/__init__.py", line 171, in apply_transforms transform.apply(**kwargs) File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/transforms/post_transforms/__init__.py", line 43, in apply self.run(**kwargs) File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/transforms/post_transforms/__init__.py", line 96, in run newnode = self.resolve_anyref(refdoc, node, contnode) File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/transforms/post_transforms/__init__.py", line 140, in resolve_anyref results.extend(domain.resolve_any_xref(self.env, refdoc, self.app.builder, File "/home/sylvestre/dev/mozilla/mozilla-central.hg/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/lib/python3.10/site-packages/sphinx/domains/javascript.py", line 447, in resolve_any_xref return [('js:' + self.role_for_objtype(obj[2]), TypeError: can only concatenate str (not "NoneType") to str ```
Comment hidden (duplicate) |
Comment 8•1 year ago
|
||
That error doesn't ring a bell specifically, but I do recall I too tried to update it once and couldn't get it working. Although I think I couldn't get the packages to resolve so looks like you got further than me!
I'm not familiar enough with Sphinx to know what's going on off the top of my head, but if we can add some print statements to that role_for_objtype
function, maybe we can narrow it down. Unfortunately I won't have time for this until after the holidays.
Comment 9•1 year ago
|
||
I can take a look. If it helps, assign the bug to me and I can chase it down in the next week.
Comment 10•1 year ago
|
||
I have an up-to-date repository and I apply that patch. Seems like I have to recompile requirements.txt
(making sure to use Python 3.9). That doesn't work initially because importlib_metadata
also needs to be updated. I update that and then recompile requirements.txt
.
Then when I run mach doc
, it looks like it's pulling in really old versions of libraries from third_party/python/
and errors out:
sphinx 5.3.0 has requirement importlib-metadata>=4.8; python_version < "3.10", but you have importlib-metadata 1.7.0.
sphinx 5.3.0 has requirement Jinja2>=3.0, but you have jinja2 2.11.3.
sphinx 5.3.0 has requirement packaging>=21.0, but you have packaging 20.9.
sphinx-js 3.2.0 has requirement markupsafe==2.0.1, but you have markupsafe 1.1.1.
Those are the versions in third_party/python/
and not the versions specified in requirements.in
.
Sylvestre: How're you testing the patch?
Assignee | ||
Comment 11•1 year ago
|
||
as i am in testing mode, I did:
rm -rf
on these directories
;)
Comment 12•1 year ago
|
||
I don't see how either deleting everything in third_party/python/
or deleting the specific packages that are having conflicts works since it looks like packaging
is needed by mach
to run. I'll spend some more time trying to figure out how to set everything up in order to get the error in comment #5, but if I can't get it today, then I don't think I can help here until someone gives me steps to reproduce.
Assignee | ||
Comment 13•1 year ago
|
||
well, i only deleted a couple directories, not all.
Here is the patch:
https://phabricator.services.mozilla.com/D164899 (it is ugly ;)
and with python 3.10
Comment 14•1 year ago
|
||
That helps a lot! I can reproduce the issue. It's this:
https://github.com/mozilla/sphinx-js/pull/178
It's one of the things that changed in sphinx-js between 3.0.1 (what firefox docs are using now) and 3.2.0 (what I just released). I'll work on fixing that in sphinx-js and do another release.
Comment 15•1 year ago
|
||
I pushed out sphinx-js 3.2.1 just now which fixes the issue in comment #5. I then did a mach doc
using sphinx-js 3.2.1 and it now completes with no errors (but a TON of warnings--wowzers!).
Assignee | ||
Comment 16•1 year ago
|
||
Assignee | ||
Comment 17•1 year ago
|
||
Updated•1 year ago
|
Updated•1 year ago
|
Comment 18•1 year ago
|
||
Pushed by sledru@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/3b6e51a87baf bump MarkupSafe in tree r=firefox-source-docs-reviewers,ahal https://hg.mozilla.org/integration/autoland/rev/281555ad04ca Update sphinx-js to allow build with python 3.10 r=firefox-source-docs-reviewers,ahal
Comment 19•1 year ago
|
||
bugherder |
Description
•