Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: unbreak dependency tracking, doc builds and multiline tables #499

Merged
merged 3 commits into from
Mar 15, 2024

Conversation

jktjkt
Copy link
Collaborator

@jktjkt jktjkt commented Mar 7, 2024

build: specify dependencies directly in setup.cfg

In tox v4, "reuse of environments" was disabled. This is then later explained to refer to exactly that thing which we were using for inheriting the dependencies from the top-level testenv all the way to the docs build. That's why the docs build in GitHub CI started failing.

IMHO, The Correct Way™ of specifying what dependencies are used for which feature are the so-called "extra dependencies". Once they are in place, it is be possible to install gnpy via, e.g., pip install oopt-gnpy[docs]. However, this process is (as far as I can tell) incompatible with requirements.txt; all my attempts at using the standard dependency syntax in that file have failed for me.

So, in order to make this happen, let's move all the dependencies from a more-or-less ad-hoc collection of files to this declarative approach right in setup.cfg. That way, the deps are listed on a single place, in a declarative manner, and as a result, the installation is now a trivial pip oneliner.

As a result, one can also remove that duplication of dependencies in docs requirements.

Fixes: 47c8962 fix docs requirements

docs: try to unbreak the readthedocs.io build

It was failing with a message:

Config validation error in build.os. Value os not found.

Apparently, the v2 config file is mandatory, so let's do that.

docs: use the default theme on ReadTheDocs.org as well

Historically, we've been using the RTD theme on the RTD site which hosts
the docs for us, and a Sphinx-default, "Alabaster" theme for other docs
builds. Doing that however started failing:

Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/html/__init__.py", line 1096, in handle_page
 output = self.templates.render(templatename, ctx)
	  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/readthedocs_ext/readthedocs.py", line 181, in rtd_render
 content = old_render(template, render_context)
	   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/jinja2glue.py", line 194, in render
 return self.environment.get_template(template).render(context)
	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/jinja2/environment.py", line 1301, in render
 self.environment.handle_exception()
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/jinja2/environment.py", line 936, in handle_exception
 raise rewrite_traceback_stack(source=source)
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/themes/basic/page.html", line 10, in top-level template code
 {%- extends "layout.html" %}
 ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/themes/classic/layout.html", line 10, in top-level template code
 {%- extends "basic/layout.html" %}
 ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/themes/default/../basic/layout.html", line 170, in top-level template code
 {%- block content %}
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/themes/default/../basic/layout.html", line 189, in block 'content'
 {%- block sidebar2 %}{{ sidebar() }}{% endblock %}
 ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/themes/default/../basic/layout.html", line 189, in block 'sidebar2'
 {%- block sidebar2 %}{{ sidebar() }}{% endblock %}
 ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/jinja2/sandbox.py", line 393, in call
 return __context.call(__obj, *args, **kwargs)
	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/jinja2/runtime.py", line 777, in _invoke
 rv = self._func(*arguments)
      ^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/themes/default/../basic/layout.html", line 63, in template
 {%- include sidebartemplate %}
 ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/jinja2glue.py", line 215, in get_source
 raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: about.html

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/cmd/build.py", line 281, in build_main
 app.build(args.force_all, args.filenames)
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/application.py", line 347, in build
 self.builder.build_update()
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 310, in build_update
 self.build(to_build,
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 376, in build
 self.write(docnames, list(updated_docnames), method)
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 571, in write
 self._write_serial(sorted(docnames))
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 581, in _write_serial
 self.write_doc(docname, doctree)
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/html/__init__.py", line 672, in write_doc
 self.handle_page(docname, ctx, event_arg=doctree)
File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/html/__init__.py", line 1103, in handle_page
 raise ThemeError(__("An error happened in rendering the page %s.\nReason: %r") %
sphinx.errors.ThemeError: An error happened in rendering the page about-project.
Reason: TemplateNotFound('about.html')

Theme error:
An error happened in rendering the page about-project.
Reason: TemplateNotFound('about.html')

I have no clue what that means because we have never requested this
about.html, nor do we reference that file from anywhere. Chances are
that it's "just" some version pinning/compatibility issue, but hey --
why mess with that when there's a perfectly good default theme that
we're using for other purposes already.

As a side effect, this also solves that long-standing issue that Esther
reported where the tables have overly long lines. Apparently, it's a
theme-specific misfeature (readthedocs/sphinx_rtd_theme/#117), and the
Alabaster one doesn't suffer from that.

All hail alabaster!

@jktjkt jktjkt force-pushed the requirements-rework branch 2 times, most recently from 8cbed34 to e980ea1 Compare March 7, 2024 11:08
In tox v4, "reuse of environments" was disabled [1]. This is then later
explained [2] to refer to exactly that thing which we were using for
inheriting the dependencies from the top-level testenv all the way to
the docs build. That's why the docs build in GitHub CI started failing.

IMHO, The Correct Way™ of specifying what dependencies are used for
which feature are the so-called "extra dependencies". Once they are in
place, it is be possible to install gnpy via, e.g., `pip install
oopt-gnpy[docs]`. However, this process is (as far as I can tell)
incompatible with `requirements.txt`; all my attempts at using the
standard dependency syntax in that file have failed for me.

So, in order to make this happen, let's move all the dependencies from a
more-or-less ad-hoc collection of files to this declarative approach
right in setup.cfg. That way, the deps are listed on a single place, in
a declarative manner, and as a result, the installation is now a trivial
pip oneliner.

As a result, one can also remove that duplication of dependencies in
docs requirements.

[1] https://tox.wiki/en/4.11.4/upgrading.html#reuse-of-environments
[2] https://tox.wiki/en/4.11.4/upgrading.html#packaging-configuration-and-inheritance

Change-Id: I34aa0c71e993b39e2b805a7de40e133b4d290318
Fixes: 47c8962 fix docs requirements
It was failing with a message:

  Config validation error in build.os. Value os not found.

Apparently, the v2 config file is mandatory, so let's do that.

Change-Id: I267d5314db026de532b2b6644f500d25de08e343
Historically, we've been using the RTD theme on the RTD site which hosts
the docs for us, and a Sphinx-default, "Alabaster" theme for other docs
builds. Doing that however started failing:

 Traceback (most recent call last):
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/html/__init__.py", line 1096, in handle_page
     output = self.templates.render(templatename, ctx)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/readthedocs_ext/readthedocs.py", line 181, in rtd_render
     content = old_render(template, render_context)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/jinja2glue.py", line 194, in render
     return self.environment.get_template(template).render(context)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/jinja2/environment.py", line 1301, in render
     self.environment.handle_exception()
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/jinja2/environment.py", line 936, in handle_exception
     raise rewrite_traceback_stack(source=source)
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/themes/basic/page.html", line 10, in top-level template code
     {%- extends "layout.html" %}
     ^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/themes/classic/layout.html", line 10, in top-level template code
     {%- extends "basic/layout.html" %}
     ^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/themes/default/../basic/layout.html", line 170, in top-level template code
     {%- block content %}
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/themes/default/../basic/layout.html", line 189, in block 'content'
     {%- block sidebar2 %}{{ sidebar() }}{% endblock %}
     ^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/themes/default/../basic/layout.html", line 189, in block 'sidebar2'
     {%- block sidebar2 %}{{ sidebar() }}{% endblock %}
     ^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/jinja2/sandbox.py", line 393, in call
     return __context.call(__obj, *args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/jinja2/runtime.py", line 777, in _invoke
     rv = self._func(*arguments)
          ^^^^^^^^^^^^^^^^^^^^^^
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/themes/default/../basic/layout.html", line 63, in template
     {%- include sidebartemplate %}
     ^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/jinja2glue.py", line 215, in get_source
     raise TemplateNotFound(template)
 jinja2.exceptions.TemplateNotFound: about.html

 The above exception was the direct cause of the following exception:

 Traceback (most recent call last):
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/cmd/build.py", line 281, in build_main
     app.build(args.force_all, args.filenames)
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/application.py", line 347, in build
     self.builder.build_update()
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 310, in build_update
     self.build(to_build,
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 376, in build
     self.write(docnames, list(updated_docnames), method)
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 571, in write
     self._write_serial(sorted(docnames))
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/__init__.py", line 581, in _write_serial
     self.write_doc(docname, doctree)
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/html/__init__.py", line 672, in write_doc
     self.handle_page(docname, ctx, event_arg=doctree)
   File "/home/docs/checkouts/readthedocs.org/user_builds/gnpy/envs/499/lib/python3.12/site-packages/sphinx/builders/html/__init__.py", line 1103, in handle_page
     raise ThemeError(__("An error happened in rendering the page %s.\nReason: %r") %
 sphinx.errors.ThemeError: An error happened in rendering the page about-project.
 Reason: TemplateNotFound('about.html')

 Theme error:
 An error happened in rendering the page about-project.
 Reason: TemplateNotFound('about.html')

I have no clue what that means because we have never requested this
`about.html`, nor do we reference that file from anywhere. Chances are
that it's "just" some version pinning/compatibility issue, but hey --
why mess with that when there's a perfectly good default theme that
we're using for other purposes already.

As a side effect, this also solves that long-standing issue that Esther
reported where the tables have overly long lines. Apparently, it's a
theme-specific misfeature (readthedocs/sphinx_rtd_theme/Telecominfraproject#117), and the
Alabaster one doesn't suffer from that.

All hail alabaster!

Change-Id: I857890f29f14b7c0f66bca201c9a9c1b1cbf8841
@jktjkt jktjkt marked this pull request as ready for review March 13, 2024 20:38
@jktjkt jktjkt changed the title build: unbreak dependency tracking build: unbreak dependency tracking, doc builds and multiline tables Mar 13, 2024
Copy link
Collaborator

@EstherLerouzic EstherLerouzic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bravo !

@jktjkt jktjkt merged commit 5b6f8c6 into Telecominfraproject:master Mar 15, 2024
17 checks passed
@jktjkt jktjkt deleted the requirements-rework branch March 15, 2024 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants