sphinx-autobuild goes into infinite loop

[rgaiacs]

When I worked in #821, I noticed that sphinx-autobuild would go into infinite loop after a first change is done by the user. I looked into reported issues for sphinx-autobuild and found sphinx-doc/sphinx-autobuild#187. I experimented with a few minimal working examples, see sphinx-doc/sphinx-autobuild#187 (comment), and with some changes in this team-compass repository and my conclusion is that

team-compass/docs/conf.py

Lines 64 to 67 in 47543e3

	from subprocess import run
	run(["python", "_data/contributors/gen_contributors.py"], check=True)
	run(["python", "_data/outreachy/outreachy_participants.py"], check=True)

are responsible for trigger a infinite loop in sphinx-autobuild. I don't remember where I read that add file creation in conf.py is a anti-pattern and, instead, people should create their own Sphinx extension.

The cross road here is that instructions in conf.py make it easy to contributors to find it but it make "impossible" for contributors to use sphinx-autobuild. IMHO, contributions to the documentation should be the low hanging fruit of a project and new contributors should have a good experience. My recommendation is to refactor https://github.com/jupyterhub/team-compass/blob/47543e390dc162502ab3c2150481c2607bca919f/docs/_data/contributors/gen_contributors.py and https://github.com/jupyterhub/team-compass/blob/47543e390dc162502ab3c2150481c2607bca919f/docs/_data/outreachy/outreachy_participants.py into a Sphinx extension to allow contributors to use sphinx-autobuild.

I'm interested to hear the thoughts of others.

[minrk]

Shouldn't the ignore in #821 prevent that?

We could move it to an extension, but IIUC it would still run every time unless we add logic to check out-of-date-ness (which would also prevent it in conf.py).

(Makefile rules address this, but make isn't run on Windows or readthedocs, which I think is why it went in conf.py)

[rgaiacs]

I believe the ignore instruction should prevent it but it is not working. And as far as I remember (confirmation required), the ignore was also not working before the changes in #821.

The problem is not the script to run on every build. The problem is the script to change a file and trigger a infinite build loop.

Moving to a Sphinx extension, means that no file is change because the data is build in memory. This avoids the file change and the infinite build loop.

I believe that now make run on ReadTheDocs CI as it is very configurable.