- A one hour Sphinx tutorial introduction.
- By Christoph Deil and Stuart Mumford .
- March 21, 2016 at PyAstro16 .
You will learn how to add Sphinx documentation to a Python package (using the
example astrospam Python package in this repo).
The focus is exclusively on technical aspects how to work with Sphinx. We will not have time to talk about how to write good documentation, i.e. what content to create and how to structure it.
You are encouraged to follow along, i.e. try out every step on your computer after we demo it.
Before we start:
- Who has run Sphinx before (i.e. run
sphinx-buildormake htmlorpython setup.py build_sphinx)? - Who has set up Sphinx before (i.e. run
sphinx-quickstart, editeddocs/conf.py)?
If you have a question, or something isn't working for you, or if I'm going too fast, please feel free to interrupt us at any time!
We'll start with a quick overview of Sphinx and related things by having a look at the following web pages.
If you want to learn more, please go back and read the info on those pages after the tutorial on your own.
- We won't talk about this Sphinx. (I don't know why the Sphinx documentation generator was given that name.)
- To get some basic info on Sphinx, read the Wikipedia on Sphinx (documentation generator) or the welcome page of the Sphinx website at http://www.sphinx-doc.org/ .
- The most useful pages in the Sphinx documentation to get started are the Sphinx tutorial and the reStructuredText Primer
- Almost all Python projects use reStructured text (RST) and Sphinx for documentation. Examples: Python Astropy, Astroplan
- As the Wikipedia article on reStructuredText (RST)
explains, RST is a markup language (like LaTeX or Markdown) that is mostly used for Python docstrings (in
.pyfiles) and high-level documentation (in.rstfiles). - Sphinx is the tool that takes RST as input and produces HTML or PDF as output.
To be more precise, Sphinx is a Python package that is mostly used via the command line tools
sphinx-quickstartandsphinx-build(which again you typically invoke via aMakefile). - Python docstrings are extracted by the Sphinx "autodoc" feature to auto-generate API (application programming interface) docs. There's a few different formats for docstrings in use that Sphinx supports.
- The one all scientific Python packages (Numpy, Scipy, Astropy, ...) use is called the Numpy docstring standard which as added as a built-in Sphinx extension called sphinx.ext.napoleon (I don't know why it was called Napoleon.)
- Once documentation is set up for your package, it's typically easy to generate HTML
output by just running
make htmlwhich callssphinx-build, or by executingpython setup.py build_sphinxwhich runs the Sphinx build. Then you can look at the output by just opening upindex.htmlin some output directory where the HTML docs have been generated. Working on documentation is then a matter of editing.rstor.pyfiles, runningmake htmland checking the HTML output or console for errors and warnings. - Finally, if you want to host the generated HTML, the free https://readthedocs.org/ and https://pages.github.com/ services are good options. We won't have time to cover those today, feel free to ask us after if you want to learn how they work or want help to set it up for your project.
Let's go ahead with our hands-on introduction to Sphinx and start using it ...
Open a terminal and type sphinx<TAB>. If this lists sphinx-* commands
(e.g sphinx-quickstart or sphinx-build), you have Sphinx installed.
Type sphinx-build --version to check the Sphinx version number.
The latest stable version is 1.3. If you have 1.2 or older, I'd suggest you update now e.g. using:
$ pip install --upgrade sphinx $ conda install sphinx
Later on we'll use the sphinx_rtd_theme . Please install it now via:
$ pip install sphinx_rtd_theme
Before we continue, everyone please check that you're set up:
$ sphinx-build --version Sphinx (sphinx-build) 1.3.6 $ python -c 'import sphinx_rtd_theme' # Should give no output. # If you get an ImportError, `sphinx_rtd_theme` isn't installed correctly.
Let's say you have a Python project consisting of a few .py files,
and would like to use Sphinx to generate HTML or PDF documentation for it.
As an example for today's tutorial, please grab this repo:
$ git clone https://github.com/cdeil/sphinx-tutorial $ cd sphinx-tutorial
As you can see, there is a Python package called astrospam:
$ tree .
.
├── LICENSE
├── README.rst
└── astrospam
├── __init__.py
├── ham.py
├── pyastro16.py
└── spam.py
But there's no HTML documentation for it. Let's change that!
To add Sphinx documentation, you run sphinx-quickstart
This will prompt you for some information and then generate a few of files.
For most questions you can just hit ENTER to accept the default. These are
the questions where you don't take the default, but actually put something:
$ sphinx-quickstart Welcome to the Sphinx 1.3.6 quickstart utility. > Root path for the documentation [.]: docs > Project name: astrospam > Author name(s): Astrospam developers > Project version: 0.1 > autodoc: automatically insert docstrings from modules (y/n) [n]: y Finished: An initial directory structure has been created.
The tool created the following files:
docs/conf.py-- Sphinx configuration file (a Python file)docs/index.rst-- Name of your master docs page (a reStructuredText, aka RST file)docs/Makefile-- Makefile as convenience to run Sphinx (for Linux and Mac OS X)docs/make.bat-- Makefile for Windows
And the following empty directories:
docs/_build-- This is where all output files (e.g. HTML) will go when Sphinx runs.docs/_static-- A place for static files, e.g. images or css (we won't use it)docs/_templates-- A place for template files (we won't use it)
Now we're all set up to generate HTML docs:
$ cd docs $ make html sphinx-build -b html -d _build/doctrees . _build/html Running Sphinx v1.3.6 making output directory... loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [html]: targets for 1 source files that are out of date updating environment: 1 added, 0 changed, 0 removed reading sources... [100%] index looking for now-outdated files... none found pickling environment... done checking consistency... done preparing documents... done writing output... [100%] index generating indices... genindex writing additional pages... search copying static files... done copying extra files... done dumping search index in English (code: en) ... done dumping object inventory... done build succeeded. Build finished. The HTML pages are in _build/html.
Now open up _build/html/index.html in your webbrowser.
On Mac you can do:
$ open _build/html/index.html
Sphinx has generated a documentation webpage for you (with a sidebar, search field, main content area, footer)!
There's some other things you can do. Type make or make help to find out:
$ make Please use `make <target>' where <target> is one of html to make standalone HTML files dirhtml to make HTML files named index.html in directories singlehtml to make a single large HTML file pickle to make pickle files json to make JSON files htmlhelp to make HTML files and a HTML help project qthelp to make HTML files and a qthelp project applehelp to make an Apple Help Book devhelp to make HTML files and a Devhelp project epub to make an epub latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter latexpdf to make LaTeX files and run them through pdflatex latexpdfja to make LaTeX files and run them through platex/dvipdfmx text to make text files man to make manual pages texinfo to make Texinfo files info to make Texinfo files and run them through makeinfo gettext to make PO message catalogs changes to make an overview of all changed/added/deprecated items xml to make Docutils-native XML files pseudoxml to make pseudoxml-XML files for display purposes linkcheck to check all external links for integrity doctest to run all doctests embedded in the documentation (if enabled) coverage to run coverage check of the documentation (if enabled)
If you have pdflatex installed, you can try making a PDF version of your docs:
$ make latexpdf sphinx-build -b latex -d _build/doctrees . _build/latex Running Sphinx v1.3.6 making output directory... loading pickled environment... done building [mo]: targets for 0 po files that are out of date building [latex]: all documents updating environment: 0 added, 0 changed, 0 removed looking for now-outdated files... none found processing astrospam.tex... index resolving references... writing... done copying TeX support files... done build succeeded. Running LaTeX files through pdflatex... ... 1000 lines of horrible LaTeX log output ... Output written on astrospam.pdf (7 pages, 43725 bytes). Transcript written on astrospam.log. pdflatex finished; the PDF files are in _build/latex.
Open up _build/latex/astrospam.pdf and have a look:
$ open _build/latex/astrospam.pdf
We're all set up to write some documentation ...
Now let's write some documentation.
This is done by adding text to docs/index.rst, or by adding extra .rst
text files in docs and writing text using RST format there.
Writing documentation is a cycle similar to writing code:
- Edit
.rstfiles - Run
make html - Check output HTML files
For the following exercises, and generally while learning how to write RST, it's very helpful to have the "reStructuredText Primer" page from the Sphinx docs open: http://www.sphinx-doc.org/en/stable/rest.html
Let's do the documentation writing cycle once:
Edit
index.rstand add this line after the title:Hello world!
Run
make htmlRefresh the browser and watch the text appear in the HTML output.
Add a sub-section called "Getting started".
Add the paragraph "The
astrospammodule provides:" followed by a list with entriesHamandspam.Add a code example:
$ python >>> import astrospam >>> astrospam.spam() Spam Spam Spam >>> exit()
Add a sub-page
docs/tutorial.rstand copy & paste the following content there:Tutorial ======== This is the ``astrospam`` tutorial. Part 1 ------ Spam, spam, spam ... Part 2 ------ More spam! https://youtu.be/anwy2MPT5RE
Now add that page to the toctree
directive in index.rst:
.. toctree:: :maxdepth: 2 tutorial
Let's see what happens if we make an RST formatting mistake.
Remove some underline characters from the title:
Welcome to astrospam's documentation! ============================
Sphinx will emit a warning pointing out the file and line number where the problem is and give a helpful message what the problem is:
docs/index.rst:7: WARNING: Title underline too short.
The HTML output will still be OK .. it's just a warning.
Let's see what happens if you make an error in a Sphinx directive.
E.g. you could change the toctree directive in index.rst to toctreeeeeee:
.. toctreeeeeee:: :maxdepth: 2
Now you'll get an error an the TOC will be missing in the output.
Run these commands:
mkdir -p docs/_templates/autosummary wget https://raw.githubusercontent.com/astropy/package-template/a956da77759743b06db99d207b8e1e1a9eaf8a87/docs/_templates/autosummary/base.rst wget https://raw.githubusercontent.com/astropy/package-template/a956da77759743b06db99d207b8e1e1a9eaf8a87/docs/_templates/autosummary/class.rst wget https://raw.githubusercontent.com/astropy/package-template/a956da77759743b06db99d207b8e1e1a9eaf8a87/docs/_templates/autosummary/module.rst
TODO: link from docstrings to docs in RST file and the other way around.
Note that Sphinx autodoc imports the Python module and accesses
docstrings stored in __doc__ attributes. This means that
module-level and class-level code is executed.
TODO: Illustrate by adding print statements.
TODO: Add code that throws an exception (e.g. import spam or 1/0 or a SyntaxError)
and show the resulting Sphinx error message.
Explain about __all__
TODO: show how to change to the readthedocs template and what changes.
- We hope that this tutorial gave you a basic understanding of what Sphinx is, how it works, and how you use it to generate the documentation for Python projects.
- You should now be able to contribute to the documentation of existing Python projects and maybe even be able to set up Sphinx for your own package (e.g. by copy & pasting the working package-template setup).
- There's many things we didn't cover that will come up if you start contributing to Sphinx documentation for projects like Astropy or Astropy-affiliated packages: plot directive, setup.py integration, doctests, ...
- Sphinx, like other documentation generators such as LaTeX or Doxygen, is a very complicated, and extremely extensible and customisable tool. Even with years of experience you can easily get stuck with an uncomprehensible error message and get frustrated. Don't be shy to ask for help!