How can I generate Python API Doc from Rust PyO3 code?

[bachue]

Can I use pydoc or pdoc or something else to generate Python api documents from rust code which uses PyO3? Thanks.

[adamreichold]

I think the main point here is that tools targeting Python documentation will not be able to process the Rust source code. So generally, you need to build your extension module and then use Python's introspection facilities to extract the documentation embedded in the extension.

For Sphinx this is done using the autodoc extension. pydoc seems to work in a similar manner, so building your module using e.g. maturin develop so that pydoc can import it should suffice.

[mhils]

pdoc can pick up type annotations from .pyi files since earlier today (full disclosure: I'm the current maintainer), which makes it relatively well suited for documenting PyO3 projects. You can author docstrings in Rust (PyO3 exposes them as __doc__), and then provide a matching .pyi file with type stubs:

lib.rs

/// Start a WireGuard server.
#[pyfunction]
fn start_server(
    py: Python<'_>,
    host: String,
    port: u16,
    private_key: String,
    peer_public_keys: Vec<String>,
    handle_connection: PyObject,
    receive_datagram: PyObject,
) -> PyResult<&PyAny> {
    /* ... */
}

my-library.pyi

from typing import Awaitable, Callable

async def start_server(
    host: str,
    port: int,
    private_key: str,
    peer_public_keys: list[str],
    handle_connection: Callable[[asyncio.StreamReader, asyncio.StreamWriter], Awaitable[None]],
    receive_datagram: Callable[[bytes, tuple[str, int], tuple[str, int]], None],
) -> WireguardServer: ...

⬇️

output.html

---

I should mention that this uses a yet-unreleased version of maturin (to include PyO3/maturin#886), but given the current release cadence that should be a matter of weeks. :)

[Haaroon]

yeah unfortunately maturin doesnt output pyi files so this wont work

[mhils]

We successfully use this approach for mitmproxy_rs:

• ffi/mitmproxy_rs/__init__.pyi is maintained manually
• .github/workflows/docs.yml#L44 ensures that the pyi file does not go out of sync
• rendered docs are here

[Haaroon]

Hello, So we have been able to do this without having to add an pyi file.

We use ReadTheDocs to host our documentation. Using this we use Sphinx and the sphinx.ext.autosummary sphinx.ext.autodoc and sphinx.ext.napoleon built-in extensions to generate and make the docs look very pretty. We then set a number of variables in the conf.py sphinx file

We have an __init__.py file located in the root of a module which lists some extra doc features we are missing.

Finally the sphinx api.rst doc file just manually lists the modules.

In our build pipeline we install the package into readthedocs environment in the pre-build script, just before it generates the docs.

[HunkBenny]

I generated .pyi-files by using this crate.
Definitely take a look at the examples!