do not regenerate every file each time

[florianhumblot]

When writing documentation with Sphinx we often need to rebuild the output to verify if the page layout etc.. are as we expect.
This isn't usually problematic and is relatively quick. However, when we enable exhale it forcefully re-generates the entire API documentation each time make html is invoked.

I've seen in the backlog of issues that the project basically would need a rewrite to support partial re-generation of docs, but I haven't found a way to temporarily disable it so that only the non-generated docs get rebuilt.

[florianhumblot]

Never mind, I had to comment both breathe and exhale out of the conf.py extension list and do a clean.

[florianhumblot]

Would still be nice to prevent regenerating the docs while keeping a version so that the links still work though.

[svenevs]

Re-opening, part of #10. Not regenerating is also a component of #94. Added some more notes there.

I think if I skip regenerating things that have the same hash then that could help a lot. It may produce false no-rebuilds though, if a given document is not reprocessed and the underlying documentation for the construct is updated (doxygen comment changed). That said, for iterative development what exhale does (spew new documents every time) is extremely inconvenient.

[florianhumblot]

It may produce false no-rebuilds though, if a given document is not reprocessed and the underlying documentation for the construct is updated (doxygen comment changed).

What might work is a local cache of hashes for each XML file? Or even the .rst files? And then only overwrite the files who changed relative to the currently existing .rst files. That way it would "only" be an extra processing step at the end of the normal breathe/exhale pipeline instead of needing to rewrite how exhale works

This would prevent sphinx from writing output, which, as far as I can tell, is the slowest part of the build-process.

[svenevs]

Wanted to add a quick follow-up, this is a good suggestion. I'll need to learn how to interface with the sphinx environment caching to implement this. But once I've got a handle on that the scheme you propose seems like it should be straightforward.

[florianhumblot]

just stumbled across this: https://docs.python.org/3/library/filecmp.html
So what might work:

1. Put all of exhale / breathe output in a temporary folder.
2. dircmp/filecmp the directories to check if the XML is updated.
   i. if they're identical stop the flow here since nothing has changed
   ii. if they're not identical, generate the necessary .rst files in a temporary directory. (optionally dircmp /filecmp this step too)
3. copy the .rst files (optionally only those that need copying) to the actual output directory, overwriting the old files, triggering sphinx to regenerate the necessary output.

Thins I'm not sure about:

• If a new .rst file is generated, that could mean links to it break and/or it links to files that might not be available.
• If a .xml/.rst file from the output disappears, should this trigger a full-copy to trigger a full-rebuild from sphinx?
• Same goes for a new file, if a new file is detected, it might replace links or something in other files, so maybe a full-rebuild should be triggered.

but just adding steps 1&2 would speed up the builds considerably whenever you're not changing the C/C++ code/docstrings.

[svenevs]

This one is going to have to wait since commenting out exhale from extensions list is a viable workaround. The changes here are substantive and would need testing.