Make-like incremental build

[woutermont]

I'm just discovering this amazing tool, and one thing that surprised me much was that it seems to be building everything anew even for the tiniest change. Would be awesome if soupault or a plugin (pre-parse hook maybe?) could compare the modified dates and only generate what's new! (Neither a Lua coder nor an OCaml one myself, alas.)

[dmbaturin]

There's caching for page preprocessor and other external commands output: https://soupault.app/reference-manual/#caching
Executing external commands is usually the worst offender in that regard, I find that enough of an improvement for my own websites, although I can see how for really large sites it may not be enough.

In soupault's approach, cache invalidation is a pretty tricky problem. For example, any config change may potentially invalidate the entire cache because there may be a widget that inserts something in every page. It gets more interesting down the line: e.g., if a page has changed, it may require rebuild of an index page of its section — if indexing is enabled for the dir where it lives and the index view doesn't exclude it. Any invalidation algorithm is a compromise in that situation — there may be a point when rebuilding a page is faster than trying to find out with certainty if its cached version is still valid.

My reasoning was that the consequences of stale data (even just user frustration from trying to figure out why their change has no effect) are worse than somewhat longer build times, most of the time.

I agree that ideally things that don't need a rebuild should not be rebuild, but I'm not sure what the best compromise is. If you have ideas for that, I'm happy to hear them. The only hard constraint on the cache invalidation algorithm that I don't want to violate is that it should never use stale data, I'm open to any suggestions that maintain that invariant.

[dmbaturin]

You may want to read https://soupault.app/blog/soupault-4.4.0-release-and-ssg-caching/ for the survey of my findings about existing SSGs.

[woutermont]

Alright, I see the complexity and will look into existing possibilities :)

[linsyking]

For the incremental build of assets, it's possible to use makefile.

In my soupault.toml:

[asset_processors]
  svg = "SRC={{source_file_path}} DST={{target_dir}}/{{source_file_base_name}}.png TOOL=SVG make"
  ipe = "SRC={{source_file_path}} DST={{target_dir}}/{{source_file_base_name}}.png TOOL=IPE make"

In my Makefile:

${DST}: ${SRC}
ifeq "${TOOL}" "SVG"
	convert -background none ${SRC} ${DST}
endif
ifeq "${TOOL}" "IPE"
	iperender -png -resolution 200 -transparent ${SRC} ${DST}
endif

make will automatically check if it needs to rebuild the assets.