[feature] Render page templates as jingoo templates with fields from index

[ryanartecona]

This is an idea for an enhancement. I have no idea how invasive a change this would be, but it would have avoided some complexity that otherwise I had to shove into a plugin.

My use case is I have some metadata about each post I want to render via a template with a certain structure so I can style it how I want with css. I made a plugin uses the same selectors I use in my index fields and imperatively moves some html elements around, but it takes a lot more effort for a simple use case that could be handled by a template.

Jekyll and related engines use yaml front-matter to define per-post metadata which then become variables that can be interpreted into templated "layouts" (jekyll layout docs). Soupault already has a nice and flexible way of pulling out per-post metadata from custom tags emedded into each post via index field definitions, but currently those can only be fed to an "index view" on an "index page", with no way to directly use the same fields within the post template itself.

Soupault already uses jingoo templates in a couple place, so it seems a natural extension would be to allow page templates to use jingoo syntax with variables that come from index fields.

[dmbaturin]

I wouldn't want to add it as a built-in, but I have an idea for a future release that will greatly simplify the task.

Why I wouldn't want it as a built-in?

• Different people will likely want different behaviour from it (e.g. different sets of available variables, so there needs to be a way to fetch data at that stage).
• If it's a built-in, I'll have to take responsibility for its correct operation across all possible inputs, which is unrealistic.
• There should be a certain level of granularity, but it's difficult to provide without operating on the tree rather than text; but it's also difficult to implement that as a widget since it will require a way to schedule widgets after index extraction.

The idea of what to do instead is to add support for hooks that run at different points of page processing. Hook files will be discovered and loaded more or less like plugins. Here's a mockup of a hook that will solve your problem in a simple way:

[hooks.pre-save]
  hook = "render-template"

String.render_template(page_text, index_entry)

The pre-save hooks will fire up when the page element tree is rendered as text and is about to be saved to disk—seems like a perfect time to do template processing using index entry variables. But! Suppose you have a page that discusses templates and has something like this:

<p>This is what a template variable looks like: <code>{{foo}}</code></p>.

If you feed the entire page to the template processor, it will indiscriminately render those blocks and ruin (or fail on) examples that aren't to be rendered. Unless the user is careful enough to wrap them into a {% raw %} every time.

Instead, it's better done by a pre-render hook that can walk through fake elements like <tmpl> and render their content with a template processor:

[hooks.pre-render]
  hook = "render-template"
  selector = "tmpl"

-- Render template tags inside <tmpl> </tmpl> pseudo-elements and unwrap their content

function render(tmpl_elem, env)
  tmpl_data = HTML.inner_html(tmpl_elem)
  tmpl_out = String.render_template(tmpl_data, env)
  html = HTML.parse(tmpl_out)
  HTML.insert_after(tmpl_elem, html)
  HTML.delete(tmpl_elem)
end

selector = config["selector"]
tmpls = HTML.select(page, selector)
List.iter(render, tmpls)

[dmbaturin]

That's implying that the pre-render hook will run when a page is ready and is about to be converted to text. That way the hook will naturally run only when the index data is ready.

Right now index extraction is done before any widgets by default, so that widgets can safely do things that will interfere with index extraction. You can schedule some widgets to run before it with extract_after_widgets if you produce some metadata with widgets, e.g. it's how the reading time on soupault.app/blog works.

[ryanartecona]

I see. That makes sense! This would work for my case.

Being more familiar with Jekyll and tools like it, I at first assumed there would be some inbuilt way of getting post metadata into a page template. It took me a bit of reading through the docs and thinking I just couldn't find where it was documented before realizing it's not a core capability.

It may be worth adding a note in the docs somewhere, even just to explicitly state page templates in soupault aren't like page/layout templates in others (unless it exists and I did just miss it).

[dmbaturin]

Well, the big idea was to avoid separation between the page and its metadata, and also avoid duplication of it. Jekyll and friends need the user to keep all machine-readablae metadata in the front matter because they can't look into the page, but to soupault, the entire page is machine-readable, so it's possible to present data on the page as you want (by hand) and also use it as metadata, like I do in the project blog:

<h1 id="post-title">Soupault 2.7.0 release</h1>
<p>Date: <time id="post-date">2021-05-12</time> </p>
<p id="post-excerpt">
Soupault 2.7.0, is available for download from <a href="https://files.baturin.org/software/soupault/2.7.0">my own server</a>
and from <a href="https://github.com/dmbaturin/soupault/releases/tag/2.7.0">GitHub releases</a>.
It adds a new <code>wrap</code> widget, ability to disable any widget in the config, and support for multiple build profiles.
</p>

I agree that rearranging metadata or displaying metadata scattered throughout the page in a compact block are valid use cases though. I just don't like the inflexibility of the approach Jekyll/Hugo/etc. force on the user, so I'd rather add ways to allow everyone to implement their own. A pre-parse hook will allow everyone implement their own front matter for example, or import pages from other SSGs.

To serve people who "just want a blog", I made https://github.com/dmbaturin/soupault-sample-blog which I dubbed "the octopress of soupault", though it needs work to be user-friendly.

Also, is your site live and is the source public? I'd be curious to see it.

[dmbaturin]

Hi @ryanartecona,

I've added three new things that make it possible to implement this.

• A "render" hook that allows Lua code to take over the page rendering process and return the result in a variable named page_source.
• A new option index.index_fist that makes soupault do a reduced first pass to extract the index first, then do a full pass to actually produce output files. Page's own index entry is visible to the hook as index_entry variable.
• New HTML.pretty_print() and HTML.to_string() functions to convert element trees to strings.

Here's a trivial example of a hook that shows the index entry in debug logs but otherwise just pretty-prints the page element tree.

[hooks.render]
  lua_source = '''
Log.debug(JSON.pretty_print(index_entry))
page_source = HTML.pretty_print(page)
'''

That's just a simple example of course, but nothing prevents using that hook to render a template using the printed HTML and the index entry fields.