feat: prqlc compile --debug-log

[aljazerzen]

Adds a framework for capturing debug information during the compilation process.

First stab at #4636

The result can be serialized into JSON or rendered into HTML (image below). Current display is not much better than plain terminal output, but this has potential for better visualization of what is going on in the compiler.

[max-sixty]

Looks great, thanks! This direction be really helpful I think.

---

Not a strong view at all but two other approaches to the logging infra:

• just write structured logs in JSON to stderr / possibly an optional log life
• then have a function to convert to a different representation; e.g. HTML
  ...that's a bit more modular (though also requires some serialization from serde)

• A specific command like prqlc debug dump foo.prql which dumps everything — my guess is that much of the time we'd want to drill down on a specific query, rather than always having some exhaust

...but as above, not a strong view

[aljazerzen]

Actually, this PR is combination of the two things you've suggested.

1. --debug-log=my_log.json will create a JSON file with all of the AST reprs emitted. This file could probably be read back and rendered to the HTML output,
2. the debug log should contain basically all intermediate information that compiler has. The idea is to replace various commands we have with these "debug log entries", so a normal compile invocation produces all that you need. ATM, we are 80% there.

[max-sixty]

Actually, this PR is combination of the two things you've suggested.

Yes! My musing was whether splitting it up made sense (but really just a musing and maybe not that useful..)

[aljazerzen]

More refinement:

[max-sixty]

I tried this out — really really cool! Very impressive you could pull together a UI so quickly!

---

FWIW when I try I don't get the highlights across the different stages (like in your query) unless I select the whole query, but possibly that's because the spans aren't serialized yet.

---

Other small thoughts (obv you are the owner and so only consider these if they're useful, am strongly not trying to say these are necessarily good)

• integrate with playground? Maybe that's more work than it saves though. Mostly similar goals (would want a check-box to show / hide, similar to what we have with outputting PL & RQ now)
• minor but it would be v nice to have the stages in each column, so we can see across them — i.e. "what RQ comes from this expression"

[aljazerzen]

integrate with playground? Maybe that's more work than it saves though. Mostly similar goals (would want a check-box to show / hide, similar to what we have with outputting PL & RQ now)

Yes, this is possible to do. We just need to expose the --debug-log argument to JS bindings somehow. We'd probably change the type signature, so the debug log is returned instead of written into a file.

---

minor but it would be v nice to have the stages in each column, so we can see across them — i.e. "what RQ comes from this expression"

This is easy to do! Newer versions are somewhat easier to parse visually, so it might not even be needed, but yes, can be done.