Add Quarto support

CHANGELOG.md

@@ -5,7 +5,13 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-<!-- ## [Unreleased] -->
+## [Unreleased]
+### Added
+ - Literate can now output [Quarto](https://quarto.org/) notebooks (markdown documents with
+ the `.qmd` file extension) by passing `flavor = Literate.QuartoFlavor()` to
+ `Literate.markdown`. This feature is marked as experimental since it has not been widely
+ tested and the Quarto-specific syntax may change before Literate version 3 depending on
+ what the community wants or needs. ([#199][github-199], [#200][github-200])
 
 ## [2.16.1] - 2024-01-04
 ### Fixed
@@ -276,6 +282,8 @@ https://discourse.julialang.org/t/ann-literate-jl/10651 for release announcement
 [github-194]: https://github.com/fredrikekre/Literate.jl/pull/194
 [github-195]: https://github.com/fredrikekre/Literate.jl/pull/195
 [github-197]: https://github.com/fredrikekre/Literate.jl/issues/197
+[github-199]: https://github.com/fredrikekre/Literate.jl/issues/199
+[github-200]: https://github.com/fredrikekre/Literate.jl/pull/200
 [github-204]: https://github.com/fredrikekre/Literate.jl/issues/204
 [github-205]: https://github.com/fredrikekre/Literate.jl/pull/205
 [github-219]: https://github.com/fredrikekre/Literate.jl/pull/219

docs/src/outputformats.md

@@ -86,6 +86,70 @@ Literate can output markdown in different flavors. The flavor is specified using
  [CommonMark](https://commonmark.org/) specification.
  - `flavor = Literate.FranklinFlavor()`: this outputs markdown meant to be used as input
  to [Franklin.jl](https://franklinjl.org/).
+ - `flavor = Literate.QuartoFlavor()`: this outputs markdown file (with file extension
+ `.qmd`) meant to be used with [Quarto CLI](https://quarto.org).
+
+#### Quarto flavor
+
+!!! warning "Experimental feature"
+ Quarto markdown output is marked as and experimental feature since it has not been
+ widely tested. Quarto-specific syntax may change before Literate version 3 depending on
+ what the community wants and/or needs. If you use this flavor non-interactively (such as
+ automatically building documentation) it is recommended to pin Literate to a good known
+ version.
+
+[Quarto](https://quarto.org/) is an open-source scientific and technical publishing system,
+which can extend the range of output formats from your Literate.jl-formatted scripts.
+Literate.jl will produce a `.qmd` file, which can be used as input to Quarto CLI to produce
+a variety of output formats, including HTML, PDF, Word and RevealJS slides.
+
+##### Literate + Quarto syntax tips
+
+- `# `(hashtag followed by a space) at the beginning of a line will be stripped and anything
+ that follows will rendered as a markdown, e.g., `# # Header level 1` in your script will
+ be rendered as `# Header level 1` in your .qmd file (ie, it will show as a header). Use
+ this for adding the YAML header at the top or any Markdown blocks in the Quarto guide.
+- `##|`(two hashtags followed by a pipe) at the beginning of a line will strip the first
+ hashtag and interpret the remainder of the line as part of the code block. This is useful
+ to provide Quarto commands in computation blocks, e.g., `##! echo: false` would be
+ rendered as `#| echo: false` and would tell Quarto not to "echo" the outputs of the
+ execution (see [Guide: Executions
+ options](https://quarto.org/docs/computations/execution-options.html) for more commands).
+- Make sure to always provide the YAML header and specify IJulia kernel when executing the
+ file by Quarto, e.g.,
+ ```
+ # ---
+ # title: "My Report"
+ # jupyter: julia-1.9
+ # ---
+ ```
+ Notice how all lines are escaped with a `# ` so Literate.jl knows to strip the hashtags
+ and render it as markdown (see [Authoring
+ Tutorial](https://quarto.org/docs/get-started/authoring/vscode.html#multiple-formats) for
+ more examples)
+- If any markdown components (e.g. headers) are not rendering correctly in your Quarto
+ outputs, make sure they are surrounded by empty lines (e.g., add an empty line before and
+ after the header) to help Quarto parse them correctly
+
+##### Configuring Quarto for Julia code execution
+
+- Install [Quarto CLI](https://quarto.org/docs/getting-started/installation.html)
+- Run `quarto check` to ensure all is installed correctly (you will need Python, Jupyter,
+ and IJulia kernel, see [Getting
+ Started](https://quarto.org/docs/get-started/computations/vscode.html))
+
+##### Steps to create reports
+
+- Make sure you have the right header specifying which IJulia kernel to use (e.g. `jupyter:
+ julia-1.9`), otherwise Quarto will use the default Python kernel.
+- Convert your Literate.jl script to a `.qmd` file, e.g.
+ ```julia
+ Literate.markdown("my_script.jl", flavor = Literate.QuartoFlavor())
+ ```
+- Run Quarto CLI to produce your desired output format, e.g.
+ ```bash
+ quarto render my_script.qmd --to html
+ ```
 
 
 ## [**4.2.** Notebook output](@id Notebook-output)

src/Literate.jl

@@ -16,6 +16,7 @@ struct DefaultFlavor <: AbstractFlavor end
 struct DocumenterFlavor <: AbstractFlavor end
 struct CommonMarkFlavor <: AbstractFlavor end
 struct FranklinFlavor <: AbstractFlavor end
+struct QuartoFlavor <: AbstractFlavor end
 
 # # Some simple rules:
 #
@@ -45,7 +46,7 @@ CodeChunk() = CodeChunk(String[], false)
 
 ismdline(line) = (occursin(r"^\h*#$", line) || occursin(r"^\h*# .*$", line)) && !occursin(r"^\h*##", line)
 
-function parse(content; allow_continued = true)
+function parse(flavor::AbstractFlavor, content; allow_continued = true)
  lines = collect(eachline(IOBuffer(content)))
 
  chunks = Chunk[]
@@ -75,8 +76,14 @@ function parse(content; allow_continued = true)
  if !(chunks[end] isa CodeChunk)
  push!(chunks, CodeChunk())
  end
- # remove "## " and "##\n"
- line = replace(replace(line, r"^(\h*)#(# .*)$" => s"\1\2"), r"^(\h*#)#$" => s"\1")
+ # remove "## " and "##\n" (strips leading "#" for code comments)
+ if flavor isa QuartoFlavor
+ # for Quarto, strip leading "#" from code cell commands, eg, "##| echo: true" -> "#| echo: true"
+ line = replace(replace(line, r"^(\h*)#(#(:? |\|).*)$" => s"\1\2"), r"^(\h*#)#$" => s"\1")
+ else
+ # all other flavors
+ line = replace(replace(line, r"^(\h*)#(# .*)$" => s"\1\2"), r"^(\h*#)#$" => s"\1")
+ end
  push!(chunks[end].lines, line)
  end
  end
@@ -282,6 +289,24 @@ function edit_commit(inputfile, user_config)
  return fallback_edit_commit
 end
 
+# All flavors default to the DefaultFlavor() setting
+function pick_codefence(::AbstractFlavor, execute::Bool, name::AbstractString)
+ return pick_codefence(DefaultFlavor(), execute, name)
+end
+function pick_codefence(::DefaultFlavor, execute::Bool, name::AbstractString)
+ return "````julia" => "````"
+end
+function pick_codefence(::DocumenterFlavor, execute::Bool, name::AbstractString)
+ if execute
+ return pick_codefence(DefaultFlavor(), execute, name)
+ else
+ return "````@example $(name)" => "````"
+ end
+end
+function pick_codefence(::QuartoFlavor, execute::Bool, name::AbstractString)
+ return "```{julia}" => "```"
+end
+
 function create_configuration(inputfile; user_config, user_kwargs, type=nothing)
  # Combine user config with user kwargs
  user_config = Dict{String,Any}(string(k) => v for (k, v) in user_config)
@@ -315,10 +340,11 @@ function create_configuration(inputfile; user_config, user_kwargs, type=nothing)
  cfg["softscope"] = type === (:nb) ? true : false # on for Jupyter notebooks
  cfg["keep_comments"] = false
  cfg["execute"] = type === :md ? false : true
- cfg["codefence"] = get(user_config, "flavor", cfg["flavor"]) isa DocumenterFlavor &&
- !get(user_config, "execute", cfg["execute"]) ?
- ("````@example $(get(user_config, "name", replace(cfg["name"], r"\s" => "_")))" => "````") :
- ("````julia" => "````")
+ cfg["codefence"] = pick_codefence(
+ get(user_config, "flavor", cfg["flavor"]),
+ get(user_config, "execute", cfg["execute"]),
+ get(user_config, "name", replace(cfg["name"], r"\s" => "_")),
+ )
  cfg["image_formats"] = _DEFAULT_IMAGE_FORMATS
  cfg["edit_commit"] = edit_commit(inputfile, user_config)
  deploy_branch = "gh-pages" # TODO: Make this configurable like Documenter?
@@ -430,14 +456,19 @@ function preprocessor(inputfile, outputdir; user_config, user_kwargs, type)
  config = create_configuration(inputfile; user_config=user_config,
  user_kwargs=user_kwargs, type=type)
 
+ # Quarto output does not support execute = true
+ if config["flavor"] isa QuartoFlavor && config["execute"]
+ throw(ArgumentError("QuartoFlavor does not support `execute = true`."))
+ end
+
  # normalize paths
  inputfile = normpath(inputfile)
  isfile(inputfile) || throw(ArgumentError("cannot find inputfile `$(inputfile)`"))
  inputfile = realpath(abspath(inputfile))
  mkpath(outputdir)
  outputdir = realpath(abspath(outputdir))
  isdir(outputdir) || error("not a directory: $(outputdir)")
- ext = type === (:nb) ? ".ipynb" : ".$(type)"
+ ext = type === (:nb) ? ".ipynb" : (type === (:md) && config["flavor"] isa QuartoFlavor) ? ".qmd" : ".$(type)"
  outputfile = joinpath(outputdir, config["name"]::String * ext)
  if inputfile == outputfile
  throw(ArgumentError("outputfile (`$outputfile`) is identical to inputfile (`$inputfile`)"))
@@ -477,7 +508,7 @@ function preprocessor(inputfile, outputdir; user_config, user_kwargs, type)
  content = replace_default(content, type; config=config)
 
  # parse the content into chunks
- chunks = parse(content; allow_continued = type !== :nb)
+ chunks = parse(config["flavor"], content; allow_continued = type !== :nb)
 
  return chunks, config
 end

test/runtests.jl

@@ -1,5 +1,6 @@
 import Literate, JSON
 import Literate: Chunk, MDChunk, CodeChunk
+import Literate: pick_codefence, DefaultFlavor, QuartoFlavor
 using Test
 
 # compare content of two parsed chunk vectors
@@ -108,6 +109,8 @@ end
  ## Line 77
  ##
  ## Line 79
+ # Line 80: Quarto Specific
+ ##| Line 81
  """
  expected_chunks = Chunk[
  MDChunk(["" => "Line 1"]),
@@ -145,10 +148,55 @@ end
  CodeChunk(["Line 64", " # Line 65", " Line 66", "Line 67"], false),
  CodeChunk(["# Line 73", "#", "# Line 75"], false),
  CodeChunk([" # Line 77", " #", " # Line 79"], false),
- ]
- parsed_chunks = Literate.parse(content)
+ MDChunk(["" => "Line 80: Quarto Specific"]),
+ CodeChunk(["##| Line 81"], false)
+ ]
+ parsed_chunks = Literate.parse(DefaultFlavor(), content)
  compare_chunks(parsed_chunks, expected_chunks)
 
+ # QuartoFlavor parsing semantics
+ expected_chunks_quarto = Chunk[
+ MDChunk(["" => "Line 1"]),
+ CodeChunk(["Line 2"], false),
+ MDChunk(["" => "Line 3", "" => "","" => "Line 5"]),
+ CodeChunk(["Line 6", "","Line 8"], false),
+ MDChunk(["" => "Line 9"]),
+ MDChunk(["" => "Line 11"]),
+ CodeChunk(["Line 12"], false),
+ CodeChunk(["Line 14"], false),
+ MDChunk(["" => "Line 15"]),
+ MDChunk(["" => "Line 17"]),
+ CodeChunk(["Line 18"], false),
+ CodeChunk(["Line 20"], false),
+ MDChunk(["" => "Line 21"]),
+ CodeChunk(["Line 22", " Line 23", "Line 24"], false),
+ CodeChunk(["Line 26", " Line 27"], true),
+ CodeChunk(["Line 29"], false),
+ CodeChunk(["Line 31", " Line 32"], true),
+ MDChunk(["" => "Line 33"]),
+ CodeChunk(["Line 34"], false),
+ CodeChunk(["Line 36"], true),
+ CodeChunk([" Line 38"], true),
+ CodeChunk(["Line 40"], false),
+ CodeChunk(["Line 42", " Line 43"], true),
+ MDChunk(["" => "Line 44"]),
+ CodeChunk([" Line 45"], true),
+ MDChunk(["" => "Line 46"]),
+ CodeChunk(["Line 47"], false),
+ MDChunk(["" => "Line 48"]),
+ CodeChunk(["#Line 49", "Line 50"], false),
+ MDChunk(["" => "Line 53"]),
+ CodeChunk(["# Line 57", "Line 58", "# Line 59", "##Line 60"], false),
+ MDChunk([" " => "Line 62", " " => "# Line 63"]),
+ CodeChunk(["Line 64", " # Line 65", " Line 66", "Line 67"], false),
+ CodeChunk(["# Line 73", "#", "# Line 75"], false),
+ CodeChunk([" # Line 77", " #", " # Line 79"], false),
+ MDChunk(["" => "Line 80: Quarto Specific"]),
+ CodeChunk(["#| Line 81"], false) # parses correctly as code cell command
+ ]
+ parsed_chunks = Literate.parse(QuartoFlavor(), content)
+ compare_chunks(parsed_chunks, expected_chunks_quarto)
+
  # test leading/trailing whitespace removal
  io = IOBuffer()
  iows = IOBuffer()
@@ -165,7 +213,7 @@ end
  foreach(x -> println(iows), 1:rand(2:5))
  end
 
- compare_chunks(Literate.parse(String(take!(io))), Literate.parse(String(take!(iows))))
+ compare_chunks(Literate.parse(DefaultFlavor(), String(take!(io))), Literate.parse(DefaultFlavor(), String(take!(iows))))
 
 end # testset parser
 
@@ -753,6 +801,19 @@ end end
  @test !occursin("EditURL", markdown)
  @test !occursin("#hide", markdown)
 
+ # flavor = QuartoFlavor()
+ # execution of Quarto markdown is not allowed
+ let expected_error = ArgumentError("QuartoFlavor does not support `execute = true`.")
+ @test_throws expected_error Literate.markdown("quarto.jl", flavor = Literate.QuartoFlavor(), execute = true)
+ end
+ Literate.markdown(inputfile, outdir, flavor = Literate.QuartoFlavor(),execute=false)
+ markdown = read(joinpath(outdir, "inputfile.qmd"), String)
+ @test occursin("```{julia}", markdown)
+ @test !occursin(r"`{3,}@example", markdown)
+ @test !occursin("continued = true", markdown)
+ @test !occursin("EditURL", markdown)
+ @test !occursin("#hide", markdown)
+
  # documenter = false (deprecated)
  @test_deprecated r"The documenter=true keyword to Literate.markdown is deprecated" begin
  Literate.markdown(inputfile, outdir, documenter = true)
@@ -880,6 +941,11 @@ end end
  @test occursin("# MD", markdown) # text/markdown
  @test occursin("~~~\n<h1>MD</h1>\n~~~", markdown) # text/html
 
+ # QuartoFlavor file extension
+ write(inputfile, "#=\r\nhello world\n=#\r\n")
+ _, config = Literate.preprocessor(inputfile, outdir; user_kwargs=(), user_config=Dict("flavor"=>Literate.QuartoFlavor()), type=:md)
+ @test config["literate_ext"] == ".qmd"
+
  # verify that inputfile exists
  @test_throws ArgumentError Literate.markdown("nonexistent.jl", outdir)
 
@@ -1406,6 +1472,16 @@ end end
  @test occursin("Link to nbviewer: www.example2.com/file.jl", script)
  @test occursin("Link to binder: www.example3.com/file.jl", script)
 
+ # Test pick_codefence function
+ default_codefence=pick_codefence(Literate.DefaultFlavor(), true, "testname")
+ @test default_codefence == ("````julia" => "````")
+ @test default_codefence == pick_codefence(Literate.FranklinFlavor(), true, "testname")
+ @test default_codefence == pick_codefence(Literate.DocumenterFlavor(), true, "testname")
+ documenter_codefence = ("````@example testname" => "````")
+ @test documenter_codefence == pick_codefence(Literate.DocumenterFlavor(), false, "testname")
+ @test ("```{julia}" => "```") == pick_codefence(Literate.QuartoFlavor(), true, "testname")
+ @test ("```{julia}" => "```") == pick_codefence(Literate.QuartoFlavor(), false, "testname")
+
  # Misc default configs
  create(; type, kw...) = Literate.create_configuration(inputfile; user_config=Dict(), user_kwargs=kw, type=type)
  cfg = create(; type=:md, execute=true)