Format Codes in Code markup

[finanalyst]

There is an issue in doc-website that raises more general Rakudoc (aka POD6) questions (see thread from here

tl;dr 13 out of 413 Raku/docs/doc sources use functionality in the first specification of POD6, but undocumented in Raku/docs/doc. Since it was undocumented, it was not included in the ProcessedPod renderer, but there are 37 instances where there is HTML escaped formating in the new website. The old functionality may interfere with new custom FormatCodes. One proposal is to deprecate the old undocumented function, the other is to respect and document the old spec.

Detail
The problem only seems to occur in code blocks, viz Pod::Blocks starting =for code or =begin code
The original spec of Rakudoc (aka POD6) can be found in S26 (Synopsis), :allow is specified.

No extra behavior is documented according to Raku/language/pod.

Of the 37 instances in 13 files, the codes allowed are C B L R. Personally, I cannot see the extra value they provide.

An extension of Rakudoc that is implemented in ProcessedPod and not contained in S26, but reasonably inferred from it, is the ability to defined custom FormatCodes. I believe this extension is valuable because it allows for new span level code and for other languages to use Unicode characters for markup.

The S26 specification of :allow in =begin code does not say much about how the code would affect the code, but the ability to extend to new custom codes makes it possible to add new text, such as Fontawesome icons.

One idea is to reduce the complexity of rendering =begin code by deprecating the underdocumented functionality. A rational for this is that snippets of code should be available for cut and paste, and to work out of the box. Adding extra text or formatting makes these code snippet un-compilable.

Another idea is to support and document the original spec.

Any preferences either way?

[Altai-man]

One idea is to reduce the complexity of rendering =begin code by deprecating the underdocumented functionality

Please do. As it is now, Pod is obscure enough so that we did not not expect doc writers to be too bothered with being super smart with it to know all the shady tricks. So the simpler it is and the more ignorant safety nets are around ("ignorant" as "against the ignorance", not "ignorant checks" :) ), the harder it is to get the mistakes (as people did with the X<> code, for example) into the doc content.

[cfa]

Copied over from the prior ticket:

These are all potentially problematic at the moment due to :allow or :lang specifications.

[finanalyst]

@cfa ++ for the llst. The only metatab for =code that I am firmly against is :allow. I am not suggesting deprecating the other metatabs.

Looking through the list, I found three metatabs other than :allow. None of the other three are documented, even in S26. The tabs are:

1. :lang is useful because it signals a different syntax highlighting mode. The default would be Raku but as new languages are added to the highlighter, they would be respected. Currently, the only language respected is Raku (actually Perl 6). There is an issue to create a new syntax highlighter.

2. :skip-test is not used by the Renderer. It can be used by a test Harness, but I don't know whether it is used in the xt tests on this site. I think this is a fossil. When the documentation was first set up, it linked explicitly to the roast suite. This is no longer true. I suggest we deprecate :skip-test

3. :preamble I do not know what this one is. It would take some effort to work it out. If anyone here knows, please drop a comment in.

4. :allow for completeness. This is documented in S26 twice. Once referring to =code but also generically that it should be respected by all block types. I do not think this is a well-designed meta tab. It is also bad for code, as @Altai-man has said. It should be deprecated

[patrickbkr]

I think (not 100% confident) :preamble is used by a test that checks the code snippets work. The preamble is not displayed but prepended to the snippets before testing.

[coke]

This is correct, only used by xt/examples-compilation.t

[coke]

See https://github.com/Raku/doc/blob/main/writing-docs/EXAMPLES.md for the tags used by the xt/ test. - there is also :ok-test, and :method; we can rename any of these as long as we update the doc files and the test itself to match.

[zag]

I'm not sure if :allow should be considered obsolete.

First, according to S26 allow is used to insert a non-ASCII character using E<>

 For example, you may wish
to emphasize a particular keyword in an example (using a B<> code). Or
you may want to indicate that part of the example is metasyntactic
(using the  R<> code). Or you might need to insert a non-ASCII
character (using the E<> code).

Moreover, it is crucial to bear in mind that documentation can be exported to formats other than HTML.

In such cases, highlighting particular keywords in examples by B<> or I<> becomes important to ensure that the documentation is clear and easily understandable.
thank you

[finanalyst]

Good points.

[finanalyst]

I have begun to study how to solve the issue of :allow and I am fixing problems with Language/pod.rakudoc. There are several inter-related questions:

1. Problems with the text of Language/pod.

• Several Format-Codes, such as underline were omitted from the source because they broke Pod-to-BigPage.
• Pod2BigPage is very broken. So examples that do not break Raku-Pod-Render, which is currently used to drive docs.raku.org should be re-inserted.
• I will raise PR on Raku/doc for these changes.
• There are about three such.

2. Problems :allow. This is covered above. Further inspection of S26 indicates :allow was to be used for =input =output and =code.

• I am implementing :allow

3. Documented but never used.

• =input and =output are documented in Language/pod, but they are not implemented by any of Pod::To::HTML, Pod::To::BigPage or Raku::Pod::Render.
• @cfa @2colours is there any reason to implement these blocks?
• I am willing to do this.

4. Undocumented and USEd
   - =nested and =begin nested are used 4 times in the documentation sources, but only in Language/pod
   - The way they are used is - to my mind - incorrect as they are being used to mask features in Pod that are not implemented by Pod::To::HTML. They are implemented by Raku::Pod::Render.
   - @cfa @2colours @coke Is there any need to implement =nested. I think that the instances of =nested can be eliminated from the Language/pod file.
5. :numbered This option is defined and should be available for =head and =item markup.
   - I shall be implementing this in due course.
   -

[finanalyst]

Documenting Raku::Pod::Render
I will be documenting some of the use of Raku::Pod::Render in Language/pod.

Since Custom blocks can be inserted, and some come as standard with the Raku::Pod::Render distribution, it seems to me that it would be cool include some examples in Language/pod, eg.

• Including a latex rendering of an equations
• Including a directed graph from dot directives
• A sample map.

[finanalyst]

This thread relates to RakuDoc version 1 (aka POD6). The revision of RakuDoc changes the context of this discussion.