Test suite for html rendered Raku documentation source

The problem

After correspondence with JJ Merelo, it is clear that there needs to be a set of tests for the rendering of a pod source. In other words, if a change is made to any of the programs in the tool-chain that constructs the documents on doc.perl6.org, are the pod files "correctly" rendered into html?

This is a serious issue because the current tool chain relies on htmlify.p6 and Pod::To::HTML in a complicated and "organic" manner. Refactoring either of them causes problems. Sometimes, a minor change has major consequences that are difficult to detect.

I am re-writing most of the tool chain.

What is needed is a set of tests that can be applied to the current tool-chain and to future proposals.

Problems that can occur:

• Format codes are not recognised
• the documented pod specifications are not implemented fully
• text is omitted due to a change in processing.

The difficulties:

• A subjective change in the way pod is rendered into html should have a predictable effect on the tests. For example, the format code B<> is currently (and probably properly) rendered as .., but it could be rendered as ... .
• Aesthetic changes are subjective and so difficult to test.
• A priori we know that the two main modules Pod::To::HTML and Pod::To::BigPage will fail because they do not provide information to the caller about items that are indexed (eg., marked with X<> )

Consequently, the tests cannot be a simple comparison of rendered html to a fixed html benchmark.

A meta problem is this: In this repository, we use "standalone" pod6 sources to document perl6. Consequently, they make heavy use of C<>, B<>, =head, L<>.

Pod was written to clarify how perl 6 programs should be documented to make it easier to write the documentation in situ and for external programs to extract it.

Since standalone pod sources (eg. for the doc repo) are not documenting code in situ, there is little (if any) use of constructs such as #|. These are supposed to be used in association with declarations such as sub, role etc. My understanding of this idea is that when a new Module is added to an interactive environment, a programmer can obtain information contained in a #| comment by referencing the name of the subroutine.

So far as I know, no pod6 renderer implements this aspect of Pod6. It is not needed for the doc repository.

The consequence of this meta issue is that the test suite being suggested here is that it is not an exhaustive test of all the functionality of pod6 renderers, but a test of functionality needed for rendering the perl6 sources.

Suggestions

1. Construct a single pod file ( a suggestion is language/strings/pod6 ) such that it contains all the pod format codes and pod constructs expected. Lets place it at render-test/doc/generic-pod.pod6
2. The rendering tool chain then writes a file render-test/html/generic-pod.html.
3. A separate file called render-test/regexen.json is used to create a hash with keys format-code-b, format-code-l, pod-block-code etc pointing to regexes that define how contents are rendered. Since this test suite is specific to the docs.perl6.org site and the site has associated CSS and JS assets that require certain classes, these regexen have to be a part of the suite for this repo.
4. Write a set of tests that are run against render-test/html/generic-pod.html

The test suite then applies the regexen to the rendered html to determine whether the correct blocks have been rendered. The suite should be structured to evaluate:

1. Visible text (that is all of the text that should be visible in a browser)
2. Mandatory Format Codes, these are the ones that are mainly used in the doc/ repository. For example, none of the pod sources use P< > (yet???).
3. Pod Blocks
4. Table of Contents

Personally, I think that there should be tests of the following. But since I know a priori that Pod::To::HTML and Pod::To::BigPage will fail these tests because they do not separately collect indexed data (X<>), and this information is not used in the current docs.perl6.org website, these tests may be moot.

1. Non-mandatory Format Codes
2. Index.

This page contains the text of issue #2539