Create custom documentations for rules

[jmarcil]

Following the comments on #28, and the following the example of pretty good open source tools like those:
https://find-sec-bugs.github.io/bugs.htm
https://security-code-scan.github.io/#Rules

We need to have non-inline documentation that tells a story about:

• Why this is marked as an issue (vulnerability explanation)
• How to correct it (vulnerability remediation)
• Give outside documentation references (because many issues are contextually important and need deeper knowledge than a simple explanation)

Since this goes over the initial goals for this tool, it would have to be with a big release like 3.0 with better support for false positive reduction.

Note that I'm now out of touch with the PHP community since I haven't touched it myself for a few years now. This project will definitely need help from the community. I'm currently considering doing the move to OWASP, as that would enable me to get people all around the world interested might be able to get back people into PHP. As far as I know, most of the OWASP PHP stuff is deprecated/abandoned and the rising usage of this tool is proof that the need is still there.

[jmarcil]

I've given some thoughts into this, and at the moment the best way would be to have the GitHub Pages inside a docs/ folder that we auto generate using a PHP script.

We could actually store the markdown of each items into the Sniff file itself. That is the PHP file not the XML config file that is given as example for people to customize their rules.

So the creation process would be:
Write markdown in the Sniff file -> Generate target page -> push to GitHub

And the consumption process would be:
Run scan -> see URL to GitHub Page for details -> click and read the page

I will need to create a new addError/addWarning util function with a setting (on by default) to show the URL.

One flat file can be used to write it all. We point to the a href to a fragment with the name of the sniff without the Sniff suffix, prefixed with the folder like this:
AssertsSniff becomes #BadFunctions-Asserts.

That all said, we can still start to write the documentation before having that implemented in code (and then just retro fit the doc into the Sniff files).

I'll proceed on a PR for that.

[jmarcil]

Unsure about the #BadFunctions- prefix. We need to keep the URLs as short as possible, and it's a bit hard to handle easily in Markdown.

[jrfnl]

Just checking: you are aware that PHPCS has a build in mechanism to generate sniff documentation ?

That would involve adding a Docs/Category/SniffNameStandard.xml file for each sniff. Have a look here for some examples of what those files should look like: https://github.com/PHPCSStandards/PHPCSExtra/tree/develop/Universal/Docs

Once those files are in place, you can then use --standard=Security --generator=Text to get the documentation on the command-line.

There are also Markdown and HTML generators available to generate documentation which can be added to a website.

Having the docs in the Standard.xml format within this repo allows people to contribute to them easily.

[jmarcil]

I don't remember if I looked into this, but as long as it fulfills those bases requirements it should be fine:

• We can update the documentation without being forced to do a release
• We can access the documentation without having the tool installed (URL in a browser)
• We can use markup language instead of plain text to have something that looks good (can apply css, use URLs)

If you think that we can still do this and have standard documentation with PHPCS, I'm 100% behind the proposition.

I said it in a few thread but this one, but those types of change should be part in a v3.0 release that coincide with the move of the project into OWASP.

Looking forward to it, and thank you very much for your last interactions in this project, they are very insightful and very welcomed for pushing this project into a clean state.

[jrfnl]

We can update the documentation without being forced to do a release

Yes and no. The documentation someone sees on the command-line will always be tied into the version of the tooling they have installed.

However, online documentation, either in a wiki or on a webpage, can be updated independently of a release by running the below command against master and then just copying and pasting the output:

$ phpcs --standard=Security --generator=HTML > yyyymmdd-docs.html

This can probably be automated using some scripting so a specific page gets automatically updated on every merge/push.

We can access the documentation without having the tool installed (URL in a browser)

People could see the XML files in the project source here on GitHub without having the tool installed. Aside from that if the wiki/webpage documentation generation is updated regularly, as I mention above, that would be available too without having the tool installed.

We can use markup language instead of plain text to have something that looks good (can apply css, use URLs)

That is the only bit which will be tricky using the XML docs as they also need to be able to display on the command-line which only handles plain text.