Monitor for missing reports

[tunetheweb]

Related to #3 we should have some alert to say if the pipeline is not completed by month end or reports are missing.

At the minute we have a pretty basic GitHub Action that runs a bash script that runs on the 28th of the month, to test for a few JSON files in GCP (the first and last report for each lense) and raise an Issue if all are not available. It works, and has alerted us to problems a good few times, but is a bit basic and doesn't cover all the reports.

We should consider a more fully featured version of that that can check all reports.

One thing I do like about the current script is that it is completely independent of the pipeline. If we make it too dependent (e.g. to run the checks after the pipeline is finished), then there's a concern the check never runs whereas that is not a concern now.

[max-ostapenko]

Generally, I think this scenario is not relevant for the new pipeline as we don't process any reports anymore.

Currently we have a few moving parts and atomic jobs.
Added error monitoring description to the readme.

Let's close this issue and observe a new pipeline.

[tunetheweb]

We do still run the reports monthly from an (unmonitored) cron job. Would be nice to automate that more.

[tunetheweb]

These are the reports that appear here btw: https://httparchive.org/reports. The SQL in the BigQuery repo runs monthly to populate JSON files on our CDN (e.g. https://cdn.httparchive.org/reports/a11yButtonName.json) which then are pulled into that website for display.

The reports are also run for each "lens" in the drop down: