[Monitoring] Create a dashboard for overall clusterfuzz health #4497
Conversation
vitorguidi
changed the title
vitorguidi
requested review from
letitz,
alhijazi,
jonathanmetzman and
oliverchang
…estcase count by status
| "displayHorizontal": false | ||
| } | ||
| }, | ||
| "title": "p50 preprocess duration by task", |
There was a problem hiding this comment.
could you add the duration unit (seconds I believe) to the title?
| "displayHorizontal": false | ||
| } | ||
| }, | ||
| "title": "p50 utask_main duration by task", |
There was a problem hiding this comment.
could you add the duration unit (seconds I believe) to the title?
| "displayHorizontal": false | ||
| } | ||
| }, | ||
| "title": "p50 postprocess duration by task", |
There was a problem hiding this comment.
could you add the duration unit (seconds I believe) to the title?
| "displayHorizontal": false | ||
| } | ||
| }, | ||
| "title": "p95 preprocess duration by task", |
There was a problem hiding this comment.
could you add the duration unit (seconds I believe) to the title?
| "displayHorizontal": false | ||
| } | ||
| }, | ||
| "title": "p95 utask_main duration by task", |
There was a problem hiding this comment.
could you add the duration unit to the title?
| "displayHorizontal": false | ||
| } | ||
| }, | ||
| "title": "p95 postprocess duration by task", |
There was a problem hiding this comment.
could you add the duration unit to the title?
| "displayHorizontal": false | ||
| } | ||
| }, | ||
| "title": "Testcase triage duration (p50) - by step", |
There was a problem hiding this comment.
could you add the duration unit to the title?
| "displayHorizontal": false | ||
| } | ||
| }, | ||
| "title": "Testcase triage duration (p90) - by step", |
There was a problem hiding this comment.
could you add the duration unit to the title?
| "displayHorizontal": false | ||
| } | ||
| }, | ||
| "title": "Testcase triage duration (p95) - by step", |
There was a problem hiding this comment.
could you add the duration unit to the title?
| display_name = "ClusterFuzz General Health" | ||
| dashboard_json = <<JSON | ||
| { | ||
| "displayName": "vitor-experimentation-sli", |
There was a problem hiding this comment.
Could you change the name here please?
Motivation
As a final step for the monitoring project, this PR creates a dashboard for overall system health.
The reasoning behind it is to have:
These are sufficient to apply the RED methodology (rate, error and duration), provide high level metrics we can alert on, and aid in troubleshooting outages with a well defined methodology.
There were two options to commit this to version control: terraform, or butler definitions. The first was chosen, since it is the preffered long term solution, and it is also simpler to implement, since it supports copy pasting the JSON definition from GCP.
Attention points
This should be automatically imported from main.tf, so it (should be) sufficient to just place the .tf file in the same folder, and have butler deploy handle the terraform apply step.
How to review
Head over to go/cf-chrome-health-beta, internally. It is not expected that the actual dashboard definition is reviewed, it is just a dump of what was manually created in GCP.
Part of #4271