-
Notifications
You must be signed in to change notification settings - Fork 108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Generic rules causes incorrect graphs in grafana #981
Comments
@sergeyshevch are you able overcome this by changing the expr in the panel? I tried to use quantile_over_time but it doesn't work with vector data. I think this has to be fixed in Pyrra generic rules code |
Pyrra should store rate/increase data instead of sum than it will be fixed. I will share it later on next week |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Generic rules (pyrra_requests/errors_total) cause incorrect calculation in Grafana
sum(rate(sum(metric))
instead ofsum(rate(metric)
that lead to spikes when one of counter resetting.Article with explanation: https://www.robustperception.io/rate-then-sum-never-sum-then-rate/
Panel from SLO Detail dashboard:
Graph with comparison to correct metric calculation
Is it possible to overtake such issues in any way? Maybe store sum(rate() instead of sum and call metric as pyrra_requests_rate?
The text was updated successfully, but these errors were encountered: