Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Measure rate of growth of metrics #1040

Open
anubhabMajumdar opened this issue Nov 20, 2024 · 0 comments
Open

Measure rate of growth of metrics #1040

anubhabMajumdar opened this issue Nov 20, 2024 · 0 comments
Assignees
@alexcastilio @ibezrukavyi
Labels
area/data-ingestion-and-visualization area/flows area/metrics control-plane/hubble control-plane/legacy help wanted Extra attention is needed priority/0 P0 type/fix Fixes something

Comments

@anubhabMajumdar
Copy link
Contributor

anubhabMajumdar commented Nov 20, 2024 •
edited
Loading

Metrics generated by Retina is supposed to of low cardinality over time. If there's a label that is unbound, the metrics volume would constantly increase, causing OOMKIll issues for agent, and may overflow storage of these metrics on an external entity.

Add a test that

  • generates lots of L4 events
  • generate lots of DNS queries

Measure rate of growth of metrics volume to make sure we don't have unbound growth.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alexcastilio alexcastilio

@ibezrukavyi ibezrukavyi

Labels
area/data-ingestion-and-visualization area/flows area/metrics control-plane/hubble control-plane/legacy help wanted Extra attention is needed priority/0 P0 type/fix Fixes something
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@alexcastilio @anubhabMajumdar @ibezrukavyi