Open telemetry .net -> open telemetry collector -> Prometheus and counter metric woes #4249
Replies: 3 comments 6 replies
-
I faced the same issue, the only difference is that I use |
Beta Was this translation helpful? Give feedback.
-
@headnoodle Could you try your scenario by directly exporting to Prometheus instead of going through Collector and check if the issue persists even when directly sending metrics to Prometheus? You could refer to this sample app on how to send metrics directly to Prometheus: https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/docs/metrics/getting-started-prometheus-grafana |
Beta Was this translation helpful? Give feedback.
-
@utpilla, I made a sample application based on a documentation you provided (link). As I added Also, I tried to work around the problem by `initializing` a counter with 0 value (by calling Thank you in advance for your help and your time! |
Beta Was this translation helpful? Give feedback.
-
Hi, I am quite new to open telemetry, Prometheus,etc but have added code to my application that sends metrics to the open telemetry collector via these .net api packages. Everything seems to work fine but I have an issue with counters and how they are implemented in Prometheus.
I have a counter metric defined in .net that can contain one of a few tags, these are sometimes incremented before the first push to the collector starts so the first value that is added to Prometheus may not be 0. This is causing me problems as if I do an increase in promql it registers as 0 as it has nothing to compare it against, and when it does have enough data it only compares it against the initial value and not 0.
Other websites I have looked at suggest initialising all counters with 0 in the client app before sending real metrics, this would also mean that I would have to initialise any of the counters with labels as 0 too but luckily I know all the label combinations so that won’t be a huge issue.
my question is, is this possible? I tried to do an Add on the metric of 0 but that didn’t seem to work.
Another option is that I somehow hijack the otlp exporter so that it sends a zero for the metric the first time it is encounter for a specific label,and on the next push it would send the correct value, but I haven’t got a clue if I can inject that functionality into the existing libraries
Any ideas would be appreciated.
Dean.
Beta Was this translation helpful? Give feedback.
All reactions