Set metrics attributes in some context

[dominikjeske]

I wonder is it possible to set some context to metrics emitted in application. Currently resource attributes are statically generated on service start and implementing some attribute in each individual metric would be stupid. I have scenario when I would like to emit some metrics during performance tests in different scenarios and name of the scenario would be ideal context for me. I could set this context dynamically and change it when scenario is changing. I found this open-telemetry/oteps#207 but it is still open. Is there any way to set such context and propagate it to OTLP exporter?

[cijothomas]

can you elaborate what you are trying to achieve, so we can offer some suggestions.
Baggage is the general purpose way to add user defined key-value pairs to any signal, not just metrics. Have you explored that already?

[dominikjeske]

@cijothomas You mentioned Baggage - if I will use Baggage in http requests Can I use it as metric label? Maybe by some transformation in OTLP collector?

[cijothomas]

https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/src/OpenTelemetry.Api#baggage-api This is the Baggage API for OpenTelemetry. (Please ignore the Activity.Baggage API.. its not really the OTel one, but it is still there prominently! Something we expect to smoothen in the future).

Baggage is not tied to Metrics. So you can use it for any signal.
One possible way for you is:
// set the baggage. this is available anywhere in the rest of the request flow. (hint: it is backed by async-local)
// this could be some middleware earlier in the pipeline.
Baggage.SetBaggage("LabelForTestResult", "itvalues");
..
..
// later in the request where you record metric.. this could be another middleware/controller later..
var labelFromBaggage = Baggage.GetBaggage("LabelForTestResult")
counter.Add(1, new keyvaluepair("key", labelFromBaggage),

Note: Baggage gets propagated automatically to next service as well, so be careful about not putting sensitive things into Baggage!.

Please see if helps your scenario. If are interested, please do consider sending a PR to modify docs to show this (Not sure which doc, but we can find a place) to help future readers.

[dominikjeske]

@cijothomas Thanks for your response but I'm looking for something slightly different. I'm aware on how to add labels and use Baggage. My problem to solve is to change label of ALL metrics. Most of them is not created by me. One place for this could be resource attributes but they are set one time on start. So best solution could be send something in headers (it could be baggage) and propagate it somehow to OTLP collector as metric label (attribute or tag - I have to say that naming convention for metadata in OTel is very confusing). But I don't know how to do this. Why I'm looking for something like this? I would like to do many stress test scenarios and each scanario would change the label so I can distinct results. Of course I could kill proccess each time and change resource attributes but it would be great to do this dynamically by header.

[cijothomas]

Got it.
There is no capability in OTel Metrics to dynamically add metrics labels! (i.e there is no equivalent of Span/ActivityProcessor , LogRecordProcessor for metrics today! MeasurementProcessor concept was explicitly removed from OTel Metrics specs. - https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/docs/metrics#metrics-enrichment)

Unfortunately, the idea of using baggage to send labels to OTLP Collector won't work. As exporting to OTLP Collector is done in background thread, and it has no access to the Baggage, as it is only obtainable in the original context only.

Since you are asking for a similar mechanism as MeasurementProcessor, it is best to open a spec issue, as this is more broadly applicable to all languages, not just OTel .NET

[dominikjeske]

@cijothomas Thanks for clarification. One last thing - what about this -> open-telemetry/oteps#207 Maybe this is what I'm talking about?