Consider changing Event struct to have non-'static metadata (Cow?)

[dvdplm]

ref. #922

Currently the Event type in tracing-core requires a &'static Metadata<'static>.

[Michael-F-Bryan]

I've run into this limitation several times and been stumped because it's not possible to dynamically construct a tracing::Metadata or tracing::Event due to the use of &'static str strings.

For a real world example, I've got a WebAssembly module which uses the tracing crate and a custom Subscriber for logging and I'd like to pass logged messages to the host's tracing subscriber. This requires the guest to serialize an Event so the host can deserialize it and reconstruct the Event and Metadata. However, this isn't possible for the host to do because the fields will be Strings defined at runtime and the Event::new() constructor requires metadata: &'static Metadata<'static>.

Leaking+caching the strings to get a &'static str isn't practical because this application is meant to run for long periods of time and the Metadata is controlled by the guest, which may not be fully trusted. For example, we could have a DOS issue where large numbers of unique Metadata objects are dynamically created by the guest, and cause the host to OOM.

[vthib]

I kind of have the same issue, where I need a dynamic Metadata and Event.

I ended up with the same kind of tricks as tracing-log, where I have a generic static Metadata, and I set all dynamic fields as additional fields of the event, only for the subscriber to pop those fields and rebuild a proper Metadata object.

This is quite ugly and cumbersome though, as this a lot of copying around and requires a custom subscriber to properly handle. I would definitely love to be able to build dynamic objects to solve those issues.

[robo-corg]

I think this limitation makes it quite hard to follow the open telemetry convention for rpc span names https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/rpc.md

Might be able to do something similar where the subscribers rewrite the events before formatting and/or sending to open telemetry.

[Michael-F-Bryan]

I ran into this issue again a couple weeks back and created a thread on the Rust User Forums.

It didn't get any responses, but I think the post itself provides some good concrete examples of what people mean when they want to dynamically generate tracing events.

---

I'm working on an application which uses the tracing library for logging, however a different component will generate events at runtime¹ and as far as I can tell it's not possible to construct a tracing::Event object because it requires 'static metadata (the thing tracking log level, target, etc.).

Here is a snippet showing the rough public API I want to expose:

use tracing::field::ValueSet;

pub enum LogLevel {
    Trace,
    Debug,
    Info,
    Warn,
    Error,
}

pub struct LogMetadata<'a> {
    pub name: &'a str,
    pub target: &'a str,
    pub level: LogLevel,
    pub file: Option<&'a str>,
    pub line: Option<u32>,
    pub module: Option<&'a str>,
}

pub enum LogValue<'a> {
    Null,
    Boolean(bool),
    Integer(i64),
    Float(f64),
    String(&'a str),
}

pub type LogValueMap<'a> = Vec<(&'a str, LogValue<'a>)>;

fn is_enabled(meta: LogMetadata<'_>) -> bool {
    tracing::dispatcher::get_default(|dispatch| {
        let meta = tracing::Metadata::from(meta);
        dispatch.enabled(&meta)
    })
}

fn log(meta: LogMetadata<'_>, message: &str, data: LogValueMap<'_>) {
    let meta = tracing::Metadata::from(meta);
    let values = value_set(message, &data);
    tracing::Event::dispatch(&meta, &values)
}

impl<'a> From<LogMetadata<'a>> for tracing::Metadata<'a> {
    fn from(_: LogMetadata<'a>) -> Self {
        todo!();
    }
}

fn value_set<'a>(message: &'a str, values: &'a [(&'a str, LogValue<'a>)]) -> ValueSet<'a> {
    todo!();
}

(playground)

The compile error shows that Event::dispatch() requires a &'static Metadata.

error[E0759]: `meta` has an anonymous lifetime `'_` but it needs to satisfy a `'static` lifetime requirement
  --> src/lib.rs:38:40
   |
37 | fn log(meta: LogMetadata<'_>, message: &str, data: LogValueMap<'_>) {
   |              --------------- this data with an anonymous lifetime `'_`...
38 |     let meta = tracing::Metadata::from(meta);
   |                                        ^^^^ ...is used here...
39 |     let values = value_set(message, &data);
40 |     tracing::Event::dispatch(&meta, &values)
   |                              ----- ...and is required to live as long as `'static` here

For more information about this error, try `rustc --explain E0759`.
error: could not compile `playground` due to previous error
Warnings

Does anyone have any solutions or workarounds for my problem?

The main thing I care about is having the log level and target in my final log event so that the dynamically generated log messages look just like any other message (styled differently based on levels, filtering by target/level works, etc.).

I could use unsafe to pretend my data is all 'static, but that's unsound and I'd prefer to avoid going down that path if possible.

Footnotes

1. In this case, a WebAssembly guest calls my application's log() function with things like the log level, target, file, message, structured data, and so on. The data is almost identical to the ValueSet and Metadata types used when constructing a tracing::Event, except everything borrows from WebAssembly linear memory instead of being compile time literals. ↩

[dpc]

Non 'static things are harder because they might require cloning, lowering the performance for everyone, even when they are not using them. So it's understandable that there might not be supported.

However I have a use case, in which I'm happy to leak the string (one per program), but I don't know how to use it as a name for a span!.

[hawkw]

However I have a use case, in which I'm happy to leak the string (one per program), but I don't know how to use it as a name for a span!.

The span! macro requires not just &'static strs but, specifically, &'static strs that are constructable in a const initializer context, because it uses the name to construct a static with a tracing Metadata struct. So, unfortunately, you can't really use a leaked string as a span name without substantial changes to how the macros work.

One potential option could be to use the dynamic string as the span's message field, instead. In that case, tracing_subscriber::fmt will, at least, format it without a field name, as span_name{the message} instead of span_name{message="the message"}, which might be a little nicer. But, it's not actually using the string as the span's name.

Sorry.

[dpc]

I appreciate the workaround suggestion. It might work.