Tracing kafka collector

[jeqo]

Fix #2872

Changes to enable self-tracing on kafka-collector. As mentioned on the issue, RequestContextCurrentTraceContext causes an issue to propagate traces.

From Armeria Slack:

We usually recommend to create a fake ServiceRequestContext in that case.
https://armeria.dev/docs/advanced-unit-testing#using-a-fake-context-to-emulate-an-incoming-request
So we can emulate the event from the kafka consumer as a request from a client.

[codefromthecrypt]

sounds good. I think we have a rate limit now on self-tracing so it should be safe to trace collectors!

[codefromthecrypt]

maybe add another inject parameter for consumerSupplier instead? you can then make a bean that is conditionally wrapped with KafkaTracing

[jeqo]

gave a try, let me know how it looks now

[jeqo]

@openzipkin/armeria what would be the best way to handle this scenario:

When self-tracing is enabled, Tracing uses Armeria's RequestContextCurrentTraceContext. When tracing the kafka consumer, which starts a trace outside an Armeria service, it throws:

2020-12-23 15:19:30.408  WARN [/] 815184 --- [pool-3-thread-1] .a.c.b.RequestContextCurrentTraceContext : Attempted to propagate trace context, but no request context available. Did you forget to use RequestContext.contextAwareExecutor() or RequestContext.makeContextAware()?

com.linecorp.armeria.common.brave.RequestContextCurrentTraceContext$LogRequestContextWarningOnce$NoRequestContextException: null
        at com.linecorp.armeria.common.brave.RequestContextCurrentTraceContext$LogRequestContextWarningOnce$ClassLoaderHack.<clinit>(RequestContextCurrentTraceContext.java:262) ~[armeria-brave-1.3.0.jar!/:?]
        at com.linecorp.armeria.common.brave.RequestContextCurrentTraceContext$LogRequestContextWarningOnce.get(RequestContextCurrentTraceContext.java:250) ~[armeria-brave-1.3.0.jar!/:?]
        at com.linecorp.armeria.common.brave.RequestContextCurrentTraceContext$LogRequestContextWarningOnce.get(RequestContextCurrentTraceContext.java:243) ~[armeria-brave-1.3.0.jar!/:?]
        at com.linecorp.armeria.common.RequestContext.mapCurrent(RequestContext.java:106) ~[armeria-1.3.0.jar!/:?]
        at com.linecorp.armeria.common.brave.RequestContextCurrentTraceContext.getRequestContextOrWarnOnce(RequestContextCurrentTraceContext.java:240) ~[armeria-brave-1.3.0.jar!/:?]
        at com.linecorp.armeria.common.brave.RequestContextCurrentTraceContext.get(RequestContextCurrentTraceContext.java:137) ~[armeria-brave-1.3.0.jar!/:?]
        at brave.Tracer.nextSpan(Tracer.java:501) ~[brave-5.13.2.jar!/:?]
        at zipkin2.server.internal.brave.TracedCall.enqueue(TracedCall.java:47) ~[classes!/:?]
        at zipkin2.collector.Collector.store(Collector.java:201) ~[zipkin-collector-2.23.2-SNAPSHOT.jar!/:?]
        at zipkin2.collector.Collector$StoreSpans.run(Collector.java:230) ~[zipkin-collector-2.23.2-SNAPSHOT.jar!/:?]
        at zipkin2.collector.Collector.accept(Collector.java:136) ~[zipkin-collector-2.23.2-SNAPSHOT.jar!/:?]
        at zipkin2.collector.Collector.accept(Collector.java:109) ~[zipkin-collector-2.23.2-SNAPSHOT.jar!/:?]
        at zipkin2.collector.Collector.acceptSpans(Collector.java:191) ~[zipkin-collector-2.23.2-SNAPSHOT.jar!/:?]
        at zipkin2.collector.Collector.acceptSpans(Collector.java:172) ~[zipkin-collector-2.23.2-SNAPSHOT.jar!/:?]
        at zipkin2.collector.kafka.KafkaCollectorWorker.run(KafkaCollectorWorker.java:116) ~[zipkin-collector-kafka-2.23.2-SNAPSHOT.jar!/:?]
        at zipkin2.collector.kafka.KafkaCollector$LazyKafkaWorkers.lambda$guardFailures$0(KafkaCollector.java:278) ~[zipkin-collector-kafka-2.23.2-SNAPSHOT.jar!/:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
        at java.lang.Thread.run(Thread.java:832) [?:?]

Not sure where this should be handled, e.g. transparently at the RequestContextCurrentTraceContext?

https://armeria.dev/docs/advanced-unit-testing#using-a-fake-context-to-emulate-an-incoming-request mentions this scenario, but the module implementing the kafka consumer does not have an armeria dependency to create a fake context.

[jcchavezs]

@trustin @huydx any feedback on this?