Producer and consumer diagnostics

[soujiro32167]

I'd like to create middleware for producers and consumers, to do things like distributed tracing, logging, metrics and so on.

Here is an example of instrumenting a producer and consumer with a variation of trace4cats

As you can see, its pretty clunky: I have to backfill all the aliases. It would be nice to be able to do something like

val tracedProducer = producer
  .tapChunk(c => onChunkProduced(c))
  .tapAck(a => onAckReceived(a))

val tracedCosumer = consumer
  .tapChunk(c => onChunkConsumed(c))
  .tapCommit(c => onCommit(c))

[svroonland]

See #1048

[erikvanoosten]

@svroonland We do not actually have diagnostics for producers yet and as far as I know there is no issue requesting those. Should we keep this one open?

[svroonland]

@soujiro32167 This could be of interest for you https://github.com/pierangeloc/zio-theseus/tree/main/modules/tracing-kafka/src/main/scala/io/tuliplogic/ziotoolbox/tracing/kafka

[svroonland]

Also we have consumer metrics since v2.7.3