Start using OTEL to send rrweb data

[ccschmitz]

Consolidate the push payload logic and just use OTEL's implementation for sending/receiving data. Should replace all our custom batching logic.

[linear]

HIG-4619 Start using OTEL to send rrweb data

[greptile-apps]

To integrate OTEL for sending rrweb data and replace the custom batching logic, follow these steps:

1. Client-Side Changes:

   • Modify /sdk/client/src/index.tsx to use OpenTelemetry's JavaScript SDK for capturing rrweb events. Implement the telemetry data collection using the OpenTelemetry API.
   • Ensure rrweb events are formatted according to the OpenTelemetry specification for logs or custom events.

2. Backend Processing:

   • In backend/public-graph/graph/resolver.go, adjust the logic for handling incoming payloads to parse data formatted according to the OpenTelemetry specification.
   • Utilize the OpenTelemetry Collector for preprocessing and routing the data as needed.

3. OpenTelemetry Collector Configuration:

   • Refer to /e2e/opentelemetry/filelog/otel-collector.yaml for configuring the OpenTelemetry Collector. Ensure it's set up to receive logs or custom telemetry data from the client-side SDK.
   • Use processors in the collector configuration to batch, filter, or transform data as required before exporting it to the backend.

4. Documentation and Examples:

   • Review /internal-docs/rfcs/001-logging.md for insights into the logging feature implementation using OpenTelemetry.
   • Check /docs-content/getting-started/7_native-opentelemetry/1_overview.md for an overview of integrating OpenTelemetry with your application.

This approach leverages OpenTelemetry's comprehensive tooling for telemetry data to streamline the process of sending rrweb data, ensuring efficient batching and processing.

References

highlight/highlight/sdk/client/src/index.tsx
highlight/highlight/backend/public-graph/graph/resolver.go
highlight/highlight/backend/kafka-queue/types.go
highlight/highlight/internal-docs/rfcs/001-logging.md
highlight/highlight/e2e/opentelemetry/filelog/otel-collector.yaml
highlight/highlight/docs-content/getting-started/7_native-opentelemetry/1_overview.md

Ask Greptile