Open Telemetry is exposing secrets

[jon-zuka]

What version of Elysia is running?

1.1.15

What platform is your computer?

Linux x86_64

What steps can reproduce the bug?

Enable opentelemetry, wirte a post route for sign-in with a body payload that contains the user's password. When you go to Jaeger, you will see the user's password logged.

What is the expected behavior?

By default, function arguments should not be logged.

What do you see instead?

I will try to write tracing manually

Additional information

I don't want to sound critical. I do love Elysia, just to be clear.

Have you try removing the node_modules and bun.lockb and try again yet?

Interesting that this question has to be here.

[ashermyers]

telemetry/telemetry.ts:

import { opentelemetry } from "@elysiajs/opentelemetry";
import { RedactingSpanExporter } from './redactingSpanExporter';
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
import { diag, DiagConsoleLogger, DiagLogLevel } from '@opentelemetry/api';
import { env } from 'bun';

// Debug outgoing telemetry logs in development
if (env.DEBUG == "true") {
    diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);
}

export default function telemetry() {
    const otlpExporter = new OTLPTraceExporter({
        url: 'https://api.axiom.co/v1/traces',
        headers: {
            Authorization: `Bearer ${process.env.AXIOM_TOKEN}`,
            'X-Axiom-Dataset': process.env.AXIOM_DATASET,
        },
    });

    const exporter = new RedactingSpanExporter(otlpExporter);
    return opentelemetry({
        //@ts-ignore - BatchSpanProcessor types are incorrect (this shouldn't cause problems)
        spanProcessors: [new BatchSpanProcessor(exporter)],
    })
}

telemetry/requestIdHandler.ts:

import { context, trace } from '@opentelemetry/api';

export default function requestIdHandler({ response, set }: {
    response: any;
    set: { headers: { [key: string]: string } };
}) {
    const span = trace.getSpan(context.active());
    const requestId = span?.spanContext().traceId || '';

    set.headers['X-Request-Id'] = requestId;

    return response;
}

telemetry/redactingSpanExporter.ts:

import { SpanExporter, ReadableSpan } from '@opentelemetry/sdk-trace-base';
import { ExportResult } from '@opentelemetry/core';

const redactedKeys = {
  request: ['password'],
  response: ['password'],
};

class RedactingSpanExporter implements SpanExporter {
  private exporter: SpanExporter;

  constructor(exporter: SpanExporter) {
    this.exporter = exporter;
  }

  export(
    spans: ReadableSpan[],
    resultCallback: (result: ExportResult) => void
  ): void {
    const redactedSpans = spans.map((span) => this.redactSpan(span));
    this.exporter.export(redactedSpans, resultCallback);
  }

  shutdown(): Promise<void> {
    return this.exporter.shutdown();
  }

  private redactSpan(span: ReadableSpan): ReadableSpan {
    const { attributes } = span;

    this.redactAttribute(attributes, 'http.request.body', redactedKeys.request);
    this.redactAttribute(attributes, 'http.response.body', redactedKeys.response);

    // Handle nested 'attributes.custom' object if present
    if (attributes.custom && typeof attributes.custom === 'object') {
      const customAttributes = attributes.custom as Record<string, any>;
      this.redactAttribute(customAttributes, 'http.request.body', redactedKeys.request);
    }

    return span;
  }

  private redactAttribute(
    attributes: Record<string, any>,
    attributeKey: string,
    keysToRedact: string[]
  ): void {
    const attributeValue = attributes[attributeKey];

    if (attributeValue && typeof attributeValue === 'string') {
      try {
        const parsedValue = JSON.parse(attributeValue);

        keysToRedact.forEach((key) => {
          if (parsedValue[key]) {
            parsedValue[key] = '[REDACTED]';
          }
        });

        attributes[attributeKey] = JSON.stringify(parsedValue);
      } catch (error) {
        console.error(`Error parsing attribute '${attributeKey}':`, error);
      }
    }
  }
}

export { RedactingSpanExporter };

Usage:

import telemetry from './telemetry/telemetry';
import requestIdHandler from './telemetry/requestIdHandler';

export const app = new Elysia()
  .use(telemetry())
  .mapResponse(requestIdHandler)

This redacts sensitive details from being sent as telemetry (which you specify)

In addition, this adds a tracable requestId header to each request