[REQUEST] Metrics direct logging?

[isaachinman]

Hi all, I've recently deployed Soketi to prod to replace our previous Pusher implementation, and it's been working great.

We're not currently using Prometheus and I'm not terribly interested in setting it up just for the sake of basic monitoring of Soketi.

Is it possible to log the metrics content to the same log thread/output as the main Soketi process? Our log drain/query setup is quite sophisticated and we'd be able to build dashboards based on that output directly.

Thanks!

[codeautopilot]

Potential solution

The user's request is to log metrics directly to the same output as the main Soketi process logs, without setting up Prometheus. To achieve this, we will implement a new method in the Log class for logging metrics, modify the Server class to periodically call this method, and implement the logMetrics method in the Metrics class to format and log the metrics data.

The reasoning for this solution is to provide a seamless integration of metrics logging into the existing logging system of Soketi. By using a consistent log level and a distinguishable prefix for metrics logs, we can ensure that the metrics data is easily identifiable and can be parsed by the user's sophisticated log drain/query setup.

Code

For the src/log.ts file:

export class Log {
    // ... existing methods ...

    static metrics(message: any): void {
        // Prefix the message with a specific identifier for metrics logs
        const metricsMessage = `[METRICS] ${this.prefixWithTime(message)}`;
        // Log the metrics message at an 'info' level or another appropriate level
        this.log(metricsMessage, 'cyan', 'mx-2');
    }

    // ... existing methods ...
}

For the src/server.ts file:

// ... (rest of the imports and Server class)

export class Server {
    // ... (existing properties and methods)

    private metricsLogInterval: NodeJS.Timer | null = null; // Add a property to hold the interval

    // ... (rest of the constructor and methods)

    async start(callback?: CallableFunction) {
        // ... (existing start logic)

        // After the server has started, set up the metrics logging if enabled
        if (this.options.metrics.enabled && this.metricsManager) {
            // Set up an interval to log metrics periodically
            this.metricsLogInterval = setInterval(() => {
                this.metricsManager.logMetrics(); // This method should be implemented in the Metrics class
            }, 10000); // Log every 10 seconds, for example
        }

        // ... (rest of the start logic)
    }

    // ... (rest of the methods)

    stop(): Promise<void> {
        // ... (existing stop logic)

        // Clear the metrics logging interval when stopping the server
        if (this.metricsLogInterval) {
            clearInterval(this.metricsLogInterval);
            this.metricsLogInterval = null;
        }

        // ... (rest of the stop logic)
    }

    // ... (rest of the Server class)
}

// ... (rest of the file)

For the src/metrics/metrics.ts file:

import * as prom from 'prom-client';
// ... other imports ...

export class Metrics implements MetricsInterface {
    // ... existing code ...

    /**
     * Log the metrics data to the standard output or a log file.
     */
    logMetrics(): void {
        if (!this.server.options.metrics.enabled) {
            return;
        }

        // Retrieve metrics as JSON or another suitable format
        this.getMetricsAsJson().then(metrics => {
            if (metrics) {
                // Format the metrics data into a log-friendly string
                const metricsLog = this.formatMetricsForLogging(metrics);

                // Use the Log class to output the formatted metrics
                Log.metrics(metricsLog);
            }
        }).catch(error => {
            Log.error('Error retrieving metrics for logging: ' + error.message);
        });
    }

    /**
     * Format the metrics data for logging.
     */
    private formatMetricsForLogging(metrics: prom.metric[]): string {
        // Implement the logic to convert the metrics array into a string
        // that can be easily read and parsed from log files.
        // This could be a JSON string, or a custom format that suits the user's log drain/query setup.
        return JSON.stringify(metrics, null, 2);
    }

    // ... existing code ...
}

This implementation ensures that metrics are logged in a structured and consistent manner, making it easy for the user to integrate with their existing log analysis tools and build dashboards as needed.

Click here to create a Pull Request with the proposed solution

Files used for this task:

Changes on src/log.ts

To address the user's request for logging metrics directly, we can add a new method to the Log class that specifically handles the logging of metrics data. This method should ensure that the metrics log entries are distinguishable from other log messages and are formatted in a way that can be easily parsed by the user's log drain/query setup.

Here's a practical proposal for implementing the new method in the Log class:

1. Add a new method called metrics to the Log class.
2. This method should accept a message parameter that contains the metrics data.
3. Format the message to include a specific prefix or structure that identifies it as a metrics log entry.
4. Use a consistent log level for all metrics messages, such as info or debug, depending on the user's preference.
5. Ensure that the message is output to the same log stream as the main Soketi process logs.

Here's an example of how the new method could be implemented in the Log class:

export class Log {
    // ... existing methods ...

    static metrics(message: any): void {
        // Prefix the message with a specific identifier for metrics logs
        const metricsMessage = `[METRICS] ${this.prefixWithTime(message)}`;
        // Log the metrics message at an 'info' level or another appropriate level
        this.log(metricsMessage, 'cyan', 'mx-2');
    }

    // ... existing methods ...
}

With this new method in place, the user can call Log.metrics(data) to log metrics data, where data is the formatted metrics information. This method will output the metrics data to the standard output or log file, depending on how the Soketi process is configured, and the log entries will be distinguishable by the [METRICS] prefix.

This implementation assumes that the metrics data is already collected and formatted appropriately before being passed to the Log.metrics method. The actual collection and formatting of metrics data would be handled elsewhere in the application, such as in the Metrics class mentioned in the task.

Changes on src/server.ts

To address the user's request for logging metrics directly without using Prometheus, we need to modify the Server class within the src/server.ts file to periodically call a new method that logs metrics to the same output as the main Soketi process logs.

Here's a practical proposal to implement this feature:

1. Create a New Method for Logging Metrics: In the Metrics class (handled in another file), create a new method called logMetrics that formats the metrics data as log entries.

2. Modify the Server Class: In the Server class within src/server.ts, we need to set up a periodic logging function that calls the new logMetrics method from the Metrics class. This can be done using setInterval to periodically invoke the logging.

3. Integrate with the Server Start Method: In the start method of the Server class, after the server has been successfully started, set up the interval to call the logMetrics method. Ensure that the interval is cleared when the server stops.

Here's a code snippet to illustrate the changes to the src/server.ts file:

// ... (rest of the imports and Server class)

export class Server {
    // ... (existing properties and methods)

    private metricsLogInterval: NodeJS.Timer | null = null; // Add a property to hold the interval

    // ... (rest of the constructor and methods)

    async start(callback?: CallableFunction) {
        // ... (existing start logic)

        // After the server has started, set up the metrics logging if enabled
        if (this.options.metrics.enabled && this.metricsManager) {
            // Set up an interval to log metrics periodically
            this.metricsLogInterval = setInterval(() => {
                this.metricsManager.logMetrics(); // This method should be implemented in the Metrics class
            }, 10000); // Log every 10 seconds, for example
        }

        // ... (rest of the start logic)
    }

    // ... (rest of the methods)

    stop(): Promise<void> {
        // ... (existing stop logic)

        // Clear the metrics logging interval when stopping the server
        if (this.metricsLogInterval) {
            clearInterval(this.metricsLogInterval);
            this.metricsLogInterval = null;
        }

        // ... (rest of the stop logic)
    }

    // ... (rest of the Server class)
}

// ... (rest of the file)

This code snippet demonstrates how to set up and clear a periodic logging function for metrics within the Server class. The actual implementation of the logMetrics method will be handled in the Metrics class (src/metrics/metrics.ts), as per the task description for that file.

Changes on src/metrics/metrics.ts

To address the user's request for logging metrics directly without using Prometheus, we can implement a new method within the Metrics class that formats the metrics data and logs it to the standard output or a log file. This method will be called logMetrics, and it will be responsible for converting the metrics data into a log-friendly format and using the Log class to output the data.

Here's a practical proposal for implementing the logMetrics method in the Metrics class:

import * as prom from 'prom-client';
// ... other imports ...

export class Metrics implements MetricsInterface {
    // ... existing code ...

    /**
     * Log the metrics data to the standard output or a log file.
     */
    logMetrics(): void {
        if (!this.server.options.metrics.enabled) {
            return;
        }

        // Retrieve metrics as JSON or another suitable format
        this.getMetricsAsJson().then(metrics => {
            if (metrics) {
                // Format the metrics data into a log-friendly string
                const metricsLog = this.formatMetricsForLogging(metrics);

                // Use the Log class to output the formatted metrics
                Log.info(metricsLog);
            }
        }).catch(error => {
            Log.error('Error retrieving metrics for logging: ' + error.message);
        });
    }

    /**
     * Format the metrics data for logging.
     */
    private formatMetricsForLogging(metrics: prom.metric[]): string {
        // Implement the logic to convert the metrics array into a string
        // that can be easily read and parsed from log files.
        // This could be a JSON string, or a custom format that suits the user's log drain/query setup.
        return JSON.stringify(metrics, null, 2);
    }

    // ... existing code ...
}

In this implementation, the logMetrics method checks if metrics are enabled, retrieves the metrics data as JSON using the existing getMetricsAsJson method, formats the data for logging, and then logs it using the Log class.

The formatMetricsForLogging method is a placeholder for the actual formatting logic. Depending on the user's log system, you might want to format the metrics data as a JSON string, or in a custom format that includes timestamps, metric names, and values in a way that's easy to parse and visualize in their dashboard.

To ensure that this method is called at appropriate intervals, you would need to set up a periodic logging function in the Server class or wherever the server's main loop is managed. This could be done using setInterval or a similar mechanism to call logMetrics at a regular interval, such as every minute or every 10 seconds, depending on the desired granularity of the metrics.

Remember to handle any potential exceptions that may occur during the retrieval or logging of metrics to ensure the stability of the Soketi process.

Disclaimer: This comment was entirely generated using AI. Be aware that the information provided may be incorrect.

Current plan usage: 36.05%

Have feedback or need help?
Discord
Documentation
[email protected]