Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create fuel-telemetry library and binary #690

Open
alfiedotwtf opened this issue Nov 21, 2024 · 0 comments
Open

Create fuel-telemetry library and binary #690

alfiedotwtf opened this issue Nov 21, 2024 · 0 comments

Comments

@alfiedotwtf
Copy link
Contributor

alfiedotwtf commented Nov 21, 2024

We want to start collecting metrics from our user facing tools (fuelup, forc, and forc plugins) to better understand how they are used. This will enable us improve our tooling over time.

This task will create a new Rust crate with the aim to make it easy to capture and push telemetry from all of our Rust tools. This crate has two parts:

  1. A library to be used within all Rust tools
  2. A binary that uses this library so it can be run manually

As our tools run on end-user machines, we cannot assume users to have an always-on connection to the internet, so we will need to cache captured metrics, and then only purge these cached metrics after a successful push to our online InfluxDB service.

To do this, we do two things when instantiating a telemetry object:

  • Tool side:

    • Open $FUELUP_DIR/telemetry/metrics.<pid> with an advisory lock (released automatically by the OS on exit)
    • Fork off a telemetry daemon
    • Append metrics as InfluxDB Line Protocol into $FUELUP_DIR/telemetry/metrics-<date>.<pid> as needed
  • Daemon side:

    • Open an advisory lock on $FUELUP_DIR/telemetry/sender.lock
    • If the lock was obtained:
      • For each non-empty, unlocked $FUELUP_DIR/telemetry/metrics-* file:
        • Send the cached metrics upstream
        • Purge the metrics file
      • Exit if there are no more locked metrics files

As the daemon will only send metrics from (non-empty) unlocked metrics files, long-running tools (such as fuel-core) will need to rotate out their own locked metrics files so that they are flushed regularly (this will be done automatically within this library).

Note: As InfluxDB works in UPSERT-mode, duplicates from stale metrics files should not matter.

Included within this repo is:

  • code to display a note or prompt the user on how to opt-out of telemetry
  • a directory to store all InfluxDB schema files
@alfiedotwtf alfiedotwtf self-assigned this Nov 21, 2024
@alfiedotwtf alfiedotwtf added code quality and removed new feature New feature labels Dec 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant