Change Request: Add rule performance dashboard

[mnkiefer]

ESLint version

v8.12.0

What problem do you want to solve?

Analysing rule performance currently requires additional scripting to collect/extract more granular timing data (lint time per file per rule):

1. Running ESLint per rule, per file and then collecting the file/rule time data output:

TIMING=1 DEBUG=eslint:cli-engine eslint --no-eslintrc --rule ... file

2. Doing a single ESLint run on all files and then extracting the file/rule time data output:

TIMING=ALL DEBUG=eslint:cli-engine eslint ... files

In addition, one needs to create an overview for an effective presentation of the results.

What do you think is the correct solution?

It would be more convenient and shareable to have:

1. the timing data exposed to the formatters (when TIMING is used)
2. a built-in formatter for that data [optional]

To demonstrate this, here is a sample implementation where (1) ESLint added the timing data (lintOrder, lintTime, lintTimePerRule to the LintResult object) and (2) the formatter html-charts has been applied, which embeds the already established html formatter into a dashboard along with two charts on rule performance:

• The overview shows the overall rule performance per run (i.e. the TIMING output), as well as the individual lint time per file per rule (from the exposed timing), which can help to detect more intricate performance issues that may be overlooked otherwise (based on rule reports only). The analysis has been carried out on a set of 28 *.js files containing valid/invalid recommended rules examples provided by ESLint.

Participation

• I am willing to submit a pull request for this change.

[nzakas]

Thanks for the suggestion. We do already have an issue open to expose timing data via the ESLint API:
#16521

We could probably expand that proposal to include timing data being passed to formatters.

In general we've been trying to avoid adding new formatters to the core, but this one is specialized enough that it might make sense.

We would need an RFC to move forward with this, and some feedback from the rest of the team would be appreciated.

[bmish]

Another related issue is #14597 (comment) where I was imagining a way to output various rule metrics (violations, disable directives, performance stats, etc). So perhaps any solution should be more general than just performance data only.

[github-actions]

Oops! It looks like we lost track of this issue. What do we want to do here? This issue will auto-close in 7 days without an update.

[mnkiefer]

@nzakas: I would love to contribute, please let me know how I can help to move this forward.

[nzakas]

@mnkiefer as noted in my previous comment, the next step would be to create an RFC explaining how this feature would work.

[github-actions]

Oops! It looks like we lost track of this issue. What do we want to do here? This issue will auto-close in 7 days without an update.

[nzakas]

We have an RFC opened for this, so we'll leave this issue open.

[mnkiefer]

@nzakas: With the RFC merged and ESLint v9.0.0 alpha coming soon, would this be a good time to introduce this feature? If so, I would love to help/contribute here too. 👍

[bmish]

Since it's not a breaking change, either before or after could work. But note that we have a lot of breaking change PRs for ESLint v9 open right now which may cause more merge conflicts than usual.

[mnkiefer]

Since it's not a breaking change, either before or after could work.

Before would be great! I would like to use/showcase it for our CDS language ESLint plugin.

But note that we have a lot of breaking change PRs for ESLint v9 open right now which may cause more merge conflicts than usual.

That shouldn't be a problem. I currently have a PR on a fork of ESLint here. Shall I just update/keep using that or open a new one?

[nzakas]

It would probably be best to hold off until after we've shipped the first alpha of v9.0.0, which will have most of the huge breaking changes. Hopefully that will be before the end of December.

[bmish]

That's fair. Perhaps re-target the PR to the main ESLint repo to start getting reviewers so that it will be lined up for then?

[mnkiefer]

That's fair. Perhaps re-target the PR to the main ESLint repo to start getting reviewers so that it will be lined up for then?

Ok, I've re-targeted the PR here. 👍

[nzakas]

#17850 actually completed the work for this issue, so closing.