You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#5776 introduced always benchmarking changes setup.py, not because the changes might influence performance, but because they might break the benchmarking altogether.
The smarter implementation would be to introduce another convenience to benchmarks/bm_runner.py that can go through a benchmark run without the long process of actually running the benchmarks. This could be run as part of standard CI on all pull requests to confirm that any changes have not fundamentally broken benchmarking.
So this basically means benchmarking two commits. ASV only works properly if running at least one benchmark, so we could perhaps introduce one to benchmarks/benchmarks/__init__ that uses artificial data generation to make a very small file then performs a very small operation in the benchmark itself.
Example command:
asv run --bench validate_bm_suite --quick <commits>
Populating <commits> would probably need similar logic to this:
馃摪 Custom Issue
#5776 introduced always benchmarking changes
setup.py
, not because the changes might influence performance, but because they might break the benchmarking altogether.The smarter implementation would be to introduce another convenience to
benchmarks/bm_runner.py
that can go through a benchmark run without the long process of actually running the benchmarks. This could be run as part of standard CI on all pull requests to confirm that any changes have not fundamentally broken benchmarking.Things that would need exercising:
So this basically means benchmarking two commits. ASV only works properly if running at least one benchmark, so we could perhaps introduce one to
benchmarks/benchmarks/__init__
that uses artificial data generation to make a very small file then performs a very small operation in the benchmark itself.Example command:
Populating
<commits>
would probably need similar logic to this:iris/benchmarks/bm_runner.py
Lines 422 to 427 in 3b0ab40
The text was updated successfully, but these errors were encountered: