You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I asked this question in pytest core repo: pytest-dev/pytest#9433
They suggested coming here in case this would be a useful feature within pytest-benchmark.
What do you think?
As always, happy to implement.
What's the problem this feature will solve?
I want to do heavy benchmarking of my pytest tests -- running them multiple times and getting average/mean statistics for the runs.
Something like pytest tests/integration/ --durations=0 --count=100.
Currently the report outputs test durations for each test separately, e.g.:
It clutters the output and makes it harder to catch outliers.
Describe the solution you'd like
I'd like to have some flag (possibly, owned by pytest-repeat) that would command to merge stats for the repeated runs durations and output them in a structured way, e.g.
I asked this question in
pytest
core repo: pytest-dev/pytest#9433They suggested coming here in case this would be a useful feature within
pytest-benchmark
.What do you think?
As always, happy to implement.
What's the problem this feature will solve?
I want to do heavy benchmarking of my pytest tests -- running them multiple times and getting average/mean statistics for the runs.
Something like
pytest tests/integration/ --durations=0 --count=100
.Currently the report outputs test durations for each test separately, e.g.:
It clutters the output and makes it harder to catch outliers.
Describe the solution you'd like
I'd like to have some flag (possibly, owned by pytest-repeat) that would command to merge stats for the repeated runs durations and output them in a structured way, e.g.
Ultimately, this would help when more precise analysis of test durations.
Alternative Solutions
I looked at
pytest-benchmark
but apparently it solves another problem of benchmarking a particular function/fixture.Or maybe my googling skills have deteriorated...
Any comments are appreciated!
If we decide this functionality is indeed useful, I'm happy to implement.
The text was updated successfully, but these errors were encountered: