Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a simple benchmark suite #33

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Add a simple benchmark suite #33

wants to merge 2 commits into from

Conversation

schneiderfelipe
Copy link
Contributor

@schneiderfelipe schneiderfelipe commented May 18, 2021 •
edited

This adds a simple benchmark suite (resolves #19). A simple report can be generated as well, with graphs such as

Captura de tela de 2021-05-18 00-33-25

The report can be generated with

julia> include("benchmark/benchmarks.jl")
savereport (generic function with 2 methods)

julia> savereport(run(suite))

Most of the benchmarks are very naive and would benefit from critical feedback.

@yurivish
Copy link
Collaborator

Hey,I generally think the Julia project's benchmarking practices are inadequate to correctly evaluate the impact of code changes, in particular the focus on minimum times, and stand by my comment about being surprised if Hyperscript offers performance advantages over any package specifically written to have good performance. :)

I'm also increasingly not the best person to continue to maintain this package, as I haven't used Julia's web ecosystem in a long time. I wonder if there's a possibility of having maintainership taken over by e.g. the folks at JuliaWeb...

@schneiderfelipe
Copy link
Contributor Author

Hey,I generally think the Julia project's benchmarking practices are inadequate to correctly evaluate the impact of code changes, in particular the focus on minimum times,...

That's OK. I understand your point. How could I change this PR to reflect a better way of evaluating the impact of code changes?

...and stand by my comment about being surprised if Hyperscript offers performance advantages over any package specifically written to have good performance. :)

This could change in the future. #13 is an example. I'm interested in working on improving the performance of Hyperscript.jl, if you don't mind. But, for that, a way of measuring performance is required.

@schneiderfelipe
Copy link
Contributor Author

I'm also increasingly not the best person to continue to maintain this package, as I haven't used Julia's web ecosystem in a long time. I wonder if there's a possibility of having maintainership taken over by e.g. the folks at JuliaWeb...

@yurivish Is there someone we could talk to to make this transfer? I could send a message to #web Slack, if you don't mind.

@yurivish
Copy link
Collaborator

@yurivish Is there someone we could talk to to make this transfer? I could send a message to #web Slack, if you don't mind.

Sure, thanks @schneiderfelipe.

How could I change this PR to reflect a better way of evaluating the impact of code changes?

The biggest improvement would be showing distributions instead of individual measurements. E.g. each test could instead be shown as a be a collection of tickmarks

untitled (33)

or a violin plot visualizing the distribution of timing values for each test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Benchmarks
2 participants
@schneiderfelipe @yurivish