Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new Bonito based referenceupdater prototype #3660

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

SimonDanisch
Copy link
Member

  • simple prototype that shows all reference images with scores from the unzipped artifact, with simple ui to switch between backends and reference/recording
  • Improve UI to be less confusing and to make it easier to switch between record/reference
  • Simple filtering and grouping for failed tests/below score/new images, only show one backend etc
  • Color code failing reference tests
  • Add updating code, allow to update groups (all failed tests, complete backend, filter?)

@MakieBot
Copy link
Collaborator

MakieBot commented Feb 28, 2024

Compile Times benchmark

Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt. All benchmark results are based on the mean time and negative percent mean faster than the base branch. Note, that GLMakie + WGLMakie run on an emulated GPU, so the runtime benchmark is much slower. Results are from running:

using_time = @ctime using Backend
# Compile time
create_time = @ctime fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @ctime Makie.colorbuffer(display(fig))
# Runtime
create_time = @benchmark fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @benchmark Makie.colorbuffer(fig)
using create display create display
GLMakie 3.46s (3.31, 3.59) 0.10+- 434.20ms (398.56, 467.01) 25.99+- 487.98ms (472.04, 499.56) 9.25+- 7.63ms (7.37, 7.85) 0.15+- 26.14ms (25.67, 26.45) 0.27+-
master 3.47s (3.38, 3.58) 0.07+- 403.00ms (390.48, 410.74) 8.27+- 483.24ms (465.18, 520.60) 18.76+- 7.45ms (7.28, 7.71) 0.18+- 25.83ms (25.36, 26.25) 0.33+-
evaluation 1.00x invariant, -0.01s (-0.08d, 0.88p, 0.08std) 0.93x slower❌, 31.2ms (1.62d, 0.02p, 17.13std) 0.99x invariant, 4.74ms (0.32d, 0.56p, 14.01std) 0.98x invariant, 0.18ms (1.12d, 0.06p, 0.16std) 0.99x invariant, 0.32ms (1.05d, 0.07p, 0.30std)
CairoMakie 3.11s (3.04, 3.15) 0.04+- 333.02ms (324.19, 345.48) 8.59+- 147.87ms (142.27, 163.76) 7.77+- 7.63ms (7.46, 7.73) 0.10+- 614.16μs (605.09, 618.04) 4.26+-
master 3.10s (3.05, 3.19) 0.05+- 332.38ms (320.87, 348.20) 9.79+- 146.82ms (141.79, 150.57) 3.03+- 7.64ms (7.57, 7.70) 0.06+- 609.54μs (600.55, 615.43) 4.78+-
evaluation 1.00x invariant, 0.01s (0.21d, 0.71p, 0.04std) 1.00x invariant, 0.64ms (0.07d, 0.90p, 9.19std) 0.99x invariant, 1.04ms (0.18d, 0.75p, 5.40std) 1.00x invariant, -0.01ms (-0.16d, 0.76p, 0.08std) 0.99x invariant, 4.62μs (1.02d, 0.08p, 4.52std)
WGLMakie 3.79s (3.75, 3.83) 0.02+- 335.44ms (329.21, 340.82) 5.37+- 9.29s (9.19, 9.42) 0.09+- 9.99ms (9.54, 11.95) 0.87+- 68.93ms (67.78, 70.03) 0.76+-
master 3.79s (3.77, 3.80) 0.01+- 337.08ms (329.32, 351.42) 7.51+- 9.27s (9.16, 9.35) 0.07+- 9.67ms (9.50, 9.92) 0.14+- 68.59ms (67.90, 69.65) 0.63+-
evaluation 1.00x invariant, 0.01s (0.35d, 0.53p, 0.02std) 1.00x invariant, -1.64ms (-0.25d, 0.65p, 6.44std) 1.00x invariant, 0.01s (0.16d, 0.78p, 0.08std) 0.97x invariant, 0.32ms (0.52d, 0.37p, 0.50std) 1.00x invariant, 0.34ms (0.49d, 0.38p, 0.69std)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants