Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tweak tuning database plot and comparison scripts #2883

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Tweak tuning database plot and comparison scripts #2883

wants to merge 3 commits into from

Conversation

bernhardmgruber
Copy link
Contributor

@bernhardmgruber bernhardmgruber commented Nov 19, 2024 •
edited
Loading

Here are some tweaks I made to @gevtushenko tuning database plot and comparison scripts. It fixes a few bugs and changes the behavior from plotting aggregated results across sub benchmarks (e.g. aggregating transform.add and transform.triad) to plotting each sub benchmark separately.

@github-actions GitHub Actions
Copy link
Contributor

🟩 CI finished in 5h 06m: Pass: 100%/400 | Total: 8d 04h | Avg: 29m 31s | Max: 1h 25m | Hits: 11%/25894

  • 🟩 libcudacxx: Pass: 100%/118 | Total: 23h 14m | Avg: 11m 48s | Max: 49m 50s | Hits: 3%/9504 

    🟩 cpu
  🟩 amd64              Pass: 100%/110 | Total: 21h 58m | Avg: 11m 59s | Max: 49m 50s | Hits:   3%/9504  
  🟩 arm64              Pass: 100%/8   | Total:  1h 15m | Avg:  9m 25s | Max: 15m 58s
🟩 ctk
  🟩 11.1               Pass: 100%/15  | Total:  1h 42m | Avg:  6m 49s | Max: 44m 06s | Hits:   3%/2182  
  🟩 11.8               Pass: 100%/3   | Total: 29m 00s | Avg:  9m 40s | Max: 15m 55s
  🟩 12.5               Pass: 100%/4   | Total:  2h 41m | Avg: 40m 21s | Max: 49m 50s
  🟩 12.6               Pass: 100%/96  | Total: 18h 21m | Avg: 11m 28s | Max: 46m 14s | Hits:   3%/7322  
🟩 cudacxx
  🟩 ClangCUDA18        Pass: 100%/12  | Total:  2h 24m | Avg: 12m 01s | Max: 18m 44s
  🟩 nvcc11.1           Pass: 100%/15  | Total:  1h 42m | Avg:  6m 49s | Max: 44m 06s | Hits:   3%/2182  
  🟩 nvcc11.8           Pass: 100%/3   | Total: 29m 00s | Avg:  9m 40s | Max: 15m 55s
  🟩 nvcc12.5           Pass: 100%/4   | Total:  2h 41m | Avg: 40m 21s | Max: 49m 50s
  🟩 nvcc12.6           Pass: 100%/84  | Total: 15h 56m | Avg: 11m 23s | Max: 46m 14s | Hits:   3%/7322  
🟩 cudacxx_family
  🟩 ClangCUDA          Pass: 100%/12  | Total:  2h 24m | Avg: 12m 01s | Max: 18m 44s
  🟩 nvcc               Pass: 100%/106 | Total: 20h 49m | Avg: 11m 47s | Max: 49m 50s | Hits:   3%/9504  
🟩 cxx
  🟩 Clang9             Pass: 100%/6   | Total: 34m 21s | Avg:  5m 43s | Max: 11m 28s
  🟩 Clang10            Pass: 100%/3   | Total: 33m 26s | Avg: 11m 08s | Max: 14m 17s
  🟩 Clang11            Pass: 100%/4   | Total: 26m 36s | Avg:  6m 39s | Max: 13m 02s
  🟩 Clang12            Pass: 100%/4   | Total: 28m 51s | Avg:  7m 12s | Max: 16m 20s
  🟩 Clang13            Pass: 100%/4   | Total: 36m 42s | Avg:  9m 10s | Max: 13m 57s
  🟩 Clang14            Pass: 100%/4   | Total: 43m 46s | Avg: 10m 56s | Max: 14m 04s
  🟩 Clang15            Pass: 100%/4   | Total: 51m 24s | Avg: 12m 51s | Max: 19m 36s
  🟩 Clang16            Pass: 100%/4   | Total: 27m 52s | Avg:  6m 58s | Max: 14m 42s
  🟩 Clang17            Pass: 100%/4   | Total: 42m 03s | Avg: 10m 30s | Max: 19m 02s
  🟩 Clang18            Pass: 100%/18  | Total:  3h 43m | Avg: 12m 23s | Max: 18m 44s
  🟩 GCC6               Pass: 100%/2   | Total:  5m 16s | Avg:  2m 38s | Max:  2m 46s
  🟩 GCC7               Pass: 100%/6   | Total: 37m 23s | Avg:  6m 13s | Max: 13m 24s
  🟩 GCC8               Pass: 100%/6   | Total: 38m 41s | Avg:  6m 26s | Max: 14m 14s
  🟩 GCC9               Pass: 100%/6   | Total: 20m 17s | Avg:  3m 22s | Max:  4m 26s
  🟩 GCC10              Pass: 100%/4   | Total: 15m 52s | Avg:  3m 58s | Max:  4m 15s
  🟩 GCC11              Pass: 100%/7   | Total:  1h 15m | Avg: 10m 42s | Max: 15m 55s
  🟩 GCC12              Pass: 100%/4   | Total: 36m 49s | Avg:  9m 12s | Max: 14m 09s
  🟩 GCC13              Pass: 100%/17  | Total:  3h 14m | Avg: 11m 25s | Max: 42m 22s
  🟩 Intel2023.2.0      Pass: 100%/3   | Total:  1h 27m | Avg: 29m 14s | Max: 32m 44s
  🟩 MSVC14.16          Pass: 100%/1   | Total: 44m 06s | Avg: 44m 06s | Max: 44m 06s | Hits:   3%/2182  
  🟩 MSVC14.29          Pass: 100%/2   | Total:  1h 23m | Avg: 41m 31s | Max: 42m 38s | Hits:   3%/4727  
  🟩 MSVC14.39          Pass: 100%/1   | Total: 46m 14s | Avg: 46m 14s | Max: 46m 14s | Hits:   3%/2595  
  🟩 NVHPC24.7          Pass: 100%/4   | Total:  2h 41m | Avg: 40m 21s | Max: 49m 50s
🟩 cxx_family
  🟩 Clang              Pass: 100%/55  | Total:  9h 08m | Avg:  9m 57s | Max: 19m 36s
  🟩 GCC                Pass: 100%/52  | Total:  7h 03m | Avg:  8m 08s | Max: 42m 22s
  🟩 Intel              Pass: 100%/3   | Total:  1h 27m | Avg: 29m 14s | Max: 32m 44s
  🟩 MSVC               Pass: 100%/4   | Total:  2h 53m | Avg: 43m 20s | Max: 46m 14s | Hits:   3%/9504  
  🟩 NVHPC              Pass: 100%/4   | Total:  2h 41m | Avg: 40m 21s | Max: 49m 50s
🟩 gpu
  🟩 v100               Pass: 100%/118 | Total: 23h 14m | Avg: 11m 48s | Max: 49m 50s | Hits:   3%/9504  
🟩 jobs
  🟩 Build              Pass: 100%/110 | Total: 20h 24m | Avg: 11m 08s | Max: 49m 50s | Hits:   3%/9504  
  🟩 NVRTC              Pass: 100%/4   | Total:  1h 34m | Avg: 23m 40s | Max: 29m 42s
  🟩 Test               Pass: 100%/3   | Total:  1h 12m | Avg: 24m 12s | Max: 42m 22s
  🟩 VerifyCodegen      Pass: 100%/1   | Total:  1m 59s | Avg:  1m 59s | Max:  1m 59s
🟩 sm
  🟩 60;70;80;90        Pass: 100%/3   | Total: 29m 00s | Avg:  9m 40s | Max: 15m 55s
  🟩 90                 Pass: 100%/4   | Total: 42m 49s | Avg: 10m 42s | Max: 13m 33s
  🟩 90a                Pass: 100%/8   | Total: 56m 11s | Avg:  7m 01s | Max: 12m 08s
🟩 std
  🟩 11                 Pass: 100%/32  | Total:  4h 17m | Avg:  8m 02s | Max: 30m 08s
  🟩 14                 Pass: 100%/32  | Total:  6h 20m | Avg: 11m 54s | Max: 44m 06s | Hits:   3%/4467  
  🟩 17                 Pass: 100%/30  | Total:  6h 34m | Avg: 13m 08s | Max: 41m 08s | Hits:   3%/2442  
  🟩 20                 Pass: 100%/23  | Total:  5h 59m | Avg: 15m 37s | Max: 49m 50s | Hits:   3%/2595

  • 🟩 cub: Pass: 100%/110 | Total: 4d 00h | Avg: 52m 25s | Max: 1h 23m | Hits: 0%/2964 

    🟩 cpu
  🟩 amd64              Pass: 100%/102 | Total:  3d 16h | Avg: 52m 04s | Max:  1h 23m | Hits:   0%/2964  
  🟩 arm64              Pass: 100%/8   | Total:  7h 35m | Avg: 56m 54s | Max:  1h 04m
🟩 ctk
  🟩 11.1               Pass: 100%/15  | Total: 12h 01m | Avg: 48m 07s | Max:  1h 02m | Hits:   0%/741   
  🟩 11.8               Pass: 100%/3   | Total:  3h 24m | Avg:  1h 08m | Max:  1h 08m
  🟩 12.5               Pass: 100%/4   | Total:  4h 45m | Avg:  1h 11m | Max:  1h 14m
  🟩 12.6               Pass: 100%/88  | Total:  3d 03h | Avg: 51m 46s | Max:  1h 23m | Hits:   0%/2223  
🟩 cudacxx
  🟩 ClangCUDA18        Pass: 100%/4   | Total:  3h 45m | Avg: 56m 15s | Max: 58m 07s
  🟩 nvcc11.1           Pass: 100%/15  | Total: 12h 01m | Avg: 48m 07s | Max:  1h 02m | Hits:   0%/741   
  🟩 nvcc11.8           Pass: 100%/3   | Total:  3h 24m | Avg:  1h 08m | Max:  1h 08m
  🟩 nvcc12.5           Pass: 100%/4   | Total:  4h 45m | Avg:  1h 11m | Max:  1h 14m
  🟩 nvcc12.6           Pass: 100%/84  | Total:  3d 00h | Avg: 51m 33s | Max:  1h 23m | Hits:   0%/2223  
🟩 cudacxx_family
  🟩 ClangCUDA          Pass: 100%/4   | Total:  3h 45m | Avg: 56m 15s | Max: 58m 07s
  🟩 nvcc               Pass: 100%/106 | Total:  3d 20h | Avg: 52m 17s | Max:  1h 23m | Hits:   0%/2964  
🟩 cxx
  🟩 Clang9             Pass: 100%/6   | Total:  4h 50m | Avg: 48m 26s | Max: 52m 18s
  🟩 Clang10            Pass: 100%/3   | Total:  2h 43m | Avg: 54m 22s | Max: 58m 11s
  🟩 Clang11            Pass: 100%/4   | Total:  3h 22m | Avg: 50m 40s | Max: 51m 08s
  🟩 Clang12            Pass: 100%/4   | Total:  3h 35m | Avg: 53m 47s | Max: 56m 02s
  🟩 Clang13            Pass: 100%/4   | Total:  3h 21m | Avg: 50m 20s | Max: 50m 57s
  🟩 Clang14            Pass: 100%/4   | Total:  3h 31m | Avg: 52m 59s | Max: 57m 25s
  🟩 Clang15            Pass: 100%/4   | Total:  3h 34m | Avg: 53m 35s | Max: 57m 21s
  🟩 Clang16            Pass: 100%/4   | Total:  3h 24m | Avg: 51m 12s | Max: 53m 33s
  🟩 Clang17            Pass: 100%/4   | Total:  3h 51m | Avg: 57m 55s | Max:  1h 10m
  🟩 Clang18            Pass: 100%/11  | Total:  9h 08m | Avg: 49m 49s | Max: 58m 07s
  🟩 GCC6               Pass: 100%/2   | Total:  1h 30m | Avg: 45m 12s | Max: 45m 38s
  🟩 GCC7               Pass: 100%/6   | Total:  4h 54m | Avg: 49m 04s | Max: 53m 53s
  🟩 GCC8               Pass: 100%/6   | Total:  5h 14m | Avg: 52m 29s | Max:  1h 06m
  🟩 GCC9               Pass: 100%/6   | Total:  5h 15m | Avg: 52m 37s | Max:  1h 02m
  🟩 GCC10              Pass: 100%/4   | Total:  3h 54m | Avg: 58m 36s | Max:  1h 02m
  🟩 GCC11              Pass: 100%/7   | Total:  7h 11m | Avg:  1h 01m | Max:  1h 08m
  🟩 GCC12              Pass: 100%/4   | Total:  3h 49m | Avg: 57m 20s | Max:  1h 05m
  🟩 GCC13              Pass: 100%/16  | Total: 10h 32m | Avg: 39m 31s | Max:  1h 23m
  🟩 Intel2023.2.0      Pass: 100%/3   | Total:  3h 03m | Avg:  1h 01m | Max:  1h 02m
  🟩 MSVC14.16          Pass: 100%/1   | Total: 59m 10s | Avg: 59m 10s | Max: 59m 10s | Hits:   0%/741   
  🟩 MSVC14.29          Pass: 100%/2   | Total:  2h 16m | Avg:  1h 08m | Max:  1h 13m | Hits:   0%/1482  
  🟩 MSVC14.39          Pass: 100%/1   | Total:  1h 16m | Avg:  1h 16m | Max:  1h 16m | Hits:   0%/741   
  🟩 NVHPC24.7          Pass: 100%/4   | Total:  4h 45m | Avg:  1h 11m | Max:  1h 14m
🟩 cxx_family
  🟩 Clang              Pass: 100%/48  | Total:  1d 17h | Avg: 51m 44s | Max:  1h 10m
  🟩 GCC                Pass: 100%/51  | Total:  1d 18h | Avg: 49m 51s | Max:  1h 23m
  🟩 Intel              Pass: 100%/3   | Total:  3h 03m | Avg:  1h 01m | Max:  1h 02m
  🟩 MSVC               Pass: 100%/4   | Total:  4h 31m | Avg:  1h 07m | Max:  1h 16m | Hits:   0%/2964  
  🟩 NVHPC              Pass: 100%/4   | Total:  4h 45m | Avg:  1h 11m | Max:  1h 14m
🟩 gpu
  🟩 v100               Pass: 100%/110 | Total:  4d 00h | Avg: 52m 25s | Max:  1h 23m | Hits:   0%/2964  
🟩 jobs
  🟩 Build              Pass: 100%/102 | Total:  3d 20h | Avg: 54m 10s | Max:  1h 16m | Hits:   0%/2964  
  🟩 DeviceLaunch       Pass: 100%/1   | Total: 17m 48s | Avg: 17m 48s | Max: 17m 48s
  🟩 GraphCapture       Pass: 100%/1   | Total:  1h 23m | Avg:  1h 23m | Max:  1h 23m
  🟩 HostLaunch         Pass: 100%/3   | Total: 54m 38s | Avg: 18m 12s | Max: 18m 41s
  🟩 TestGPU            Pass: 100%/3   | Total:  1h 24m | Avg: 28m 14s | Max: 39m 11s
🟩 sm
  🟩 60;70;80;90        Pass: 100%/3   | Total:  3h 24m | Avg:  1h 08m | Max:  1h 08m
  🟩 90a                Pass: 100%/4   | Total:  1h 34m | Avg: 23m 44s | Max: 25m 25s
🟩 std
  🟩 11                 Pass: 100%/30  | Total:  1d 01h | Avg: 51m 15s | Max:  1h 14m
  🟩 14                 Pass: 100%/29  | Total:  1d 01h | Avg: 53m 46s | Max:  1h 08m | Hits:   0%/1482  
  🟩 17                 Pass: 100%/27  | Total:  1d 00h | Avg: 53m 47s | Max:  1h 13m | Hits:   0%/741   
  🟩 20                 Pass: 100%/24  | Total: 20h 17m | Avg: 50m 43s | Max:  1h 23m | Hits:   0%/741

  • 🟩 thrust: Pass: 100%/109 | Total: 2d 23h | Avg: 39m 05s | Max: 1h 25m | Hits: 20%/13180 

    🟩 cpu
  🟩 amd64              Pass: 100%/101 | Total:  2d 18h | Avg: 39m 20s | Max:  1h 25m | Hits:  20%/13180 
  🟩 arm64              Pass: 100%/8   | Total:  4h 48m | Avg: 36m 01s | Max: 42m 25s
🟩 ctk
  🟩 11.1               Pass: 100%/15  | Total:  9h 31m | Avg: 38m 06s | Max:  1h 17m | Hits:   0%/2636  
  🟩 11.8               Pass: 100%/3   | Total:  2h 30m | Avg: 50m 07s | Max: 55m 54s
  🟩 12.5               Pass: 100%/4   | Total:  5h 25m | Avg:  1h 21m | Max:  1h 24m
  🟩 12.6               Pass: 100%/87  | Total:  2d 05h | Avg: 36m 56s | Max:  1h 25m | Hits:  25%/10544 
🟩 cudacxx
  🟩 ClangCUDA18        Pass: 100%/4   | Total:  2h 01m | Avg: 30m 17s | Max: 35m 20s
  🟩 nvcc11.1           Pass: 100%/15  | Total:  9h 31m | Avg: 38m 06s | Max:  1h 17m | Hits:   0%/2636  
  🟩 nvcc11.8           Pass: 100%/3   | Total:  2h 30m | Avg: 50m 07s | Max: 55m 54s
  🟩 nvcc12.5           Pass: 100%/4   | Total:  5h 25m | Avg:  1h 21m | Max:  1h 24m
  🟩 nvcc12.6           Pass: 100%/83  | Total:  2d 03h | Avg: 37m 15s | Max:  1h 25m | Hits:  25%/10544 
🟩 cudacxx_family
  🟩 ClangCUDA          Pass: 100%/4   | Total:  2h 01m | Avg: 30m 17s | Max: 35m 20s
  🟩 nvcc               Pass: 100%/105 | Total:  2d 21h | Avg: 39m 26s | Max:  1h 25m | Hits:  20%/13180 
🟩 cxx
  🟩 Clang9             Pass: 100%/6   | Total:  3h 29m | Avg: 34m 57s | Max: 44m 04s
  🟩 Clang10            Pass: 100%/3   | Total:  1h 56m | Avg: 38m 56s | Max: 41m 13s
  🟩 Clang11            Pass: 100%/4   | Total:  2h 20m | Avg: 35m 07s | Max: 37m 49s
  🟩 Clang12            Pass: 100%/4   | Total:  2h 39m | Avg: 39m 51s | Max: 50m 28s
  🟩 Clang13            Pass: 100%/4   | Total:  2h 20m | Avg: 35m 06s | Max: 37m 32s
  🟩 Clang14            Pass: 100%/4   | Total:  2h 36m | Avg: 39m 02s | Max: 43m 57s
  🟩 Clang15            Pass: 100%/4   | Total:  2h 37m | Avg: 39m 26s | Max: 49m 22s
  🟩 Clang16            Pass: 100%/4   | Total:  2h 25m | Avg: 36m 21s | Max: 39m 40s
  🟩 Clang17            Pass: 100%/4   | Total:  2h 36m | Avg: 39m 09s | Max: 49m 14s
  🟩 Clang18            Pass: 100%/11  | Total:  5h 13m | Avg: 28m 31s | Max: 37m 34s
  🟩 GCC6               Pass: 100%/2   | Total:  1h 06m | Avg: 33m 20s | Max: 36m 23s
  🟩 GCC7               Pass: 100%/6   | Total:  3h 57m | Avg: 39m 38s | Max: 46m 34s
  🟩 GCC8               Pass: 100%/6   | Total:  3h 48m | Avg: 38m 07s | Max: 39m 50s
  🟩 GCC9               Pass: 100%/6   | Total:  3h 35m | Avg: 35m 55s | Max: 40m 01s
  🟩 GCC10              Pass: 100%/4   | Total:  2h 31m | Avg: 37m 50s | Max: 40m 32s
  🟩 GCC11              Pass: 100%/7   | Total:  5h 10m | Avg: 44m 25s | Max: 55m 54s
  🟩 GCC12              Pass: 100%/4   | Total:  2h 34m | Avg: 38m 35s | Max: 43m 02s
  🟩 GCC13              Pass: 100%/14  | Total:  6h 08m | Avg: 26m 19s | Max: 43m 10s
  🟩 Intel2023.2.0      Pass: 100%/3   | Total:  2h 54m | Avg: 58m 08s | Max:  1h 02m
  🟩 MSVC14.16          Pass: 100%/1   | Total:  1h 17m | Avg:  1h 17m | Max:  1h 17m | Hits:   0%/2636  
  🟩 MSVC14.29          Pass: 100%/2   | Total:  2h 21m | Avg:  1h 10m | Max:  1h 12m | Hits:   0%/5272  
  🟩 MSVC14.39          Pass: 100%/2   | Total:  1h 51m | Avg: 55m 56s | Max:  1h 25m | Hits:  50%/5272  
  🟩 NVHPC24.7          Pass: 100%/4   | Total:  5h 25m | Avg:  1h 21m | Max:  1h 24m
🟩 cxx_family
  🟩 Clang              Pass: 100%/48  | Total:  1d 04h | Avg: 35m 20s | Max: 50m 28s
  🟩 GCC                Pass: 100%/49  | Total:  1d 04h | Avg: 35m 23s | Max: 55m 54s
  🟩 Intel              Pass: 100%/3   | Total:  2h 54m | Avg: 58m 08s | Max:  1h 02m
  🟩 MSVC               Pass: 100%/5   | Total:  5h 30m | Avg:  1h 06m | Max:  1h 25m | Hits:  20%/13180 
  🟩 NVHPC              Pass: 100%/4   | Total:  5h 25m | Avg:  1h 21m | Max:  1h 24m
🟩 gpu
  🟩 v100               Pass: 100%/109 | Total:  2d 23h | Avg: 39m 05s | Max:  1h 25m | Hits:  20%/13180 
🟩 jobs
  🟩 Build              Pass: 100%/102 | Total:  2d 21h | Avg: 40m 51s | Max:  1h 25m | Hits:   0%/10544 
  🟩 TestCPU            Pass: 100%/4   | Total: 51m 16s | Avg: 12m 49s | Max: 26m 43s | Hits:  99%/2636  
  🟩 TestGPU            Pass: 100%/3   | Total: 42m 18s | Avg: 14m 06s | Max: 17m 27s
🟩 sm
  🟩 60;70;80;90        Pass: 100%/3   | Total:  2h 30m | Avg: 50m 07s | Max: 55m 54s
  🟩 90a                Pass: 100%/4   | Total:  1h 33m | Avg: 23m 18s | Max: 27m 02s
🟩 std
  🟩 11                 Pass: 100%/30  | Total: 16h 40m | Avg: 33m 20s | Max:  1h 13m
  🟩 14                 Pass: 100%/29  | Total: 20h 33m | Avg: 42m 31s | Max:  1h 23m | Hits:   0%/5272  
  🟩 17                 Pass: 100%/27  | Total: 19h 34m | Avg: 43m 30s | Max:  1h 24m | Hits:   0%/2636  
  🟩 20                 Pass: 100%/23  | Total: 14h 13m | Avg: 37m 06s | Max:  1h 25m | Hits:  50%/5272

  • 🟩 cudax: Pass: 100%/54 | Total: 5h 35m | Avg: 6m 12s | Max: 18m 48s | Hits: 2%/246 

    🟩 cpu
  🟩 amd64              Pass: 100%/50  | Total:  5h 16m | Avg:  6m 20s | Max: 18m 48s | Hits:   2%/246   
  🟩 arm64              Pass: 100%/4   | Total: 18m 20s | Avg:  4m 35s | Max:  4m 56s
🟩 ctk
  🟩 12.0               Pass: 100%/19  | Total:  2h 11m | Avg:  6m 55s | Max: 18m 32s | Hits:   2%/123   
  🟩 12.5               Pass: 100%/2   | Total: 23m 18s | Avg: 11m 39s | Max: 12m 09s
  🟩 12.6               Pass: 100%/33  | Total:  3h 00m | Avg:  5m 27s | Max: 18m 48s | Hits:   2%/123   
🟩 cudacxx
  🟩 nvcc12.0           Pass: 100%/19  | Total:  2h 11m | Avg:  6m 55s | Max: 18m 32s | Hits:   2%/123   
  🟩 nvcc12.5           Pass: 100%/2   | Total: 23m 18s | Avg: 11m 39s | Max: 12m 09s
  🟩 nvcc12.6           Pass: 100%/33  | Total:  3h 00m | Avg:  5m 27s | Max: 18m 48s | Hits:   2%/123   
🟩 cudacxx_family
  🟩 nvcc               Pass: 100%/54  | Total:  5h 35m | Avg:  6m 12s | Max: 18m 48s | Hits:   2%/246   
🟩 cxx
  🟩 Clang9             Pass: 100%/2   | Total: 10m 07s | Avg:  5m 03s | Max:  5m 55s
  🟩 Clang10            Pass: 100%/2   | Total: 10m 11s | Avg:  5m 05s | Max:  6m 01s
  🟩 Clang11            Pass: 100%/4   | Total: 18m 45s | Avg:  4m 41s | Max:  5m 44s
  🟩 Clang12            Pass: 100%/4   | Total: 18m 20s | Avg:  4m 35s | Max:  5m 14s
  🟩 Clang13            Pass: 100%/4   | Total: 17m 57s | Avg:  4m 29s | Max:  5m 32s
  🟩 Clang14            Pass: 100%/4   | Total: 29m 36s | Avg:  7m 24s | Max: 18m 32s
  🟩 Clang15            Pass: 100%/2   | Total:  7m 59s | Avg:  3m 59s | Max:  4m 09s
  🟩 Clang16            Pass: 100%/4   | Total: 17m 15s | Avg:  4m 18s | Max:  4m 37s
  🟩 Clang17            Pass: 100%/2   | Total:  7m 34s | Avg:  3m 47s | Max:  3m 51s
  🟩 Clang18            Pass: 100%/2   | Total: 22m 55s | Avg: 11m 27s | Max: 18m 48s
  🟩 GCC9               Pass: 100%/2   | Total:  9m 29s | Avg:  4m 44s | Max:  5m 57s
  🟩 GCC10              Pass: 100%/4   | Total: 19m 10s | Avg:  4m 47s | Max:  6m 07s
  🟩 GCC11              Pass: 100%/4   | Total: 18m 52s | Avg:  4m 43s | Max:  5m 49s
  🟩 GCC12              Pass: 100%/7   | Total:  1h 06m | Avg:  9m 33s | Max: 18m 32s
  🟩 GCC13              Pass: 100%/3   | Total: 13m 00s | Avg:  4m 20s | Max:  4m 56s
  🟩 MSVC14.36          Pass: 100%/1   | Total: 12m 04s | Avg: 12m 04s | Max: 12m 04s | Hits:   2%/123   
  🟩 MSVC14.39          Pass: 100%/1   | Total: 11m 47s | Avg: 11m 47s | Max: 11m 47s | Hits:   2%/123   
  🟩 NVHPC24.7          Pass: 100%/2   | Total: 23m 18s | Avg: 11m 39s | Max: 12m 09s
🟩 cxx_family
  🟩 Clang              Pass: 100%/30  | Total:  2h 40m | Avg:  5m 21s | Max: 18m 48s
  🟩 GCC                Pass: 100%/20  | Total:  2h 07m | Avg:  6m 22s | Max: 18m 32s
  🟩 MSVC               Pass: 100%/2   | Total: 23m 51s | Avg: 11m 55s | Max: 12m 04s | Hits:   2%/246   
  🟩 NVHPC              Pass: 100%/2   | Total: 23m 18s | Avg: 11m 39s | Max: 12m 09s
🟩 gpu
  🟩 v100               Pass: 100%/54  | Total:  5h 35m | Avg:  6m 12s | Max: 18m 48s | Hits:   2%/246   
🟩 jobs
  🟩 Build              Pass: 100%/49  | Total:  4h 08m | Avg:  5m 03s | Max: 12m 09s | Hits:   2%/246   
  🟩 Test               Pass: 100%/5   | Total:  1h 26m | Avg: 17m 23s | Max: 18m 48s
🟩 sm
  🟩 90                 Pass: 100%/1   | Total:  4m 45s | Avg:  4m 45s | Max:  4m 45s
  🟩 90a                Pass: 100%/1   | Total:  3m 20s | Avg:  3m 20s | Max:  3m 20s
🟩 std
  🟩 17                 Pass: 100%/29  | Total:  2h 41m | Avg:  5m 34s | Max: 16m 00s
  🟩 20                 Pass: 100%/25  | Total:  2h 53m | Avg:  6m 56s | Max: 18m 48s | Hits:   2%/246

  • 🟩 cccl: Pass: 100%/6 | Total: 27m 50s | Avg: 4m 38s | Max: 5m 14s

     
    🟩 cpu
  🟩 amd64              Pass: 100%/6   | Total: 27m 50s | Avg:  4m 38s | Max:  5m 14s
🟩 ctk
  🟩 11.1               Pass: 100%/2   | Total:  7m 52s | Avg:  3m 56s | Max:  4m 13s
  🟩 12.0               Pass: 100%/2   | Total:  9m 48s | Avg:  4m 54s | Max:  5m 03s
  🟩 12.6               Pass: 100%/2   | Total: 10m 10s | Avg:  5m 05s | Max:  5m 14s
🟩 cudacxx
  🟩 nvcc11.1           Pass: 100%/2   | Total:  7m 52s | Avg:  3m 56s | Max:  4m 13s
  🟩 nvcc12.0           Pass: 100%/2   | Total:  9m 48s | Avg:  4m 54s | Max:  5m 03s
  🟩 nvcc12.6           Pass: 100%/2   | Total: 10m 10s | Avg:  5m 05s | Max:  5m 14s
🟩 cudacxx_family
  🟩 nvcc               Pass: 100%/6   | Total: 27m 50s | Avg:  4m 38s | Max:  5m 14s
🟩 cxx
  🟩 Clang9             Pass: 100%/1   | Total:  4m 13s | Avg:  4m 13s | Max:  4m 13s
  🟩 Clang14            Pass: 100%/1   | Total:  5m 03s | Avg:  5m 03s | Max:  5m 03s
  🟩 Clang18            Pass: 100%/1   | Total:  5m 14s | Avg:  5m 14s | Max:  5m 14s
  🟩 GCC6               Pass: 100%/1   | Total:  3m 39s | Avg:  3m 39s | Max:  3m 39s
  🟩 GCC12              Pass: 100%/1   | Total:  4m 45s | Avg:  4m 45s | Max:  4m 45s
  🟩 GCC13              Pass: 100%/1   | Total:  4m 56s | Avg:  4m 56s | Max:  4m 56s
🟩 cxx_family
  🟩 Clang              Pass: 100%/3   | Total: 14m 30s | Avg:  4m 50s | Max:  5m 14s
  🟩 GCC                Pass: 100%/3   | Total: 13m 20s | Avg:  4m 26s | Max:  4m 56s
🟩 gpu
  🟩 v100               Pass: 100%/6   | Total: 27m 50s | Avg:  4m 38s | Max:  5m 14s
🟩 jobs
  🟩 Infra              Pass: 100%/6   | Total: 27m 50s | Avg:  4m 38s | Max:  5m 14s

  • 🟩 cccl_c_parallel: Pass: 100%/2 | Total: 9m 46s | Avg: 4m 53s | Max: 7m 34s

     
    🟩 cpu
  🟩 amd64              Pass: 100%/2   | Total:  9m 46s | Avg:  4m 53s | Max:  7m 34s
🟩 ctk
  🟩 12.6               Pass: 100%/2   | Total:  9m 46s | Avg:  4m 53s | Max:  7m 34s
🟩 cudacxx
  🟩 nvcc12.6           Pass: 100%/2   | Total:  9m 46s | Avg:  4m 53s | Max:  7m 34s
🟩 cudacxx_family
  🟩 nvcc               Pass: 100%/2   | Total:  9m 46s | Avg:  4m 53s | Max:  7m 34s
🟩 cxx
  🟩 GCC13              Pass: 100%/2   | Total:  9m 46s | Avg:  4m 53s | Max:  7m 34s
🟩 cxx_family
  🟩 GCC                Pass: 100%/2   | Total:  9m 46s | Avg:  4m 53s | Max:  7m 34s
🟩 gpu
  🟩 v100               Pass: 100%/2   | Total:  9m 46s | Avg:  4m 53s | Max:  7m 34s
🟩 jobs
  🟩 Build              Pass: 100%/1   | Total:  2m 12s | Avg:  2m 12s | Max:  2m 12s
  🟩 Test               Pass: 100%/1   | Total:  7m 34s | Avg:  7m 34s | Max:  7m 34s

  • 🟩 python: Pass: 100%/1 | Total: 15m 29s | Avg: 15m 29s | Max: 15m 29s

     
    🟩 cpu
  🟩 amd64              Pass: 100%/1   | Total: 15m 29s | Avg: 15m 29s | Max: 15m 29s
🟩 ctk
  🟩 12.6               Pass: 100%/1   | Total: 15m 29s | Avg: 15m 29s | Max: 15m 29s
🟩 cudacxx
  🟩 nvcc12.6           Pass: 100%/1   | Total: 15m 29s | Avg: 15m 29s | Max: 15m 29s
🟩 cudacxx_family
  🟩 nvcc               Pass: 100%/1   | Total: 15m 29s | Avg: 15m 29s | Max: 15m 29s
🟩 cxx
  🟩 GCC13              Pass: 100%/1   | Total: 15m 29s | Avg: 15m 29s | Max: 15m 29s
🟩 cxx_family
  🟩 GCC                Pass: 100%/1   | Total: 15m 29s | Avg: 15m 29s | Max: 15m 29s
🟩 gpu
  🟩 v100               Pass: 100%/1   | Total: 15m 29s | Avg: 15m 29s | Max: 15m 29s
🟩 jobs
  🟩 Test               Pass: 100%/1   | Total: 15m 29s | Avg: 15m 29s | Max: 15m 29s

👃 Inspect Changes

Modifications in project?

Project
+/- CCCL Infrastructure
libcu++
CUB
Thrust
CUDA Experimental
python
CCCL C Parallel Library
Catch2Helper

Modifications in project or dependencies?

Project
+/- CCCL Infrastructure
+/- libcu++
+/- CUB
+/- Thrust
+/- CUDA Experimental
+/- python
+/- CCCL C Parallel Library
+/- Catch2Helper

🏃‍ Runner counts (total jobs: 400)
# Runner
326 linux-amd64-cpu16
31 linux-amd64-gpu-v100-latest-1
28 linux-arm64-cpu16
15 windows-amd64-cpu16

@bernhardmgruber
Customize tuning sol and compare scripts
b1a1c73 
* Mark compare as executable
* Plot and compare each sub bench
* Avoid confusing use of 'bw'
* Don't filter the runs
@bernhardmgruber
Show relative change compared to first series
d5ee482
bernhardmgruber
bernhardmgruber commented Nov 26, 2024
View reviewed changes
benchmarks/scripts/sol.py
Comment on lines +110 to +115
def print_speedup(medians):
m = medians.groupby(['alg', 'hue'], sort=False).mean()
m['speedup'] = (m['bw'] / m.groupby(['alg'])['bw'].transform('first'))
print('# Speedups:')
print()
print(m.drop(columns='bw').sort_values(by='speedup', ascending=False).to_markdown())
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels a bit misplaced here, but it's great to have the speedup's from the plot as a copyable table as well., since compare.py works on a much more detailed level.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@miscco miscco miscco approved these changes

Assignees
No one assigned
Labels
None yet
Projects
CCCL
Status: In Review
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bernhardmgruber @miscco