Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Something is happening, and interfering, with testing #101

Open
PallHaraldsson opened this issue Nov 14, 2022 · 1 comment
Open

Something is happening, and interfering, with testing #101

PallHaraldsson opened this issue Nov 14, 2022 · 1 comment

Comments

@PallHaraldsson
Copy link
Contributor

PallHaraldsson commented Nov 14, 2022

All numbers go up (for apparently no good reason, at least see no recent PRs explaining and such should affect individual languages):
https://niklas-heer.github.io/speed-comparison/pages/2022-11-14T142222.html

e.g. Julia (AOT compiled) which was fastest, so not all go up by same fraction. But the machine is likely loaded while testing. It's still interesting that Julia goes up more than the others, or might well be a random factor I just notice more, since I'm interesting in Julia.

How do you test (and make sure not loaded, or should I say not make sure)? I usually use hyperfine myself. And if important to get right turn off (or disable) the web browser. For a testing machine, noting else should be running, not practical for myself.

This has happened before:
https://niklas-heer.github.io/speed-comparison/pages/2022-10-24T132444.html

@niklas-heer
Copy link
Owner

@PallHaraldsson the tests are run through GitHub Actions on GitHubs/Azures infrastructure inside VMs. They get automatically started by earthly. The used infrastructures varies and that can contribute to difference in results. See #60.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants