Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Imrove golden image tests #414

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

toaster
Copy link
Contributor

@toaster toaster commented Mar 5, 2024

This PR contains some improvements which I made when working on a fuzzy image test.
It does not affect any logic or test outcome.

But it changes the test behaviour: missing golden images now let the tests fail. Thus, they are detected by CI runs.

@coveralls
Copy link

Coverage Status

coverage: 73.322%. remained the same
when pulling bdf57ba on toaster:imrove_golden_image_tests
into 7f3b594 on davidbyttow:master.

@tonimelisma
Copy link
Collaborator

Hey, I think you're right that slowing down the testing is not really worth it for the fuzzy matchin. I'm not sure if you realized but not having existing reference images causes creation of them so with this PR it would be impossible to create new reference images.

Thanks for the thoughtfulness but I don't think we want to venture down this path with the test automation.

@toaster
Copy link
Contributor Author

toaster commented Mar 8, 2024

[…] so with this PR it would be impossible to create new reference images.

That’s not correct, the missing golden images are still being created.

@toaster
Copy link
Contributor Author

toaster commented Mar 13, 2024

@tonimelisma I would kindly ask you to reconsider your decision.
This PR makes the missing golden images more visible by failing the tests and therefore the CI run but it still creates the missing golden images.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants