Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] Same code scan on linux vs Mac reports differently #249

Open
MadaManu opened this issue Oct 23, 2024 · 0 comments
Open

[bug] Same code scan on linux vs Mac reports differently #249

MadaManu opened this issue Oct 23, 2024 · 0 comments
Assignees
Labels
c/bug Category: A issue or PR related to a bug

Comments

@MadaManu
Copy link

🐞Bug Report

Bug Description

On Linux it is reported as:
NOT COVERED CONDITIONALS_NEGATION at utils.go:14:9

On Mac it is reported as:
LIVED CONDITIONALS_NEGATION at utils.go:14:9

Therefore the aggregated results are different:

On Linux:

Test efficacy: 100.00%
Mutator coverage: 95.24%

On Mac:

Test efficacy: 95.24%
Mutator coverage: 100.00%

Which makes it hard to

To Reproduce

Steps to reproduce the behavior:

  1. Have a utils.go file with:
// other logic
if err != nil {
 // handling of error
}
  1. Do not cover this case for CONDITIONALS_NEGATION
  2. Unleash the gremlins on it in Mac
  3. Unleash the gremlins on the same code in Linux
  4. Observe the different reporting

Found behaviour

Discrepancy between the same failure between OSes.

Expected behaviour

The report and the status of the finding should be the same so that consistent thresholds can be set, regardless of the OS that executes the unleash.
This would make things consistent between local runs of development (MacOS) and the build agents, which are linux.

Operating System

  • OS: Oracle Linux Server 8.10 and MacOS 14.7
@MadaManu MadaManu added the c/bug Category: A issue or PR related to a bug label Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c/bug Category: A issue or PR related to a bug
Projects
None yet
Development

No branches or pull requests

2 participants