Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to understand the statistics output file #795

Open
bungdanar opened this issue Aug 28, 2023 · 2 comments
Open

How to understand the statistics output file #795

bungdanar opened this issue Aug 28, 2023 · 2 comments

Comments

@bungdanar
Copy link

Hello, I have question regarding the statistics file.

I have used evomaster to fuzz my simple application that has 3 endpoints. After the fuzzing was done, I checked my application log and there were many requests causing internal server error (500).

When I check the generated statistics file, it shows only 3 errors5xx and 3 distinct500Faults. Does this mean that even if there are many 500 errors on 1 endpoint, they are still counted as 1 error?

Here it is my statistics file content:

Metric Count
evaluatedTests 167
individualsWithSqlFailedWhere 0
evaluatedActions 892
elapsedSeconds 302
generatedTests 4
generatedTestTotalSize 4
coveredTargets 14
lastActionImprovement 430
distinctActions 3
endpoints 3
covered2xx 1
gqlNoErrors 0
gqlErrors 0
gqlErrorsPerLines 0
errors5xx 3
distinct500Faults 3
failedOracleExpectations 0
potentialFaults 6
@arcuri82
Copy link
Collaborator

Hi,
from Statistics.kt:

// errors5xx - counting only the number of endpoints with 5xx, and NOT last executed line
add(Pair("errors5xx", "" + errors5xx(solution)))
//distinct500Faults - counts the number of 500 (and NOT the other in 5xx), per endpoint, and distinct based on the last
//executed line
add(Pair("distinct500Faults", "" + solution.overall.potential500Faults(idMapper).size ))

@bungdanar
Copy link
Author

I am sorry, it is still not clear to me.

Just to be sure, so evomaster only counts the number of endpoints that have 500 error and doesn't count the number of unique requests/payloads that cause 500 error?

Or does distinct500Faults actually describe the number of unique requests/payloads that cause 500 error?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants