Q: Why does restarting/recompiling often generate corpus? #257
Comments
|
I wish I knew! I’ve observed the same thing and spent some time trying to understand why. I don’t yet. My best theory is that it has to do with the reminimization that occurs on startup, but I don’t have concrete evidence for or an explanation of that. |
|
Somewhat related... I have some mutations implemented locally that are expensive but effective. I’ve been thinking for a while about monitoring coverage and if progress stalls for a while, switch into a “rattle-the-cage” mode where we do a bunch of expensive mutations to try to reach new territory. This is not an unfamiliar approach: I read a paper about folks using a SAT solver (way more expensive!) in such a way. I also tried adding tracing so that I could tell what mutations introduced new coverage. (This is trickier than it sounds; the code base is not set up for it.) The idea was then to treat the problem as a multi-armed bandit, re-weighting the mutations over time to favor the effective ones. Then you can throw in anything. I never got it fully working, but my initial experiments were not positive. Although I did learn that of the two dozen mutations in use, a handful are responsible for the vast majority of the new coverage. I’m happy to share some more details about my more promising experiments if someone is interested in doing some work to getting them integrated. Otherwise, I’ll eventually finish my slow process of cleaning up the worthwhile ones and upstreaming them incrementally. |
|
Yes, it's frequently quite hard to understand what exactly happens inside of a fuzzer and why. Minimization is a possible cause, which suggests that if we implement the "corpus rotation" idea in go-fuzz: |
|
Re multi-armed bandit, I think we need to start with some framework that allows to more explicitly distribute interations (time?) across multiple strategies and with statically assigned priorities. The second (optional) step would be to learn/re-learn the probabilities. |
|
If you read the AFL mailing list, there are lots of patches /papers that improve AFL's coverage fuzzing in certain cases. It might be interesting to see which of those we would want to import into go-fuzz. |
Add flag for automatic shutdown after a specified duration. It seems there can be advantages of restarting fuzzing at regular intervals as seen in dvyukov#257 Also it can be useful to run fuzz tests for a short period to check for regressions. To facilitate this, add a `-duration` parameter. After the fuzzer has been running this duration it will do a regular exit. This will always exit with code 0. It could make sense to exit with non-zero if there it one or more crashers.
Add flag for automatic shutdown after a specified duration. It seems there can be advantages of restarting fuzzing at regular intervals as seen in dvyukov#257 Also it can be useful to run fuzz tests for a short period to check for regressions. To facilitate this, add a `-duration` parameter. After the fuzzer has been running this duration it will do a regular exit. This will always exit with code 0. It could make sense to exit with non-zero if there it one or more crashers.
When running fuzz tests, I very often come across cases where it will run for hours without generating additional cases:
However, when restarting, it will generate new cases, usually within minutes:
To me (no expert) this seems like a missed opportunity, and I am wondering if it is possible to do this without having to manually restart the process?
The text was updated successfully, but these errors were encountered: