Home

JQF is a feedback-directed fuzz testing platform for Java.

JQF is built on top of junit-quickcheck, which itself lets you write Quickcheck-like generators and properties in a Junit-style test class. JQF enables better input generation using state-of-the-art fuzzing tools such as AFL.

Quickstart

Write a Junit-like test class annotated with @RunWith(JQF.class) and write some test methods annotated with @Fuzz. The arguments to the test methods will be fuzzed by JQF:

@RunWith(JQF.class)
public class DateFormatterTest {

    /* Input params will be generated by JQF, many times. */
    /* Exceptions listed in the "throws" clause are considered normal (tests will pass on throw) */
    @Fuzz
    public void fuzzSimple(Date date, String format) throws IllegalArgumentException {
        // Create a simple date formatter using the input format string
        // May throw IllegalArgumentException for invalid formats
        DateFormat df = new SimpleDateFormat(format);

        // Format the date using the constructed formatter
        df.format(date);
    }
}

Fuzz with AFL:

$ jqf/bin/jqf-afl-fuzz DateFormatterTest fuzzSimple

Grab a coffee while AFL does its thing:

Ooh! We found some crashes. Let's reproduce one such test case and see what the error was.

$ jqf/bin/jqf-repro DateFormatterTest fuzzSimple fuzz-results/crashes/id:000000
java.lang.ArrayIndexOutOfBoundsException: 127
	at java.text.SimpleDateFormat.subFormat(SimpleDateFormat.java)
	at java.text.SimpleDateFormat.format(SimpleDateFormat.java)
	at java.text.SimpleDateFormat.format(SimpleDateFormat.java)
	at java.text.DateFormat.format(DateFormat.java)

This shouldn't happen! DateFormat.format() does not specify that it will throw ArrayIndexOutOfBoundsException. Time to file a bug report :-)

How does this work?

Unlike vanilla Quickcheck, which generates inputs using a random number generator, JQF receives feedback about the test program (such as the execution path for a given test case) that can be used to better guide input generation and reach corner cases that would not be executed by random sampling alone. Thus, JQF enables coverage-guided mutational fuzzing, among other things.

JQF receives feedback from the program by performing on-the-fly bytecode instrumentation using the ASM toolkit. JQF is quite fast, and can usually execute several hundred test cases per second for complex tests such as HTTP request parsing using Apache Tomcat and PNG decoding using Java ImageIO.

More details

Building JQF

To build JQF, you need Maven (mvn) and GNU Make (make) installed and on your path.

Clone the JQF repo and run setup.sh.

git clone https://github.com/rohanpadhye/jqf
jqf/setup.sh 

Using JQF

Writing tests

JQF builds on top of junit-quickcheck, so any valid quickcheck property and input generator works with JQF. The junit-quickcheck documentation is a great place to learn to write quickcheck tests and generators. Just remember to update the annotations on your test class from @RunWith(JUnitQuickcheck.class) to @RunWith(JQF.class) and the annotations on your test methods from @Property to @Fuzz if you want them to work with JQF.

We also have a JQF-centric tutorial on writing an effective fuzz target.

Fuzzing with AFL

JQF ships with out-of-the-box support for fuzzing with American Fuzzy Lop (AFL). AFL has been used to find numerous bugs in C (and LLVM) programs using lightweight instrumentation and coverage-guided fuzzing.

AFL executes an instrumented C program or binary in an infinite loop, collecting coverage information to guide its input generation. If the program exits abruptly (e.g. because of a segfault), then AFL marks the corresponding input as a crash.

JQF provides a wrapper program called jqf-afl-fuzz that works similar to the program afl-fuzz, but invokes a JVM and runs the provided test class with JQF, which in turn instruments the application classes and provides coverage feedback to AFL. If the test fails due to an AssertionError or an unexpected RunTimeException, then JQF informs AFL that the program has crashed. Read the article on fuzzing Java with AFL for more details.

Hacking on JQF

One of JQF's core features is its ease of extensibility. Although a coverage-guided fuzz-with-AFL mode is provided out of the box, you can easily change the front-end to use different types of feedback (other than branch coverage) and input generation techniques other than AFL. This can be achieved by extending the Guidance interface.

If you delve into the API exposed by JQF, you may also be interested in knowing more about the program instrumentation that JQF uses and the way it generates trace events for use in guided fuzzing. We've outlined some key points in the article on implementation details.

Contact the developers

We want your feedback! (haha, get it? get it?)

If you've found a bug in JQF or are having trouble getting JQF to work, please open an issue on the issue tracker. You can also use this platform to post feature requests.

If it's some sort of fuzzing emergency you can always send an email to the main developer: Rohan Padhye.