"A crash was detected, but the fuzzer cannot recover the crashing input."

[teymour-aldridge]

I've recently been trying to write a fairly complex grammar using the grammar-based AST mutator, but I've run into a bug where Fuzzcheck outputs an error message stating that

running 1 test
test expr::fuzz_expr ... 0ms  START
2ms  
================ SIGNAL 11 ================
0 simplest_cov(0 cov: 0/58 cplx: 0.00) diverse_cov_20(0) max_each_cov_hits(0 sum: 0) diverse_cov_1(0) max_total_cov_hits(0) failures(0) iter/s 0 
2ms  
=================== CRASH DETECTED ===================
A crash was detected, but the fuzzer cannot recover the crashing input.
This should never happen, and is probably a bug in fuzzcheck. Sorry :(
error: test failed, to rerun pass '--lib'

I've also encountered a series of other errors (sometimes with different grammars the tests do work, but it uses ~2-3 GB of RAM to generate the mutator, and then runs as expected using 40-50MB or so, and sometimes the code to build the Rc<Grammar> hangs forever - no stack overflows though, so I'm not really sure what the problem is there). Unfortunately I struggled to produce a minimal reproduction of those, so I've just reported this error.

// This works (just calls `test_mutator` with a timeout of 60s)
#[cfg(test)]
#[test]
fn simple_expression_language() {
    use std::{sync::mpsc::channel, time::Duration};

    use fuzzcheck::mutators::{
        grammar::grammar_based_ast_mutator, testing_utilities::test_mutator,
    };

    #[derive(cgisf::Grammar)]
    #[cgisf_file = "src/expr.grammar"]
    struct ExprLang;

    enum Status<E> {
        Ok,
        Err(E),
    }

    let (sender, receiver) = channel();

    let t1 = std::thread::spawn(move || {
        let grammar = ExprLang::top();
        let mutator = grammar_based_ast_mutator(grammar);
        test_mutator(mutator, 1024.0, 1024.0, false, true, 1024, 1024);
    });

    std::thread::spawn(move || match t1.join() {
        Ok(_) => sender.send(Status::Ok),
        Err(e) => sender.send(Status::Err(e)),
    });

    match receiver.recv_timeout(Duration::from_secs(60)) {
        Ok(_) => (),
        Err(e) => {
            panic!("test failed with: {e:?}");
        }
    }
}

// This fails with error output:
// running 1 test
// test expr::fuzz_expr ... 0ms  START
// 2ms  
// ================ SIGNAL 11 ================
// 0 simplest_cov(0 cov: 0/58 cplx: 0.00) diverse_cov_20(0) // max_each_cov_hits(0 sum: 0) diverse_cov_1(0) max_total_cov_hits(0) // failures(0) iter/s 0 
// 2ms  
// =================== CRASH DETECTED ===================
// A crash was detected, but the fuzzer cannot recover the crashing input.
// This should never happen, and is probably a bug in fuzzcheck. Sorry :(
// error: test failed, to rerun pass '--lib'
#[cfg(test)]
#[test]
fn try_fuzz() {
    let grammar = ExprLang::top();
    let mutator = grammar_based_ast_mutator(grammar).with_string();

    let s1: fuzzcheck::builder::FuzzerBuilder1<(String, AST), _> =
        fuzz_test(|(string, _): &(String, AST)| {
            true
        });
    let s2: fuzzcheck::builder::FuzzerBuilder2<_, _, (String, AST)> = s1.mutator(mutator);
    let s3: fuzzcheck::builder::FuzzerBuilder3<_, _, (String, AST)> = s2.serde_serializer();
    let s4 = s3.default_sensor_and_pool();
    let s5 = s4.arguments_from_cargo_fuzzcheck();
    let res = s5.launch();
    assert!(!res.found_test_failure);
}

// Recursive expansion of cgisf::Grammar! macro
// =============================================

#[derive(Clone)]
struct __GrammarCtx {
    rules: ::std::collections::BTreeMap<
        ::std::string::String,
        ::std::rc::Weak<::fuzzcheck::mutators::grammar::Grammar>,
    >,
}
impl ExprLang {
    pub fn top() -> ::std::rc::Rc<::fuzzcheck::mutators::grammar::Grammar> {
        let ctx = __GrammarCtx {
            rules: ::std::collections::BTreeMap::new(),
        };
        Self::Expr(ctx.clone())
    }
    fn Expr(mut ctx: __GrammarCtx) -> ::std::rc::Rc<::fuzzcheck::mutators::grammar::Grammar> {
        ::fuzzcheck::mutators::grammar::recursive(|Expr| {
            let mut ctx: __GrammarCtx = ctx.clone();
            ctx.rules.insert("Expr".to_string(), Expr.clone());
            ::fuzzcheck::mutators::grammar::alternation([
                ::fuzzcheck::mutators::grammar::alternation([
                    ::fuzzcheck::mutators::grammar::alternation([
                        Self::Term(ctx.clone()),
                        Self::Factorial(ctx.clone()),
                    ]),
                    Self::Addition(ctx.clone()),
                ]),
                Self::Multiplication(ctx.clone()),
            ])
        })
    }
    fn Factorial(ctx: __GrammarCtx) -> ::std::rc::Rc<::fuzzcheck::mutators::grammar::Grammar> {
        ::fuzzcheck::mutators::grammar::concatenation([
            ::fuzzcheck::mutators::grammar::concatenation([
                ::fuzzcheck::mutators::grammar::literal('!'),
            ]),
            ::fuzzcheck::mutators::grammar::recurse(ctx.rules.get("Expr").expect(
                "failed to retrieve element to \
                        recurse to (this is a bug in `cgisf`!)",
            )),
        ])
    }
    fn Addition(ctx: __GrammarCtx) -> ::std::rc::Rc<::fuzzcheck::mutators::grammar::Grammar> {
        ::fuzzcheck::mutators::grammar::concatenation([
            ::fuzzcheck::mutators::grammar::concatenation([
                ::fuzzcheck::mutators::grammar::recurse(ctx.rules.get("Expr").expect(
                    "failed to retrieve element to \
                        recurse to (this is a bug in `cgisf`!)",
                )),
                ::fuzzcheck::mutators::grammar::concatenation([
                    ::fuzzcheck::mutators::grammar::literal('+'),
                ]),
            ]),
            ::fuzzcheck::mutators::grammar::recurse(ctx.rules.get("Expr").expect(
                "failed to retrieve element to \
                        recurse to (this is a bug in `cgisf`!)",
            )),
        ])
    }
    fn Multiplication(ctx: __GrammarCtx) -> ::std::rc::Rc<::fuzzcheck::mutators::grammar::Grammar> {
        ::fuzzcheck::mutators::grammar::concatenation([
            ::fuzzcheck::mutators::grammar::concatenation([
                ::fuzzcheck::mutators::grammar::recurse(ctx.rules.get("Expr").expect(
                    "failed to retrieve element to \
                        recurse to (this is a bug in `cgisf`!)",
                )),
                ::fuzzcheck::mutators::grammar::concatenation([
                    ::fuzzcheck::mutators::grammar::literal('*'),
                ]),
            ]),
            ::fuzzcheck::mutators::grammar::recurse(ctx.rules.get("Expr").expect(
                "failed to retrieve element to \
                        recurse to (this is a bug in `cgisf`!)",
            )),
        ])
    }
    fn Term(ctx: __GrammarCtx) -> ::std::rc::Rc<::fuzzcheck::mutators::grammar::Grammar> {
        ::fuzzcheck::mutators::grammar::alternation([
            ::fuzzcheck::mutators::grammar::concatenation([
                ::fuzzcheck::mutators::grammar::literal('0'),
            ]),
            ::fuzzcheck::mutators::grammar::concatenation([
                ::fuzzcheck::mutators::grammar::literal('1'),
            ]),
        ])
    }
}

[loiclec]

Sorry about this :( I will try to have a look at it this weekend (along with your PR!)

[loiclec]

it's not going to be this weekend in the end, I'm sorry again :/ I hope to look into it not too far in the future though

[teymour-aldridge]

Do you have any suggestions for what I should investigate if I wanted to try and fix this?

[teymour-aldridge]

Sorry to keep asking, but I was wondering if you had had any time to look at this? I completely understand if not, and it would be useful to know if this is something which is unlikely to be investigated in the immediate future.

[loiclec]

Hi Teymour, don't be sorry, I am sorry! These past two months went pretty badly for me, and I avoided thinking about a few responsibilities to focus on other stuff and keep my mood up. It's slowly getting better though :)
I can't give you a timeline because honestly I might not follow it. So I won't look into it immediately, but I'll get to it eventually. Again, I am really sorry :(