Replacng vec with tinyvec

[stevefan1999-personal]

This library is able to make both array/stack based vector and heap-allocated vector based on the need (https://docs.rs/tinyvec/latest/tinyvec/). It also claimed to have near correspondence to alloc::vec::Vec which means we can just use it out of the box given the low amount of API usage.

This is extremely useful on situations like repetition, especially when we have finite count bounds, since we can simply just make an array limited by max bound. And we can just use a heap vector if the upper count is unbounded.

And we can also replace these into try_append (https://docs.rs/tinyvec/latest/tinyvec/struct.ArrayVec.html#method.try_append) somewhere else

rust-peg/peg-macros/translate.rs

Lines 672 to 722 in f62301b

	let (repeat_vec, repeat_step) =
	if result_used || min.is_some() || max.is_some() || sep.is_some() {
	(
	Some(quote_spanned! { span => let mut __repeat_value = vec!(); }),
	Some(quote_spanned! { span => __repeat_value.push(__value); }),
	)
	} else {
	(None, None)
	};
	let max_check = max.map(|max| {
	quote_spanned! { span=> if __repeat_value.len() >= #max { break } }
	});
	let result_check = if let Some(min) = min {
	quote_spanned! { span=>
	if __repeat_value.len() >= #min {
	::peg::RuleResult::Matched(__repeat_pos, #result)
	} else {
	::peg::RuleResult::Failed
	}
	}
	} else {
	quote_spanned! { span=> ::peg::RuleResult::Matched(__repeat_pos, #result) }
	};
	quote_spanned! { span=> {
	let mut __repeat_pos = __pos;
	#repeat_vec
	loop {
	let __pos = __repeat_pos;
	#match_sep
	#max_check
	let __step_res = #inner;
	match __step_res {
	::peg::RuleResult::Matched(__newpos, __value) => {
	__repeat_pos = __newpos;
	#repeat_step
	},
	::peg::RuleResult::Failed => {
	break;
	}
	}
	}
	#result_check
	}}
	}

Not only this would eliminate the hidden allocation once and for all, making the overall parsing more deterministic, but this would also greatly help optimization, or even benefit from SIMD. The only thing left might be about lifetime bound issue, that use-after-free of the ArrayVec could be possible during parser production action. Maybe we should relax it from Vec into &[T], and let the user collect the value to heap if they want to. But this would be an API breaking change that needs special care so I suggest discussing first.

We just need to tell user that if you want to have unbounded repetition then you need to use the alloc feature.

[stevefan1999-personal]

POC usage:

use tinyvec::{array_vec, tiny_vec};

fn main() {
    const MIN: usize = 2;

    let mut unbounded_arr = tiny_vec!([_; MIN]);
    for _ in 1..=4 {
        unbounded_arr.push("test");
    }
    // ["test", "test", "test", "test"]
    println!("{:?}", unbounded_arr);

    let mut bounded_arr = array_vec!([_; MIN]);
    for _ in 1..=MIN {
        bounded_arr.push("test");
    }
    // ["test", "test"]
    println!("{:?}", bounded_arr);

    // Panic!
    bounded_arr.push("test");
}

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=d1938c425d060f67846e2d4d78f3e3bd

[stevefan1999-personal]

Oh I realized why this is not made in the first place. It is not necessary for the production value to have Default. I saw that with the macro ASTs