Explicit bias (Range)

[moodmosaic]

Context

I consider using fast-check in production, and so I started diving into some of the internals.

I like the fact that fast-check has its own 'Size' approach (vs typical QuickCheck style where 'Size' just goes from 0 up to 99 as number of tests increase), and as far as I can tell, these are the options to have some control over 'Size':

• Relative size: "-3", "-2", "-1", "=", "+1", "+2", "+3" — offset the global setting baseSize by the passed offset
• Explicit size: "xsmall", "small", "medium", "large" or "xlarge" — use an explicit size
• Exact value: "max" — generate entities having up-to maxLength items
• Automatic size: undefined — if maxLength has not been specified or if the global setting defaultSizeToMaxWhenMaxSpecified is false then "=", otherwise "max"

I haven't checked all combinators but I assume that some of them are affected by 'Size' while some are not. Please correct me if I am wrong.

Discussion

I just wanted to point out an alternative approach, the one we've been using in Hedgehog, where combinators are parameterized by a 'Range' type.

A 'Range' describes the 'Bounds' of a number to generate, which may or may not be dependent on a 'Size'.

(Pardon my F# and Haskell.)

-- Haskell
data Range a =
  Range a (Size -> (a, a))

// F#
type Range<'a> =
    | Range of ('a * (Size -> 'a * 'a))

• The first parameter is the 'Origin' (the value we'd like to shrink towards to).
• The second parameter is a function that from 'Size' it gives you the 'Bounds' of the value to be generated.

Based on that, you can construct various types of ranges, as per the pseudo-code below, and have fine-grained control over the scope and shrinking of generated values.

Singleton range

Unaffected of 'Size'

/// Construct a range which represents a constant single value.
let singleton (x : 'a) : Range<'a> =
    Range (x, fun _ -> x, x)

Constant range

Unaffected by 'Size'. Shrinks towards the origin.

/// Construct a range which is unaffected by the size parameter with a
/// origin point which may differ from the bounds.
let constantFrom (z : 'a) (x : 'a) (y : 'a) : Range<'a> =
    Range (z, fun _ -> x, y)

/// Construct a range which is unaffected by the size parameter.
let constant (x : 'a) : ('a -> Range<'a>) =
    constantFrom x x

Linear range

Affected by 'Size'. As 'Size' increases the range increases. Shrinks towards the origin.

/// Construct a range which scales the bounds relative to the size
/// parameter.
let inline linearFrom (z : 'a) (x : 'a) (y : 'a) : Range<'a> =
    Range (z, fun size ->
        let xSized =
          // ...
        let ySized =
          // ...
        xSized, ySized)

/// Construct a range which scales the second bound relative to the size
/// parameter.
let inline linear (x : 'a) : ('a -> Range<'a>) =
  linearFrom x x

Examples

Here's how the concept or Range looks like in F# (which can be closer to TypeScript) and how existing generators are modified to work with the Range type.

(You might notice that these PRs are from 2017, still I'd be happy to discuss further in case you find the concept interesting.)

[dubzzz]

Hi @moodmosaic,

In fast-check the size à-la-QuickCheck has been divided into two distinct parts.

The bias itself - Historically, we introduced a bias that will generate extreme values with higher probability for first runs and lower as we go in the runs. Contrary to the solutions adopted by jsverify, QuickCheck and many others, we don't say: "first runs must only generate small values". Doing that may miss issues as some bugs may appear for long arrays with only small values. More details on the historical bias here.

The size - Starting at the middle of the v2, we introduced the idea of size. The main reasons for this approach was: We want our users to write the contract the code should handle without having to include runtime constraints inherent to the complexity of the algorithm or to fast-check itself. We also want them to be able to select how large there values will be depending on some criteria like is in CI, is in pre-release... So size has two roles:

• Stop users doing like fc.array(arb {maxLength: 1000}) // I want large arrays for this property but I don't care if 1k or 2k... it's arbitrary for me now replaced by fc.array(arb {size:"+2"}) // I want quite large arrays compared to my other structures. With such indication the framework know that 1000 is not a limit that should never be passed but just a runtime/execution constraint .
• Adopt large data in CI, in prerelease... In a setup script, you just have to run fc.configureGlobal({baseSize: 'large'}) and all the generated values that follow the size will be based on the large size, if for CI you want them small you may do the same but with small...

[moodmosaic]

Thank you, @dubzzz. This is helpful.

(How do I mark this as answered?)

[dubzzz]

Ideas cannot be marked as answered, I switched it to Q&A so that you can mark it as resolved