You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In our tests, some of the input values generated by arbitraries are not as important as others. For example, in this function, the values are clearly important, while the date does not matter much:
function sortNumbersAscending(values: number[], currentDate: Date) {
log(currentDate, "Sorting");
// actually sort, etc.
return sortedValues;
}
When writing the tests on our project, we are interested in using arbitraries for the values, but we also interested in making clear to the reader that any date works. So we end up with tests such as one:
fc.assert(
fc.property(fc.array(fc.integer()), fc.date(), (data, date) => {
const sortedData = sortNumbersAscending(data, date);
for (let i = 1; i < data.length; ++i) {
expect(sortedData[i - 1]).toBeLessThanOrEqual(sortedData[i]);
}
})
);
The issue however is that fast-check will try to vary all inputs, including the date, even though the date does not prove anything interesting about the code. This reduces the number of relevant combinaisons of values passed to the code, while we'd prefer fast-check to vary the data more aggressively, and the date less, or not at all.
We could, of course, pass a hard-coded date (eg. sortNumbersAscending(data, new Date("2000-01-01"))), but then it begins to look suspicious to the reader who might wonder if there is anything special about this date. We think that passing a fast-check-generated value instead makes clearer that any value here is acceptable.
We've started addressing this by introducing helpers for those cases that ensure that only a small number of possible values are generated. Here is an example:
export function fcUnvaryingDate(): Arbitrary<Date> {
return fc.constantFrom(...fc.sample(fc.date(), 1));
}
In some cases, this significantly reduces the number of necessary runs, especially when dealing with enums.
It seems that this is a problem that others might also have. Are there well known patterns to address this issue?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
In our tests, some of the input values generated by arbitraries are not as important as others. For example, in this function, the
values
are clearly important, while thedate
does not matter much:When writing the tests on our project, we are interested in using arbitraries for the
values
, but we also interested in making clear to the reader that any date works. So we end up with tests such as one:The issue however is that fast-check will try to vary all inputs, including the date, even though the date does not prove anything interesting about the code. This reduces the number of relevant combinaisons of values passed to the code, while we'd prefer fast-check to vary the
data
more aggressively, and thedate
less, or not at all.We could, of course, pass a hard-coded date (eg.
sortNumbersAscending(data, new Date("2000-01-01"))
), but then it begins to look suspicious to the reader who might wonder if there is anything special about this date. We think that passing a fast-check-generated value instead makes clearer that any value here is acceptable.We've started addressing this by introducing helpers for those cases that ensure that only a small number of possible values are generated. Here is an example:
In some cases, this significantly reduces the number of necessary runs, especially when dealing with enums.
It seems that this is a problem that others might also have. Are there well known patterns to address this issue?
Beta Was this translation helpful? Give feedback.
All reactions