-
-
Notifications
You must be signed in to change notification settings - Fork 175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
Give fc.statistics a return value #4342
Comments
dubzzz
added
馃挕 Idea
鉁旓笍 Idea to investigate
馃殌 Feature Request
鉁旓笍 Feature Accepted
and removed
馃挕 Idea
鉁旓笍 Idea to investigate
labels
Oct 21, 2023
What you are suggesting is close to the feature request: #342 馃 |
It is related but I think my request is more focused. Just return the data that is already there :). No need for you to come up with any new APIs for labeling properties or asserting on statistical significance. Others can experiment with those using the data returned from this existing API. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
馃挕 Idea
If fc.statistics returned the distribution of classifications in addition to printing it, I'd have a straightforward way to property test my arbitraries. Would be nice to also have the number of skipped generated values.
Motivation
I've been trying my hand at building custom arbitraries, and noting that bugs can definitely sneak into those. In particular, I'm concerned that the arbitraries might not be well distributed across the input space, and manually printing out statistics is a bit of a pain. So my property tests could be passing but not really finding edge cases (e.g. imagine if my custom X generator just always returns empty X for some reason). Even if I get it right, someone else could introduce a regression and we could silently lose coverage that we thought we had. So I'm looking for ways to property test the arbitraries themselves.
Example
Thanks for considering!
The text was updated successfully, but these errors were encountered: