Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mechanism for configuring the number of tests run when doing randomized search #244

Open
byorgey opened this issue Jul 16, 2020 · 1 comment
Labels
C-Low Hanging Fruit Shouldn't take too much time; ideal issues for new contributors. S-Nice to have Minor importance U-Testing Z-Feature Request Z-Student Good project for a student.

Comments

@byorgey
Copy link
Member

byorgey commented Jul 16, 2020

For example (this is using the yet-unmerged prop branch):

Disco> holds (exists a:N,b:N,c:N. (0 < a < b < c) and (a^2 + b^2 == c^2))
true
Disco> :test (exists a:N,b:N,c:N. (0 < a < b < c) and (a^2 + b^2 == c^2))
  - No example was found: ∃ a, b, c. 0 < a < b < c and a ^ 2 + b ^ 2 == c ^ 2
    Checked 100 possibilities.

It would be nice in this case to be able to crank up the number of tests (which is currently hardcoded to 100) enough to be able to find an example.

@byorgey byorgey added C-Moderate Effort Should take a moderate amount of time to address. S-Nice to have Minor importance U-Testing Z-Feature Request labels Mar 5, 2022
@byorgey
Copy link
Member Author

byorgey commented Mar 5, 2022

Note that whatever mechanism we come up with, it should be usable with both the :test command and with !!! properties.

Maybe just a built-in primitive function testWith : N * Prop -> Prop? Hmm, no, that feels weird because the number of test cases is not an inherent semantic property of a proposition. It is runtime configuration.

What about this: we define syntax for a "configured test" to be something like {! <options> !} <Prop>, but where options part is optional. Then both :test and !!! will take a "configured test". So you could write something like e.g.

:test {! numTests = 1000 !} exists a:N, b:N, c:N. (0 < a < b < c) and (a^2 + b^2 == c^2)

@byorgey byorgey added C-Low Hanging Fruit Shouldn't take too much time; ideal issues for new contributors. and removed C-Moderate Effort Should take a moderate amount of time to address. labels Aug 14, 2024
@byorgey byorgey added the Z-Student Good project for a student. label Oct 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-Low Hanging Fruit Shouldn't take too much time; ideal issues for new contributors. S-Nice to have Minor importance U-Testing Z-Feature Request Z-Student Good project for a student.
Projects
None yet
Development

No branches or pull requests

1 participant