Looks pretty nice. I only have a little exposure to jsverify. How does fastcheck compare to that? On the surface it looks a bit more modern using TS, but that doesn’t tell one much.
Looks like it's able to automatically figure out the reversal when you map an arbitrary to an arbitrary rather than having to specifically define it like in jsverify, which is nice.
[I'm the author of fast-check] Here are some of the benefits that you won't have in jsverify and that pushed me to create fast-check:
- oneof can shrink
- map does not need unmap to be passed to be able to shrink
- can find bug (see bias in the doc) - in this example jsverify will not find any bugs: jsc.assert(jsc.forall(jsc.integer(),n => Math.abs(n) <= 50))*
- replay of previous runs based on seed + path
- race conditions detection
- model based testing
*The reason why it does not find the bug is that by default it will always generate small values. If you ask for jsc.integer(0, 1000000) then the test jsc.assert(jsc.forall(jsc.integer(0, 1000000),n => Math.abs(n) > 50)) will always be green too. Not with fast-check, it will find the issue for both cases.
This seems similar in concept to Haskell’s QuickCheck[1]. Both generate tests based on an invariant/“property” of the output and can narrow down to the simplest failing test case.
I got into property-based testing when doing a lot of Scala, and I'm a big fan. Beyond trivial tests, it begins to require a whole different bag of techniques to write good tests. But I think it's worth the investment. The key benefit, IMO, is that it helps remove "happy path bias", in which I, as the author, have a hard time getting out of my assumptions about inputs.
The clojure/clojurescript tooling for QuickCheck is quite powerful - as it has a wide range of generators which are able to generate smarter / more specific data than simply matching on types.
And since clojurescript can be compiled and exported to JavaScript, this makes it available to JavaScript projects. A few years ago I packaged this up as https://github.com/glenjamin/checkers. I expect there are newer versions of the underlying library available now, but as I haven't been working in this ecosystem for a while, I haven't kept the package up to date.
9 comments
[ 2.8 ms ] story [ 35.3 ms ] thread- oneof can shrink
- map does not need unmap to be passed to be able to shrink
- can find bug (see bias in the doc) - in this example jsverify will not find any bugs: jsc.assert(jsc.forall(jsc.integer(),n => Math.abs(n) <= 50))*
- replay of previous runs based on seed + path
- race conditions detection
- model based testing
*The reason why it does not find the bug is that by default it will always generate small values. If you ask for jsc.integer(0, 1000000) then the test jsc.assert(jsc.forall(jsc.integer(0, 1000000),n => Math.abs(n) > 50)) will always be green too. Not with fast-check, it will find the issue for both cases.
[1]: https://hackage.haskell.org/package/QuickCheck
Property based testing framework for JavaScript (like QuickCheck) written in TypeScript
source: https://github.com/dubzzz/fast-check
And since clojurescript can be compiled and exported to JavaScript, this makes it available to JavaScript projects. A few years ago I packaged this up as https://github.com/glenjamin/checkers. I expect there are newer versions of the underlying library available now, but as I haven't been working in this ecosystem for a while, I haven't kept the package up to date.