So does Clojure's test.check. Even though Clojure ain't typed you can define "specs" of your data and then have generators generating random data (it's not the only use of specs: they can also be used as a way to verify inputs to functions). The generators data can be used to, say, quickly create fake database with lots of data or fed into test.check which then does fuzzy testing and then tries hard to reduce failing test cases to minimal cases.
Comby looks great. I've been wondering recently why there isn't a mature tool for both language agnostic lining and transforms. I started using semgrep for linting various languages but I just as soon want to run transformations.
Maybe Comby is what I'm looking for for both cases.
Comby looks nice, I miss the "tops" tool from the old MacOSX ecosystem.
For a decade or so I've been using the "delta" tool for this[1,2], although it sometimes requires manual intervention to get an extremely small testcase because it has very little understanding of the source language's syntax. On the other hand, that genericness can be an advantage for custom languages and it's usually pretty fast at getting to the nut of the matter anyway.
(On the other hand, I don't have any evidence to suggest that a compiler bug report with a 5-line testcase gets looked at any sooner than one with a 1000-line testcase.)
Aside from "minimization" or "reduction" this technique is also sometimes called "delta debugging"[3].
17 comments
[ 3.2 ms ] story [ 47.8 ms ] threadAny John Hughes talk is worth watching; here’s one where he demonstrates the functionality.
https://youtu.be/gPFSZ8oKjco
I've submitted dozens of bugs with it. It's pretty great.
There are a bunch of related stuff that's interesting:
- code slicing (known as code-splitting in JS community).
- round-trip compilation
https://github.com/marxin/cvise http://embed.cs.utah.edu/creduce/
Maybe Comby is what I'm looking for for both cases.
And exactly what I was looking for when hitting compiler errors in Zig.
Thank you!
Not sure is I should report this bug.
For a decade or so I've been using the "delta" tool for this[1,2], although it sometimes requires manual intervention to get an extremely small testcase because it has very little understanding of the source language's syntax. On the other hand, that genericness can be an advantage for custom languages and it's usually pretty fast at getting to the nut of the matter anyway.
(On the other hand, I don't have any evidence to suggest that a compiler bug report with a 5-line testcase gets looked at any sooner than one with a 1000-line testcase.)
Aside from "minimization" or "reduction" this technique is also sometimes called "delta debugging"[3].
[1] https://github.com/dsw/delta [2] https://tracker.debian.org/pkg/delta [3] https://en.wikipedia.org/wiki/Delta_debugging