> Or in other words, a fuzzer is a program that tries to create source code that finds bugs in a compiler.
This is a very narrow definition of a fuzzer. There are a lot of types of fuzzer that do not generate source code, and are not intended to test compilers.
When I was an undergrad in 1997, I created a fuzzer (well, we didn't know to call it that back then) to insert one byte changes into Java classfiles, and then used that to test our bytecode verifier along with Sun and Microsoft's.
Tangentially related, speaking of Java, I wrote a path guided one recently for code running on the JVM: https://github.com/cretz/javan-warty-pig. It's found bugs for me on internal things, but I haven't run it on anything popular to build a trophy case.
Very cool! I think fuzzers are SO interesting. One of my favorite examples is lcamtuf’s post on generating valid JPGs from nothing [1]. Just so awesome.
> This is a very narrow definition of a fuzzer. There are a lot of types of fuzzer that do not generate source code, and are not intended to test compilers
Yeah, that's true, I completely missed out other uses for 'fuzzers', thanks for clarifying that
I wonder, instead of an AST-generating fuzzer, if they could have used AFL on an instrumented binary that may find more possibilities since it just works on paths and bytes without being constrained by what this fuzzer knows are certain constructs.
Can you easily run AFL on .NET (or jitted) code? My understanding was that AFL works by instrumenting paths taken and kinda needed a gcc-produced binary with normal Linux debug symbol stuff.
What AFL needs is _instrumentation_ and of course the easiest way is to get that at compilation step, however you are not constrained by anything to get that part via other means. Check https://github.com/ivanfratric/winafl that uses DynamoRIO.
So, to answer your question -- it wouldn't be easy but it can be done.
I'm glad you liked that! (I spent more time that I should've trying to show that in a meaningful way and I was pretty glad when I remembered about 'Beyond Compare')
19 comments
[ 2.7 ms ] story [ 51.1 ms ] threadThis is a very narrow definition of a fuzzer. There are a lot of types of fuzzer that do not generate source code, and are not intended to test compilers.
Suffice it to say, we found lots of bugs, the most famous one being what my grad advisor called the vacuum bug that could read a web client's environment variables (among other things): https://archive.nytimes.com/www.nytimes.com/library/cyber/un...
1: https://lcamtuf.blogspot.com/2014/11/pulling-jpegs-out-of-th...
Yeah, that's true, I completely missed out other uses for 'fuzzers', thanks for clarifying that
So, to answer your question -- it wouldn't be easy but it can be done.
On a side node, my old project https://github.com/dyjakan/interpreter-bugs along with short presentation I did on WarCon 2017 https://github.com/dyjakan/conference-talks/blob/master/2017... (references might be interesting for others).