-
The Art and Science of Testcase Reduction for Compiler Bugs
Test case reduction is a common problem faced by programmers where some large input (or more generally, some complicated set of circumstances) causes a program to fail, but we wish to know the smallest input (or the simplest circumstances) that causes the same failure. Reduced test cases are important because: Since the bulk of the…
-
Csmith 2.1 Released
We’ve released version 2.1 of Csmith, our random C program generator that is useful for finding bugs in compilers and other tools that process C code. The total number of compiler bugs found and reported due to Csmith is now more than 400. All Csmith users should strongly consider upgrading. New features in this release…
-
Draft Paper about Better Fuzzing
The other day I posted about a simple, low-effort way to improve the bug-finding performance of a random tester. We now have a draft paper about this topic, it’s joint work between my group at Utah and Alex Groce’s group at Oregon State. The key claim is: … for realistic systems, randomly excluding some features…
-
Draft Paper about Integer Overflow
Last Spring I had a lucky conversation. I was chatting with Vikram Adve, while visiting the University of Illinois, and we realized that we working on very similar projects — figuring out what to do about integer overflow bugs in C and C++ programs. Additionally, Vikram’s student Will and my student Peng had independently created…
-
Overflows in SafeInt
Update from Friday 9/23: The SafeInt developers have already uploaded a new version that fixes the problems described in this post. Nice! I have a minor obsession with undefined behaviors in C and C++. Lately I was tracking down some integer overflows in Firefox — of which there are quite a few — and some…
-
Testing Commercial Compilers
A few weeks ago a reader left this comment: Just out of curiosity John, have you approached any of the big commercial compiler companies about getting free licenses for their products? I don’t work in the compiler business but if a university research time offered to rigorously test my software, for free, I’d say yes.…
-
Proposal for Automated Compiler Bug Reports
[Yesterday I submitted a proposal to Google for a modest amount of money to work on turning large random programs that expose compiler flaws into concise bug reports. Below is a transcription that is mostly faithful (citations are omitted and I changed the example bug report from a floating figure into inline text). Feedback is…
-
Split Vote
In my group’s recent compiler testing paper we wrote: We have never seen an “interesting” split vote where randomized differential testing of a collection of C compilers fails to produce a clear consensus answer Randomized differential testing is just a fancy way of describing this process: Randomly generate a test input Run it through several…
-
Safe From Compiler Bugs?
A few people have asked me: Does there exist a subset of the C language that is not, in practice, miscompiled? The intuition behind the question is perfectly reasonable. First, it is clear that there exist C features, such as bitfields and volatile variables, whose compiler support is not so reliable. Second, there exist C…
-
An Executable Semantics For C Is Useful
The goal of a C/C++ compiler is to turn every sequence of ASCII characters into executable instructions. OK, not really — though it does seem that way sometimes. The real goal of a C/C++ compiler is to map every conforming input into executable instructions that correspond to a legal interpretation of that input. The qualifiers…