It’s not uncommon for an empirical CS researcher to get a review saying something like “Sure, these results look good, but we need to reject the paper since the authors never proved anything about the worst case.” Similarly, when I interviewed for faculty jobs ten years ago, a moderately famous professor spent a while grilling [...]
While teaching a CS class I spend quite a bit of time looking over the shoulders of students whose code doesn’t work. Sometimes they have a simple mistake and I’ll either point it out or ask a question that will lead them to the problem. However, other times the code is just generally not very [...]
For a number of years I’ve been asking: If the cost of memory safety bugs in C/C++ codes is significant, and if solutions are available, why aren’t we using them in production systems? Here’s a previous blog post on the subject and a quick summary of the possible answers to my question: The cost of [...]
Reading code is an important skill that doesn’t get enough emphasis in CS programs. There are three main aspects: the external view of the code: documentation, comments, APIs, white papers, information from developers, etc. the static view: reading the code like a book the dynamic view: reading the code as it executes, probably with help from [...]
[This post explains a paper that we recently made available; it's going to be presented at PLDI 2013.] Random testing tools, or fuzzers, are excellent at finding bugs that human testers miss. A particularly important use case for fuzzing is finding exploitable bugs, and companies such as Google use clusters to do high-throughput fuzzing. Whether [...]
“Stochastic Superoptimization” is a fancy way to say “randomized search for fast machine code.” It is also the title of a nice paper that was presented recently at ASPLOS. Before getting into the details, let’s look at some background. At first glance the term “superoptimization” sounds like nonsense because the optimum point is already the best one. [...]
It is often said that exhaustively testing a piece of software is equivalent to performing a proof of correctness. Although this idea is intuitively appealing—and I’ve said it myself a few times—it is incorrect in a technical sense and also in practice. What’s wrong with exhaustive testing in practice? The problem comes from the question: [...]
An idea that I’ve been interested in for a while is that a good test suite should be able to directly support formal verification. How would this work, given that testing usually misses bugs? The basic insight is that a test case is usually telling us about more than just one execution: it’s telling us [...]
[NOTE: Reading this post only makes sense if you read and cared about this previous post.] Ok, evaluating the submissions has been more work than I anticipated, and also things have gotten pretty busy at work, so I’m going to split the evaluation of the submissions into two parts. This first part will discuss objective [...]
UPDATE: As of Saturday March 9, the contest is closed. Results will be posted in a few days. I’ve created a Github repository containing all submissions and my test harness. Regular readers will know that I’m obsessed with integer overflows. One apparently simple piece of code that many programmers get wrong in this respect is [...]