I’m preparing a longer piece on compiler correctness but thought this idea was worth putting into its own short post. The question is:
Which would you choose: A compiler that crashes more often, or one that silently generates incorrect code more often?
Of course, all real compilers have bugs, and all real compilers display both kinds of symptoms. Typically these bugs start to show up when a project pushes the boundaries a bit. People who develop embedded software often encounter both kinds of bugs depressingly often.
I suspect that many readers will have a gut preference for the crashy compiler, based on the idea that it’s better to know early when something is wrong with the compiler. But consider that if you need to ship a product, the crashy compiler is likely to be a showstopper until a workaround is found. Upgrading the compiler is usually totally unacceptable, so the workaround has to be on the application side. On the other hand, a bit of wrong code in some module is probably not a showstopper: development and testing of the rest of the system can proceed.
It is sometimes possible to choose between failure modes for a given bug. Failure oblivious computing (FOC) is based on the idea that we prefer a highly available system over a crashy one. Applied to a compiler, FOC would (at least sometimes) change crash errors into wrong code errors. I’ve talked to compiler developers who use a very FOC-like approach to avoid compiler crashes; they had a compiler pass that would walk various data structures repairing anything that looked fishy. These were smart people and they decided to spend their time masking bugs rather than fixing them. In contrast, other compiler developers use assertions to convert wrong code bugs into crash bugs, and to change dirty crash bugs (segfaults) into clean crashes. What I’m really asking here is not what kind of bugs people should put into code — we have little enough control over that! — but whether we want developers to invest in systems for making bugs show up earlier (assertions) or not at all (FOC).
[poll id=”2″]
5 responses to “Poll: Do You Want Compiler Bugs to Be Quiet or Loud?”
I’d like both options please. When I am not under a deadline pressure and can consider how to handle these bugs, I would like a crashy compiler. Then I can consider if I want to change/upgrade the compile or tweak the application code and I have the time to do so. But on the night before the paper deadline, I’ll take the erroneous compiler in a jiffy. Just produce some code that allows me to produce my graphs.
Both are important and what is really important is a switch to go from one to another.
Any thoughts on such a switch?
I guess the precondition is that the compiler knows there is a bug and two options, crash or hide bug, for it to deal with the bug. More likely, the developer knows the code better than the compiler. Hence the developer has a better chance to know a much safer workaround to mask the bug. The crashy compiler might be better.
People do have some deadline issues. For a demo, a paper or a homework :), it is totally OK to take the little risk of crashing your system during screening. A product aiming at the consumer market should not play such dangerous trick. Intel’s Pentium and iPhone 4’s antenna are the samples.
Given the FOC paper, think about the workaround for the apache server just redirect the browser to somewhere else rather than Amazon after verifying your VISA card…
Can I vote “both”?
Run normally with the buggy one, since the common paths are the ones less likely to have bugs. But also have a continuous monitoring program that compiles it with the crashy one, and uses vcs blame to bug the people with possibly-broken code to fix it.
When chasing an issue, there’s nothing more annoying than something irrelevant getting in the way. (I’ve been bothered by this for a while in VS, where StyleCop prevents debugging if you have so much as one “for(” instead of “for (“.)
I voted for the erroneous compiler, because it allows (unit) testing the output and inspecting the generated assembly code, rather than some semi-documented intermediate representation of code (if possible), or simply getting an unspecific error without knowing where to look for it..
Of course, also making errors be loud as early as possible is also nice. But I can deal with miscompiled code more easily than with compiler crashes.
Voted crashy, but it’s the nature of the small systems I am working with today: if it crashes, I can just change the code, if it’s wrong, there is nothing anyone can do about it.
Working with a large system over many years, I prefered wrong. We knew there were bugs in the product. If something came out wrong, the end user could work around it, because there was a lot of flexibility and user control in the big system, and the user would know about it if it was critical, because there were external systems.