Computer Systems Evaluation Mistake #1

Tonight I wrote this in a paper review:

The results section of this paper contains what is probably the most elementary and most annoying flaw that could possibly plague a computer systems paper.  All of the numbers are relative and we’re never told the absolute values.  It’s great that you can achieve a 30% reduction in overhead, but it matters whether this is 30% of 0.1% or 30% of 10000%.  I have to recommend against acceptance of a paper that does this.

Enough said.