Most people — including many scientists — understand the process of science to be repeated application of the scientific method. In this model, a hypothesis is formulated, experiments are conducted to test the hypothesis, data is analyzed, and the results usually lead to a new hypothesis. This adequately captures the “99% perspiration” aspect of doing science, but misses out on the best part where you synthesize something new, that is not a direct consequence of the data.

In computer science, the writing of papers often works like this: A research group leader says something like “this year we’re going to submit three papers to ISCA / SOSP / SIGGRAPH / other top conference.” The group then chooses the most promising ideas, implements and evaluates them, and finally prepares papers that claim the ideas are good ones.

The goal-driven mode of writing papers has at least three problems. First, it leads to deadline-driven development and writing, which makes it very tempting to dismiss unexpected findings. Second, since the overhead of writing a paper is high, papers start to feel like supertankers that are hard to steer once they start heading in a certain direction. New developments and insights are just not easy to factor into the overall message. Third, the publish-or-perish factor (and also the fear of getting scooped) often leads us to publish too early, decreasing the odds that we’ll have developed enough perspective on the work to see its contributions clearly.

Even when interesting and unexpected results make it into a paper (as opposed to being dismissed outright either by the PI or by a student doing the work) the discussion of them is often buried deep in some subsection of the the paper. When this happens — and the interesting development is not even mentioned in the abstract or conclusion — I call it a “cryptocontribution.” Sometimes these hidden gems are the most interesting parts of what are otherwise pretty predictable pieces of work. When authors are too focused on getting the thing submitted, it’s really easy to shove interesting findings under the rug. Certainly I’ve done it, though I try hard not to.

For years I have enjoyed other people’s cryptocontributions. However, since starting a blog I’ve tried to become more attuned to them in my own writing. Because blogs are different then papers — they’re not high overhead or goal-oriented — I often manage to notice before pressing “publish” that the most interesting part of a post is actually paragraph 17. When this happens I cut out the paragraph and save it in my blog todo list for later elaboration. A fair amount of the time, I end up scrapping the post too — basically it was just a writing exercise for me. (At least one of my colleagues, if he reads this, will be surprised to learn that I scrap anything.)

In summary:

• The world-view promoted by the scientific method is one where progress is linear and incremental. This misses the equally important, orthogonal process of noticing something wrong, or something out of the ordinary, and leaping off in a new direction. By definition, these unexpected results are not direct consequences of the current hypothesis.
• Escaping the constraints of the regular publication system, and instead writing about science in a short, non-goal-directed format such as a blog, makes it easier to bring cryptocontributions out of hiding. I would argue that these unlooked-for contributions are often as valuable as the ones being explicitly sought, and shedding light on them will improve the quality of the science being performed.