On his blog today, Bertrand Meyer responded to a soon-to-appear paper that criticizes some of his previous work. Frank, open debates about the merits of various research approaches and results are important and yet, for various reasons, the vast majority of these debates are hidden inside program committee meetings, hallway discussions, reading groups, and (to a lesser extent) Q/A periods following conference talks.
The problem with keeping these debates private is that they are inaccessible to the people who probably stand to most benefit from seeing them: students and other new practitioners trying to understand the dynamics of a field. In fact, a few public debates made up some of the most interesting and useful reading I encountered as a grad student:
- Cheriton, Skeen, and Birman’s debate:
- The Linus/Tanenbaum debate
- Leveson and Knight’s reply to their critics
- The DeMillo, Lipton, and Perlis article and its responses; this paper hit such a nerve that people are still rebutting it 30 years later!
- Various responses to the Sokal Affair
Definitely I’m not advocating flame wars, personal attacks, or even rudeness. In general the debates I’ve listed have stayed pretty civil, and certainly Meyer’s response above is a great example of how this kind of thing can be done in a friendly way. Obviously criticism can quickly become personal when careers and reputations are at stake — but this risk is a poor reason to not air technical criticism.
I’ve recently written a couple of critical posts, here and here. Also, some of my own work has been criticized publicly, for example in this thread. It would be fair to say that regardless of which side of the debate I find myself on, I’m struggling a bit to find the right tone. It’s very easy to write a quick blog post or dash off a mailing list response that one isn’t 100% happy to find immortalized on the Internet, and also quite easy for everyone to find using a search engine.
It used to be the case that a public debate had to happen in the “letters” part of some professional publication. Today, obviously, blogs and reddits and such have created a completely different environment. We should take advantage of it.
Update: Coincidentally, today Derek Jones posted an article criticizing use of statistics in software engineering papers. I have not read the papers he describes, but it’s fair to say that there are some systematic problems with the quality of statistical analysis in the CS literature.