Peer review is a bureaucratic, consensus-based approach to making decisions. Thus, it is not inherently entertaining and authors like myself need to amuse ourselves as best we can. One of the games I like to play is peer review poker, where certain combinations of review scores are more desirable than others.
Straight: Review scores form a continuous sequence. A straight, particularly if it is anchored at the high end of the spectrum, is extremely desirable because provoking a diversity of reactions means that there is something interesting about the work. The best kind of straight — which I have accomplished only one time, that I can remember — is when each possible score is received exactly once.
Flush: All reviews are the same score. This is not particularly desirable as it indicates a predictable, uncontroversial paper. Of course, if all scores are “strong accept” then predictability doesn’t seem so bad — but this hardly ever happens. The other extreme, all “strong reject” scores, is certainly achievable.
Obviously, two of a kind, etc. are very common peer review outcomes and are not particularly interesting. By taking reviewer expertise into account it should be possible to define a royal flush but probably that’s pushing the analogy too far.
4 responses to “Peer Review Poker”
If you play too much, people don’t break your legs. Do they?
Daniel, the peer review process could be greatly improved if serious threats of leg-breaking were allowed.
A great practical joke is to wait until about the day before the accept/reject, then send out a fake acceptance email. Actually, this has happened “by accident” at least twice that I know of, and once to me. (Later, I got another email that the paper had actually been rejected.) I recall how I felt at the time, and thought this would be a terribly fun thing for everyone to experience. No?
What an excellent analogy!
A full house can sometimes be pretty interesting too by the way.