Interviewing PhD Students

Although I work with a great collection of students, I’ve come to realize that my selection of students is based too much on luck and not enough on my skill in evaluating them. Therefore, I have a serious question for the professors out there, and particularly for the computer science professors in systems-related areas: Given a student who is interested in working with your group (particularly at the PhD level), how do you decide whether the student is the right one?

To get things started, these are the basic skills I need in a research student:

  1. Ability to clearly explain their thinking and their results. If someone cannot, the rest of the group (and I) become stuck in a protracted struggle to reverse engineer their ideas and results. Also, people who cannot explain themselves clearly seem less receptive to explanations and instructions that I think are totally clear.
  2. Willingness to work hard, and remain interested in a problem and motivated to work on it, over a period of years. Luckily, failures in this area usually manifest early. However, when someone loses focus late it creates problems that are tough to solve since both sides have invested so much time and energy by then.
  3. Capacity to be self-monitoring: there has to be a tight mental feedback loop where they notice errors in thinking, math, and code, and correct them spontaneously. A failure of self-monitoring greatly reduces a student’s productivity because so much time is spent recursively exploring unpromising approaches and just being stuck. My productivity also takes a big hit because the student requires micromanagement and extra oversight: I have to double-check results and end up spending quite a bit of time worrying that I haven’t asked the crucial question that would have uncovered a faulty line of thought. Since most students are doing the right thing 98% of the time, the probing questions I ask at group meetings are without a doubt quite annoying. However, since a low-level error can easily invalidate a high-level result, I have to err on the side of paranoia.
  4. Ability to develop a second, much looser feedback loop where they think about things learned by experience, by conversations, and by reading, and apply the results to their work. They have to be able to learn good taste in problems and solutions and to be willing to try new things. Failure at this level is perhaps the easiest to deal with out of the problems I’ve listed here because I need to supply high-level course corrections anyway, as part of the advising process.
  5. Competence in computer science, in program development, and in math. Ideally, people not fulfilling these requirements are filtered out by our graduate admissions process, and in fact most people we admit are smart enough and good enough programmers to earn a PhD.

Reading back over this, it looks like the most important thing I need to do is develop a collection of interview questions that do a better job evaluating potential in areas 1-3 above. I’d appreciate any advice.