How to determine the skills of a software developer in a job interview?

Question

I have come to a new challenge in my career that involves handling some job interviews, and I must determine if the applicant really has mastered the skills he/she lists in his/her CV.

How do I accomplish this? Do I:

• Prepare an assignment?
• Ask concrete technical questions related to what we need?
• Do something else?

Edit: There will be two persons conducting the interview. The other colleague is from the HR department. I must only be sure about the technical knowledge (Programming and architectural abilities.) of the applicant.

Answer 1

This can be challenging, it is always possible that someone tests well but can't perform in the "real world". In the past when I've interviewed candidates I've prioritized as follows:

1. Personality. You can always train a new hire to become technically competent. You can't ever train them to be a decent human being if they aren't one already. Under this general heading I include traits like honesty, discretion, directness, courage etc. All the "soft skills" that make someone easy to work with... but are notoriously hard to gauge.
2. Mental acuity. If you are looking at a long-term hire it is better to get someone who can break a problem down, think it through and then execute on the plan they develop over someone who can't really think. You could use a logic puzzle of some kind to test them on this.
3. Ability to learn. I wouldn't hire someone who hadn't a clue about what needed to be done in the position they were put in, but tech changes and if they can't keep on learning you will be stuck with someone who makes great punch-cards when you really need someone who can code in JAVA.

Answer 2

Firstly, the interview as a predictor of future performance has a very low validity. Even structured, it is not helpful or very useful. Testing for cognitive ability and conducting "job tests" are two of the leading indicators of performance, with a validity approaching or around r0.5 (Hunter & Hunter 1984).

So creating assignments or test questions is the where you want to go. However, testing predictors must be tested themselves. Is this is not an easy task. Those that create selection tools invest a ton of time and money and the reliability and validity of these tools are still questionable. Might be best to purchase what has already been developed and tested, but be prepared to spend.

If you are unable, get real comfortable with a 50/50 chance of selecting correctly.

Answer 3

Building on Marcin's response.

The Guerilla guide to interviewing (latest version is best!) is very useful.

It helped me a lot when I first had to evaluate technical staff for hiring few years ago during an expansion spurt (I was hiring electronics development and embedded programming, not just general purpose coders for enterprise software)

I'm not 100% sure of all his assertions - Don't really think that always having 6 people interview one person one after the other is a valuable way to spend 6 people's time when they also have project workload, but have to agree that "smart and gets things done" is pretty much the best summary of any good employee you will ever get, and if you keep those two indicators in mind, plus maybe a pinch of "technical culture" you'll be on your way to getting the technical side of things evaluated.

Also - there is nothing more true than this:

"Never say “Maybe, I can’t tell.” If you can’t tell, that means No Hire. It’s really easier than you’d think. Can’t tell? Just say no! If you are on the fence, that means No Hire. Never say, “Well, Hire, I guess, but I’m a little bit concerned about…” That’s a No Hire as well. Mechanically translate all the waffling to “no” and you’ll be all right."

As someone who did once give the nod to a maybe that turned out bad (I was making excuses for bad aptitude test performance and bad explanations in the interview because they seemed confident - but now I know I shouldn't have) I can say that it's the sort of thing that causes a lot of damage and distress to a development team....

Apart from the info from Joel, you need to use whatever tools you have available to get your new team members without taking up all of your productive time.

I had a recruitment agent our company uses, who has basic maths and logic tests, and over a few different hiring sessions these turned out to be a great indicator for people that did well at technical interviews and at actual work afterwards. From what I've seen, you should expect any viable applicant to a technical job to score in the top 5% of the population for these things, even on a really bad day... The recruiter also did a psychological "DISC" analysis report which was always interesting to read, but I think I could live without that if I was paying the recruiting bill out of my own pocket....

For the programming side of my evaluations, I also used a couple of softball C questions, like described in that article. I'd consider them directed discussions rather than actual tests - It's not just that they can do it, but how fast they are and how confident, and the discussions you have along the way. For bonus points, mine were made of simplified parts from our existing code base so they also gave me a bit of a view into how their thinking process worked with our existing "code culture", and gave me a look at how they handled things that are important to us, like state machines.

Interestingly enough, I even tested a junior staff member who did great at the basic aptitude tests but with no real C experience by feeding him the "general knowledge C details" as needed for one of these tests and concentrating on his logic to explain the workings, and then add functionality. He turned out really good.

Answer 4

The most effective interview technique I have found for interviewing programmers is a white board session where we work through a real life case study followed by a programming test.

The white board session gives a good idea of how well the candidate can articulate and share both their understanding of the problem and how they would approach solving it.

The programming test gives an indication, just an indication mind, of how good their skills in the language they will need to use in the job are. I like to use something language specific, such as the 'Elastic Racetrack' problem in Flash and an exercise where the candidate has to write some unit tests to test the behaviour of an API such as a simple sort method.

I typically do programmer interviews as a pair consisting of a senior developer and me - the project manager.

Answer 5

The time during the interview is too small in order to learn about the coding skills of a developer, however it is important to know more about his personality, so have a face-to-face discussion.

When I get the CV of a new candidate I immediately start checking for code from her or him on github in order to see how he or she does the coding. If I don't find any I give him an exercise to see how he is doing. I'm sending out a link to this repository with a short description of the exercise. I don't explain too much, because I want to know how he can solve problems on his own. As you can see one guy did the assignment and he did great. If I don't know too much about the domain I ask an experienced developer for help. With this approach I know what kind of developer is the candidate. The team work and other social skills can be checked during the face-to-face interview.

There was a time when the company spotify didn't ask for CVs, they asked for github repository links.

Answer 6

Would you hire a magician without asking them to show you some magic tricks? - Joel Spolsky.

Read Jole's guide: The Guerrilla Guide to Interviewing

Answer 7

The interview is only one part of the hiring process. Speaking with references and past employers is another important aspect. Use the interview to learn more about the candidate, their experiences and expertise claimed. Then, speak with references and past employers to validate/get another perspective on the same experiences and skills claimed.

Answer 8

As to me the better way is make a short pair programming session with SMB from the team. Here is an An Example of Cyber-dojo Usage in Job Interview (video).

It takes from 20 minutes to 1 hour and shows the true level of skills.

Answer 9

If you're a project manager, then you may want to consider leaving the technical aspects of the interviewing to the team, or to the functional managers. As a project manager, even if you have technical skills, the technical and tactical details are really not your responsibility.

Spending time interviewing can also take away from your duties of actually doing the project management work.

However, if you do participate in interviews, focus on the soft skills. As a project manager, you will have to work with this person, and you'll want to be sure that this person will be a team player and will be someone who you'll be able to work and communicate with.

In this case, it sounds like you may want to participate as an observer in the interview, mostly with the HR interview, and possibly as an observer in the technical interview.