How to determine the skills of a software developer in a job interview?

Question

I have come to a new challenge in my career that involves handling some job interviews, and I must determine if the applicant really has mastered the skills he/she lists in his/her CV.

How do I accomplish this? Do I:

• Prepare an assignment?
• Ask concrete technical questions related to what we need?
• Do something else?

Edit: There will be two persons conducting the interview. The other colleague is from the HR department. I must only be sure about the technical knowledge (Programming and architectural abilities.) of the applicant.

Answer 1

This can be challenging, it is always possible that someone tests well but can't perform in the "real world". In the past when I've interviewed candidates I've prioritized as follows:

1. Personality. You can always train a new hire to become technically competent. You can't ever train them to be a decent human being if they aren't one already. Under this general heading I include traits like honesty, discretion, directness, courage etc. All the "soft skills" that make someone easy to work with... but are notoriously hard to gauge.
2. Mental acuity. If you are looking at a long-term hire it is better to get someone who can break a problem down, think it through and then execute on the plan they develop over someone who can't really think. You could use a logic puzzle of some kind to test them on this.
3. Ability to learn. I wouldn't hire someone who hadn't a clue about what needed to be done in the position they were put in, but tech changes and if they can't keep on learning you will be stuck with someone who makes great punch-cards when you really need someone who can code in JAVA.

Answer 2

Firstly, the interview as a predictor of future performance has a very low validity. Even structured, it is not helpful or very useful. Testing for cognitive ability and conducting "job tests" are two of the leading indicators of performance, with a validity approaching or around r0.5 (Hunter & Hunter 1984).

So creating assignments or test questions is the where you want to go. However, testing predictors must be tested themselves. Is this is not an easy task. Those that create selection tools invest a ton of time and money and the reliability and validity of these tools are still questionable. Might be best to purchase what has already been developed and tested, but be prepared to spend.

If you are unable, get real comfortable with a 50/50 chance of selecting correctly.

Answer 3

Building on Marcin's response.

The Guerilla guide to interviewing (latest version is best!) is very useful.

It helped me a lot when I first had to evaluate technical staff for hiring few years ago during an expansion spurt (I was hiring electronics development and embedded programming, not just general purpose coders for enterprise software)

I'm not 100% sure of all his assertions - Don't really think that always having 6 people interview one person one after the other is a valuable way to spend 6 people's time when they also have project workload, but have to agree that "smart and gets things done" is pretty much the best summary of any good employee you will ever get, and if you keep those two indicators in mind, plus maybe a pinch of "technical culture" you'll be on your way to getting the technical side of things evaluated.

Also - there is nothing more true than this:

"Never say “Maybe, I can’t tell.” If you can’t tell, that means No Hire. It’s really easier than you’d think. Can’t tell? Just say no! If you are on the fence, that means No Hire. Never say, “Well, Hire, I guess, but I’m a little bit concerned about…” That’s a No Hire as well. Mechanically translate all the waffling to “no” and you’ll be all right."

As someone who did once give the nod to a maybe that turned out bad (I was making excuses for bad aptitude test performance and bad explanations in the interview because they seemed confident - but now I know I shouldn't have) I can say that it's the sort of thing that causes a lot of damage and distress to a development team....

Apart from the info from Joel, you need to use whatever tools you have available to get your new team members without taking up all of your productive time.

I had a recruitment agent our company uses, who has basic maths and logic tests, and over a few different hiring sessions these turned out to be a great indicator for people that did well at technical interviews and at actual work afterwards. From what I've seen, you should expect any viable applicant to a technical job to score in the top 5% of the population for these things, even on a really bad day... The recruiter also did a psychological "DISC" analysis report which was always interesting to read, but I think I could live without that if I was paying the recruiting bill out of my own pocket....

For the programming side of my evaluations, I also used a couple of softball C questions, like described in that article. I'd consider them directed discussions rather than actual tests - It's not just that they can do it, but how fast they are and how confident, and the discussions you have along the way. For bonus points, mine were made of simplified parts from our existing code base so they also gave me a bit of a view into how their thinking process worked with our existing "code culture", and gave me a look at how they handled things that are important to us, like state machines.

Interestingly enough, I even tested a junior staff member who did great at the basic aptitude tests but with no real C experience by feeding him the "general knowledge C details" as needed for one of these tests and concentrating on his logic to explain the workings, and then add functionality. He turned out really good.

Answer 4

The most effective interview technique I have found for interviewing programmers is a white board session where we work through a real life case study followed by a programming test.

The white board session gives a good idea of how well the candidate can articulate and share both their understanding of the problem and how they would approach solving it.

The programming test gives an indication, just an indication mind, of how good their skills in the language they will need to use in the job are. I like to use something language specific, such as the 'Elastic Racetrack' problem in Flash and an exercise where the candidate has to write some unit tests to test the behaviour of an API such as a simple sort method.

I typically do programmer interviews as a pair consisting of a senior developer and me - the project manager.

Answer 5

The time during the interview is too small in order to learn about the coding skills of a developer, however it is important to know more about his personality, so have a face-to-face discussion.

When I get the CV of a new candidate I immediately start checking for code from her or him on github in order to see how he or she does the coding. If I don't find any I give him an exercise to see how he is doing. I'm sending out a link to this repository with a short description of the exercise. I don't explain too much, because I want to know how he can solve problems on his own. As you can see one guy did the assignment and he did great. If I don't know too much about the domain I ask an experienced developer for help. With this approach I know what kind of developer is the candidate. The team work and other social skills can be checked during the face-to-face interview.

There was a time when the company spotify didn't ask for CVs, they asked for github repository links.

Answer 6

Coding Exercises:

You'll hear 'coding exercises' a lot as the answer to this question. And while these exercises certainly have some value (they certainly test a person's desire for the position as they have to do up to 20 hours of unpaid work!) you should use them carefully.

One problem is that, more often than not, these exercises are not related to the project/roll. Which seems crazy to me. If you do use exercises with a candidate, make sure they are designed to test their ability to do the job.

There are many other ways to discover if a programmer is a good candidate - without looking at a single line of code!

Get some Face Time:

You can tell so much from just talking to someone. From the inflection and tone of their voice to their body language. Restricting yourself to an audio or email exchange denies you of that valuable character insight.

Many freelancer platforms attempt to protect themselves from being disintermediated by heavily discouraging direct phone contact. Instead, they rely on simple online tests and gameable peer rating systems. While there is nothing wrong with either of the methods, they need to be combined with more personal techniques to build a more complete accurate picture of a candidate.

Ask for their Opinion:

Ask them about their favorite programming language and why. The way they answer will reveal a lot. If someone has a strong opinion about a topic it’s a good indicator they are passionate about it

Check them out on GitHub:

How much do they contribute to the industry at large and open source projects? As well as giving you an insight into their coding this gives you a great insight into their mindset and passion. Open source projects are a way people give back to the industry by helping to fix things for a greater good. Now that sounds like someone I would want on my team

How often do they deliver what they code?

Tinkering with code is one thing. Being able to ship a complete product is another. Find out about projects they have finished, which they haven't and what happened

How well spoken are they?

Are they good communicators? If they behave like a black box then you will struggle to work with them and hire a team around them.

How well do they write?

This is similar to the above point but distinct enough to matter. A great writer will often be better at their job, whether that be marketing or programming. My advice is simple here: when stuck between similar looking candidates, always hire the better writer.

Use a trial period:

It's a safe and fair way to test them on a project without not paying them for their time. At Scalable Path, we give clients a month to evaluate a developer. If it’s a good or bad fit, it will be obvious. And if not, both sides save themselves a lot of trouble and risk by testing out the situation first.

Apart from the coding exercises, these methods can be used by non-programmers and so are very useful for PM's when interviewing.

Answer 7

Would you hire a magician without asking them to show you some magic tricks? - Joel Spolsky.

Read Jole's guide: The Guerrilla Guide to Interviewing

Answer 8

The interview is only one part of the hiring process. Speaking with references and past employers is another important aspect. Use the interview to learn more about the candidate, their experiences and expertise claimed. Then, speak with references and past employers to validate/get another perspective on the same experiences and skills claimed.

Answer 9

As to me the better way is make a short pair programming session with SMB from the team. Here is an An Example of Cyber-dojo Usage in Job Interview (video).

It takes from 20 minutes to 1 hour and shows the true level of skills.

Answer 10

If you're a project manager, then you may want to consider leaving the technical aspects of the interviewing to the team, or to the functional managers. As a project manager, even if you have technical skills, the technical and tactical details are really not your responsibility.

Spending time interviewing can also take away from your duties of actually doing the project management work.

However, if you do participate in interviews, focus on the soft skills. As a project manager, you will have to work with this person, and you'll want to be sure that this person will be a team player and will be someone who you'll be able to work and communicate with.

In this case, it sounds like you may want to participate as an observer in the interview, mostly with the HR interview, and possibly as an observer in the technical interview.