Measuring code productivity between team members

Question

First question, so here goes!

Between two programmers on the same team, one has produced more code but more defects, whilst the other has produced around 20% less code but significantly less defects by KLOC.

Does established convention (and academic agreement) dictate that the more productive programmer is the first individual? - Defined by his raw output, or can a case be made for the higher quality code?

Answer 1

I understand where you are coming from. However, I am unable to understand what you are trying to achieve from this kind of measurement.

This practice of capturing productivity per programmer / developer based on defects and KLOC has reached its dead end. The practice of measuring was only during the Mainframe days (extended until Procedural programming days) and it was considered as the yardstick for measurement.

This means, you assign the same module to each of the developers in a team. Now, if you try measuring their productivity by individual, it would make sense to some extent. This is because the complexity, dependency, etc. would all remain same. Now, only the intellectual capacity has to be applied. Apart, all other constraints remain the same.

In your case, Developer 'A' would have been assigned with a complex module as compared to Developer 'B'. This could be one of the reason for the increased defect density. For the Developer 'B' the defect density would have been lower today, but it might carry technical debt which could have been overlooked.

Now, it all becomes subjective when you grade Developer 'B' as the best.

Because of such reasons, the productivity measure by defects and KLOC are now not considered as the primary measure. Demotivation is another parameter that was seen between the Developers due such measurement.

This is because every individual is unique, their intelligence and approach towards addressing the problem is different.

Today, the times have changed exponentially by means of innovating in all the phases of SDLC. Adapting and embracing in new ways to produce software (read value) is the recommended procedure.

What makes the developer shine is by following the SDLC patterns and produce value to the Customer as a Team.

I would further want to listen and learn from the Process experts on their perspective.

Hope this helps.

Answer 2

This is essentially impossible to answer. Consider that...

• the second dev - through more careful work or personal testing and fixing - ends up with less defects. But without knowing how much effort it is to fix all the remaining defects it's hard to compare the two measurements.
• you might not care about the defects at all. Your situation might call for getting something delivered asap.
• you might not care about all the defects. Some of them might not be severe enough to warrant spending time to fix them (just yet) when there are still features left to implement.
• even less severe bugs can have a detrimental effect on the codebase that collectively outweighs their face value. They can lead to a broken window syndrome where devs end up caring less about bug-freeness.
• higher defect ratio can go hand in hand with less readable and maintainable code which will make working on the code base more difficult in the future.
• KLOCs are a poor measurement of work accomplished. Different coding styles, can lead to noticeably more or less lines of code. Different type of tasks might require many lines of code that are easy to write where as others require few lines which are very compact and difficult.
• one dev might end up with more complex tasks simply because they prefer that sort of work or are more experienced in particular areas which are more ripe with heavier tasks.
• KLOCs don't capture other desireable activities like coaching, learning, knowledge transfer, documentation, etc.
• a 20% difference is nothing in the grand scheme of things. Skill disparities in programming can take on FACTORS of 20.

Even setting aside all the difficulties with this metric, what do you use if for? Understand that any Metric that is known to matter tends to produce strategies that games it.

Answer 3

There is your question, and there is a question under that which deserves to be addressed. First, your question:

Productivity is actually easy to measure. How many useful discrete units are produced. If you're measuring lines of code, then how many lines of code have a bug in it? The person with the higher number of bug-free lines is more productive.

This article from Harvard Business Review all the way back in 1983 addresses some statistical ways to look at defects and productivity. Nothing has really changed since then.

Now the question under the question: does it matter? It is likely that it doesn't.

If I gave you a file with 10000 lines of code, would it help you? Probably not. It has to be the right code that does the right things. Even to the point of quality, does a bug exist in a single line of code? Sometimes, but often it stems from how multiple lines of code interact. Or, to be almost silly about it, would you rather have this code:

 if (a > b) 
 {
   return a; 
 } 
 else 
 {   
    return b; 
 }

than this code:

return (a > b) ? a : b;

If you don't happen to code, those two snippets do exactly the same thing.

Therefore, it can help us to think less in terms of productivity and more in terms of effectiveness. When a business problem arises, how long does it take for us to solve that problem? How effective are we at prioritizing the most important work?

What I encourage teams to do is to look at how their users measure the value of the development team. One team I worked with automated process, so the primary value driver was the amount of time their users took to successfully complete a particular transaction. This could be achieved through fewer steps in the automated process, reduced user error, and many other things. Measuring how effective the team is at serving their users is almost always more useful than measuring how many artifacts they produce.

Of course, it's almost impossible to separate who did what at an individual level, but that's probably fine. As Yves Morieux elegantly shares in his TED talk, when we demand individual accountability, what we often get is a situation where we may never succeed, but we'll always know who to blame when we fail.

Answer 4

It depends on your overall value proposition. Albeit simplified, consider two different types of home builders: one that produces a bulk of homes in a new development, largely the same, a ton of pre-built pieces, where speed to build, minimizing costs due to thin margins, and higher volume sales prevail; the other a custom builder, hired by a discriminating homeowner, larger margins, lower volume, and where lack of quality will put you out of business.

These two different but both valuable missions create two different needs from a quality spec perspective, time perspective, and skill perspective from labor.

Time, cost, and quality: pick two. Depending on what you pick will determine the type of talent you need and how you consider this idea of "productive."