52

I am trying to implement KPI for employees of my company. One of the KPI have listed is the quality of code, as measured by bugs created by developers.

Now my problem is: I am planning to give 40 points (my scoring system) to a developer, if he/she creates bug free code.

There can be bugs which are severe, medium and low in nature. How do I rate my developers and give points based on number of bugs that they create?

24
  • 51
    Just for curiosity, have you worked as a programmer, analyst, etc.? And was this kind of measurement suggested by a programmer, analyst, etc.? Perhaps having the affected people involved could help you clarifying this question. Mar 29, 2012 at 12:33
  • 9
    I second the opinion of @woliveirajr. This is the complete opposite way you should go.
    – Tommy B.
    Mar 29, 2012 at 13:07
  • 62
    This idea demonstrates a complete lack of understanding of what makes developers tick and how the development process works. I'll be sitting on the sidelines with a bowl of popcorn watching it crash and burn. Mar 29, 2012 at 13:31
  • 38
    All developers write buggy code. A good developer will find more bugs than a bad developer. If you reward the people who aren't finding bugs ("bug-free" code), you're not rewarding your good developers. Besides, it's not like we're motivated by reward anyway.
    – zzzzBov
    Mar 29, 2012 at 14:06
  • 23

25 Answers 25

144

Please be careful when using these kind of measurements as KPIs.

If you do this, I predict the following:

  • arguments about whether bugs are caused by development or by poor quality analysis
  • arguments about which developers caused which bugs
  • defensive programming, which will increase the maintenance costs
  • developers slowing down (no code means no bugs!)
  • a lack of collaboration between developers
  • and the beginning of a blame culture, if you don't already have one.

This article by Esther Derby suggests alternatives to using KPIs in pay reviews. There's a growing body of evidence that suggests that using performance as a way of measuring pay is destructive. My experience is certainly that it encourages heroism and a blame culture rather than team work, particularly when the KPIs are measuring unwanted items.

15
  • 19
    +1 for @Lunivore answer and I would add one thing. The most interesting metric regarding bugs (IMHO) is the number of bugs that slipped into production. But... You should use it not to evaluate individuals but to spot problems in how the quality process works in your team. So think in which ways you can lower that number. You have lots of means: better tests, CI, training your devs, etc. Use it not to imply penalties but to improve. Mar 29, 2012 at 10:06
  • 10
    What @marcin-niebudek said. Also, add common sense. I found one team whose bug count went up despite their efforts to fix them. It turned out the users had spotted they were fixing the bugs and started reporting more of them.
    – Lunivore
    Mar 29, 2012 at 11:03
  • 3
    I second everything. I would like to add that developers are smart people; more than that, they LOVE to outsmart the others (the system, a computer, etc..). The best devs are challenge driven. In a team where I worked the PM forced this kind of metric. Everyone was so interested in "beating the system" that work eventually become to a halt, with many working on a sw to predict how to maximize their score... Mar 29, 2012 at 13:17
  • 13
    And I am surprised nobody cited Dilbert yet... dilbert.com/strips/comic/1995-11-13 Mar 29, 2012 at 13:20
  • 4
    @Rup It's more code that developers have to read through in order to understand what the code is doing - and code that will never normally be used in production. All it does is say, "Oh, there's a bug, and it isn't in my code so it must be in yours." It's a habit from blame cultures, and collaborative teams with collective code ownership don't need it. Treat every piece of code you write as a liability. Less is better.
    – Lunivore
    Mar 29, 2012 at 15:13
30

I think you're also missing a few KPIs...

Negative points every time a Product Owner changes a requirement. They should have know what they wanted before development started.

Negative points every time a tester puts in duplicate points. Perhaps they get positive points ever time they find a bug?

Negative points to HR every time a "good" developer leaves to go work somewhere else. And positive points every time they can talk a developer into coming to work for you b/c that's gonna be a tough one ;-)

30

This is a bad idea on so many levels.

Lets list the ways:

  • Builds blame culture: everyone will be trying to blame everyone else for bugs that are found because it will affect their review
  • Encourages people not to take responsibility. If you do accept responsibility (it's my bug) then you get penalised
  • Hurts inter-team relations. Developers and testers don't usually get on, but can you imagine the animosity the developers will feel towards testers if every bug a tester finds effectively damages a developer in their review.
  • Slows velocity and costs more. Each spec is going to need to be written in huge detail or the developers won't touch it. There will be a massive increase in developer testing in order to ensure there are no bugs (and this probably won't work anyway).
  • Risk-averse culture. Nobody will want to try anything new because new means risky. Forget about new techniques, new technologies, new organisational mechanisms.
  • Flight to safely. Nobody will want to work on the "hard" parts of the system because they will be more likely to introduce bugs. Everyone will want to write the admin screens, nobody will actually want to code the core functionality.

The behaviors you want to encourage are:

  • Responsibility. People who take responsibility for problems and fix them should be rewarded.
  • Proactiveness. People who come up with ideas to make things better should be rewarded
  • Experimentation. We find out what works well by trying different things and learning new technologies
  • Velocity. At the end of the day, everyone has the same goal: to get the project done and delivered. You want to do this as fast as possible.

So:

  • I think you're doing the wrong thing
  • You shouldn't be having this discussion with us, you should be having it with your developers
  • Continuing down this path will do 3 things for you:

    1. Reduce the morale of your development teams

    2. Increase staff turnover as they look for work in more sensible development environments

    3. Reduce the speed of delivery for your clients.

You now need to choose how to proceed.

3
  • 2
    No, it's a good question on a bad idea. Better he ask and get honest answers warning him of the consequences should he try to implement his bad idea, than just go ahead with it. Up to "So:" your answer was actually quite useful, but regarding the rest: Your answer should be downvoted. Mar 30, 2012 at 10:45
  • Good challenge. My disapointment with the question is as follows: he seems to accept this as a good idea and has either not asked his developers if this works for them or has ignored their answer. A straw poll of the dev team I'm in produced an almost instant and unanimous response that this was a terrible idea. So yes, the final part of my response was harsh. If you feel strongly about downvoting me please do, it's good that people voice opinions. Mar 30, 2012 at 10:57
  • +1 - I don't think there was anything inappropriate about your answer, and this is all really good advice, to one PM, from another PM's perspective.
    – jmort253
    Apr 1, 2012 at 2:26
21

Performance reviews based on bugs is a pretty bad idea imho.

Even in best case scenarios where testers are assumed to be doing perfect work, it's very hard to implement some sort of bug tracker that doesn't produce side effects.

What you might consider is using some sort of metric for re-work hours. That tends to work better because:

  1. You aren't telling them to do the impossible, which is to erase bugs.
  2. It's easier to quantify
  3. That's the ACTUAL business impact the company is suffering.

Move the emphasis from bugs to better performing software, it just makes more sense.

1
  • 2
    Not only that, but testers may be focusing more on certain features, and thus reporting more bugs. Even if that code is higher quality there may be a higher bug count, since it's not a count of actual bugs but rather reported bugs. Mar 31, 2012 at 20:41
11

I am planning to give 40 points (my scoring system) to a developer, if he creates bug free code.

How about asking the programmers how much (money) they will fork out if somebody could show them how to create bug free code? Professional coding is no easy task and there are much better ways of judging good programmers.

Here is some reasons one ends up with buggy code:

1.Management decides to have some new features that would contradict some other features previously implemented. I mean who doesn't want new features?

2.Management would want these in the next release, but no time to refactor and clean up the existing code, I mean if the customer is not gonna see anything new, what is the management gonna present? We have no new features in this version but now everything works as it was suppose to? yeah try saying that in front of an audiance

What you really need is called code review, it is done by professional programmers going through the code in pairs and fixing the issues, but for this to work you need to have at least one good programmer. These a days anyone who can type is considered a programmer.

PS : The above comments are made from the view points of both a prgrammer and a manager,

8
  • 2
    So... 40 points for creating "bug free code"? That sounds like ... worthless to even try ! :) I totally agree with your answer, and, as a programmer (that "hates" being a manager), I think bug free code cannot be written without really knowing exactly what you are doing/coding and seeing/understanding the "big picture" of the project. Ideally, a new feature should seemingly "stick" to the existing code.
    – leoinfo
    Mar 29, 2012 at 13:56
  • 2
    I'd agree with everything @leoinfo says if everything after "without really knowing" is removed. Bug free code cannot be written. Period. Mar 29, 2012 at 18:54
  • 1
    @DavidStockton that's a bit of an extreme statement. Ever heard of formal verification? en.wikipedia.org/wiki/Formal_verification
    – zooone9243
    Mar 30, 2012 at 9:33
  • 2
    @zooone9243 In theory, theory and practice are the same. In practice, they're not.
    – Izkata
    Mar 30, 2012 at 19:24
  • 1
    @zooone9243 Formal proofs are disconnected from the code. Transcription between the two can easily introduce or hide errors. The only thing that could possibly be considered bug-free is trivial programs like Hello World - which I have seen bugs in occasionally, usually with regards to charsets.
    – Izkata
    Apr 2, 2012 at 16:33
9

Setting a target like number of bugs is something easy to trick, and it is easy to find somebody who the others can blame. This will kill your team cohesion. Please don't measure people by the bugs they introduced. If you do so, they won't commit anything until they are 100% that it is bug free, and you'll never have a release. While I wrote this answer I came up with 7 different ways to trick this KPI, so I'll look like a pretty good developer, while I'm doing nothing good for the organization.

I have a different proposal. Look for reoccurring problems. When you evaluate employees focus on their progress, and award those who are constantly improving, let those go who aren't improving.

Let's say we work together, and I do a lot of coding mistakes. We agree on that I learn more about the language we are using. You'll check my progress and when my coding style improves you award me. If not, then we have something to talk about.

1
  • 2
    +1 for "Look for reoccurring problems" - you want to develop a culture like TPS/Lean that looks for ways to remove reoccurring problems, rather than blame individuals. Mar 29, 2012 at 23:37
7

The objections all of the contributors are on target and these are true with all KPIs. You establish a metric to increased desired behavior, that desired behavior is "paid for" by the removal of other behaviors, some of which are also desired. This is why establishing your KPIs is very challenging and you need to do so with care.

A balanced approach is called for here, where if you are measuring the reduction of defects in quality, you need to also measure, and reward, those other behaviors that will likely be affected, too. For example, if I am being measured on quality, I am going to slow down and double check my work. So you have to counter that with a measurement on time. I'd also take less risks, which in many environments you do not want. So you have to counter that. You have to also be careful with cause and effect, the relationship between the independent variable (what you manipulate) and the dependent variable (the result). What seems intuitive, logical, and even "common sense" is too many times not true. You would think increasing someone's wage would have positive effects on morale and work effort. In fact, morale is not touched, or it actually drops, and work effort will eventually decrease as the labor supply curve suggests.

It might be that scoring defects is not the solution to increasing quality but rather scoring those behaviors that lead to quality, i.e., leading indicators. We know increased skill should mean higher performance, so perhaps there is a KPI or two that will motivate skill acquisition and mastery.

1
  • 3
    "You establish a metric to increased desired behavior, that desired behavior is "paid for" by the removal of other behaviors, some of which are also desired." This behavior is explored in depth in Robert Austin's book Measuring and Managing Performance in Organizations. (dorsethouse.com/books/mmpo.html) Mr. Austin also describes how almost all of the time some desired behaviors will not be measurable, one is left measuring a proxy, but the proxy will not be perfect and people will increase the proxy value while decreasing the desired behavior. Mar 29, 2012 at 16:17
6

To be honest, I believe you shouldn't. In my opinion there is no fair way to measure this. No programmer, no matter how experienced he is, will create bug free code. Sometimes bugs are obscured by other bugs. Sometimes bugs evolve from interaction of two separate pieces of code that on their own are free of bugs. Maybe really severe bugs will stay undiscovered for months or years.

I believe this is as faulty as trying to measure performance by number of lines of code produced per day.

1
  • +1 - Programming isn't like changing your spark plugs on your car, where there are very detailed specs and instructions on exactly what you need to do. In many programming scenarios, you may be doing things that no one else has done before, which means the software may be a little volatile until bugs are discovered and fixed.
    – jmort253
    Apr 1, 2012 at 2:29
6

Quality is a process and not a simple measure of "defect" rates. You improve quality by creating a better process, from start to finish, stretching across all levels of the organizations. Penalizing one role in the value chain will not help you accomplish your goal.

6

I once worked in that kind of environment and guess what? I left this place with rage and in very bad terms. And that is what you're going to suffer from if you indeed try to measure your developer's proficiency based on their bugs.

You're doing it all wrong. Here's how we fixed it back in the days:

Make a complex yet short printed list of the most important points.

  • Did you check that code for exploits and critical security holes?
  • Did you commented it well so if someones else go in there he or she could fixes it right away.
  • etc.

If someone does NOT check that list before releasing THEN you can blame him or her.

This is how things should go. Because if you start measuring bug, you'll gain very bad "karma" from your employee, everyone will hate you and act with you as if you were a dictator. And also you will give credit to those who does absolutely sweet-nothing!

Everyone makes mistakes and there's always THAT little thing that we don't remember before releasing, and this is why hotfixes exist!

1
  • Can you share such checklists if you have one?
    – Krunal
    Jun 9, 2014 at 8:32
5

This is a prime example of "measurement disfunction".

Any attempt to reward or penalise an employee based on a hard metric will fail. The employee will be smart enough to game the metric, and usually to the opposite effect that was intended.

It's been covered very well by Joel Spolsky in these blog posts:

Joel's articles includes many examples, including the topic of this question:

Suppose you decide to pay a bonus to the developer with the fewest bugs. Now every time a tester tries to report a bug, it becomes a big argument, and usually the developer convinces the tester that it’s not really a bug. Or the tester agrees to report the bug “informally” to the developer before writing it up in the bug tracking system. And now nobody uses the bug tracking system. The bug count goes way down, but the number of bugs stays the same.

Developers are clever this way. Whatever you try to measure, they’ll find a way to maximize, and you’ll never quite get what you want.


4

If you tie someone's compensation to any numerical scoring scheme of any kind, you push that person to work to the scoring system rather than working to actually achieve business results. "I get full points if I don't write any bugs? Cool! I won't write any code!"

The corollary to this rule is that you can't come up with a scoring scheme that can't be "gamed" in some non-useful fashion.

For a system that is sufficiently complex to do real work of value, you will never, ever have bug-free code. You'll have less-buggy and more-buggy code. But there's going to be a bug in there somewhere.

1
  • 1
    There is also a tremendous cost to 100% bug free code, which needs to be weighed against the benefit of releasing sooner (and patching afterward). For human rated software (such as a dialysis machine module), eliminating bugs is critical. For many other systems, not so much. Apr 15, 2012 at 13:29
3

I think you are just going to aggravate your colleagues and make some enemies.

Bugs should be picked up by tester. If that tester feels that certain employee is breaking things constantly, then it makes sense to speak to that employee and see how his performance can be improved. Rather than making everybody paranoid and insecure about their job.

3

As with everyone else so far, I'm pretty much against this. A couple reasons come to mind:

Outside Pressure

Most companies allow business to pressure/control software development. What I mean is very few places ask a developer how long it will take to do something, and then let him have that long. You give a timeline that the product owner doesn't like and you're likely to face pressure from them to get it done earlier.

While it's still the developer writing bugs, there are a lot of forces at play. He doesn't want to get dinged for not meeting business demands. But if they keep pressuring him to write a feature in 1 day that should take 1 week, you can bet there will be bugs.

Multiple Tiers

With multiple tiers or layers of software, a bug could be found in the upper layer but actually caused by something lower down. Additionally, "bugs" could be introduced because the layers misunderstood what the other was doing.

Decrease in productivity

Actual productivity will tank as you now not only have to QA the project, but also take time to figure out the true root cause and person to blame. While this might seems simple, I'd wager that it will blow up into something bigger. If my performance is based on me not getting any bugs and you're trying to assign a bug to me, you can bet I'm going to argue that it's not. This isn't going to be a simple email that says "This was John's fault not mine." Because now John is going to respond. In the end, you'll be sitting in hours long meetings at least once a week trying to sort through all the bugs.

Developers need to be professional

Approaches like this tend to diminish the professionalism of developers. Think of CEOs, they're often evaluated on the overall growth of the company, right? If they lost a $100k a year customer, that might be bad, but it's often downplayed if they still grew the company by $500k that year.

Plans that try to tie developers to metrics like these (number of bugs released, or number of lines of code written, features released, tests written etc) take developers from the realm of a professional employee and place them in the category of hourly worker.

Imagine you have an hourly worker mailing letters. His goal might be 100% accuracy, and you decide to dock him 1 point for each letter he screws up.

That's essentially what you're doing to developers. You're taking a group of people who are typically:

  1. Creative
  2. Dedicated
  3. Innovative
  4. Passionate about their work

And you're turning them into factory workers. They'll start "punching the clock." They'll start looking quickest easiest way to do something RIGHT NOW with no thought to how it will play out in the future. After all, why build an entire sum routine when I can just hard code the values for 1*1 to 12*12. That way I know there are no bugs.

Caveat

All that in mind, however, I am NOT saying that developers should not be held accountable. I absolutely think they should, and honestly, any developer worth keeping around isn't going to want to release bugs. If you have developers that don't mind releasing bugs, you should fire them right now (seriously, do it before lunch.)

TL;DR

I cannot think of any other professional environment or occupation where people try to apply these types of metrics to one another.

If a manager cannot tell who his top performers are and who is bottom performers are then that person should not be managing software developers, because he/she is clueless.

Before you apply a metric, you need to spend a LOT of time getting the right one, or else you risk running off your top performers since they don't want to be treated like little kids or assembly line workers (and in the end you might decide to not have metrics.)

3

I've seen this happen. A typical new PM that came from the business (in this case retail multinational) had thought every department is the same - if there is a problem then slap the wrong doer and keep everyone lese on their toes. Never works with IT. Most projects that require a PM at all mean multiple coders, and sys an, bus an, user support, architects, etc etc etc. All with their oars in the water. Scope creep, task movement, delays, milestone constraints, upstream/downstream system changes, staffing/sickness/leave/reassignment/contract changes, external system issues, hardware issues, domain/envirnment issues, set up issues, legacy code/systems, poor code coming in to the project as the base (i.e. initial code requiring ammendment) the list is almost endless - so many things cause things to change at the coal face. Good dev teams will compensate, do the extra hours, go the extra mile - and throw their contingency out the window. This means bugs. This is not a problem in itself, it is part of the process - its not like a chair manufacturer making a crap chair that goes in the bin, bugs can be fixed. If you want to run KPI's do it on the number of bugs fixed as a team, not raised. Otherwise, bugs will be hidden or "merged" - blame will be fired in all directions (including upwards!). It will also make for a restrictive environment. As a Dev lead, if you came to me mid project and pushed for a change I'd tell you to wait until the next phase - there would be no scope creep at all unless you are willing to write off all bugs and take the KPI hit instead.

This appraoch will back fire on you big time - think again!

6
  • I disagree with "do it on the number of bugs fixed as a team, not raised". Put me on that team, and I will spend an hour each morning adding visible bugs that are easy to fix. Woo hoo - here comes my bonus! Mar 30, 2012 at 4:10
  • I think this is a fallacy. It would be a pretty poor team and a very low team member that did this. For the most part people want to be professional. It is easy to say what you did, but I doubt you actually would do that. People generally liked to be though of as good at their job, and dislike to bring their team down. If team members did as you said, then there are much more issues with the team as a whole than individual performance. It would also not be hard to spot such an individual and much easier to remove them from the team. PM's do not just sit in an office looking at metrics.
    – Wolf5370
    Mar 30, 2012 at 5:22
  • 1
    The comment was tounge-in-cheek but it points to a flaw in the metric. A team that creates bugs and fixes them scores better than a team that gets it right the first time. Imagine a developer has spent five days working on a new feature. He's already surpassed his estimated time and just wants the feature finished. He can see a non-showstopping bug in this functionality. Does he spend the extra hour to fix the bug now? Or, does he mark the work as done, knowing it will be fixed some time in the future? If he knows that the latter option would help some score go up, then he'll leave the bug. Mar 30, 2012 at 6:10
  • I don't disagree that such metrics as a performance review indicator is silly, this is why I specifically stated "as a team". I meant it in a pat-on-the-back sort of way rather than bonus linked etc. As my final comment suggested, KPI on peformance relating to bugs is never useful. I did get that you spoke toungue in cheek, but I guess we'd be surpised how many PMs out there would believe such a scenario not only possible but likely. I was a developer a long time, I would never have left a showstopper to go through the system knowingly - I would much rather go cap in hand to the PM.
    – Wolf5370
    Mar 30, 2012 at 6:36
  • @Wolf5370 - You mentioned a very great point. Buggy software isn't the same as a bug in hardware, like the chair in your example. Bugs in software can just be fixed. A bug is basically an unfinished feature. It's not like the entire project has to be scrapped and started over because of a bug, not like if it were a buggy circuit board design or a car alternator...
    – jmort253
    Apr 1, 2012 at 2:35
2

A good developer will be the kind of person who cheerfully tells the testers and QA team, "Please - I want you to find bugs with my code". He'll actively encourage testers to try to break his code, so that he can improve it. His motivation is to find all those bugs before it's released to Production.

If you motivate the developer to minimise the number of bugs reported, do you think he'll be so keen?

2

I'm thinking the only thing that might be worse is paying developers based on the number of bugs they fix. I can almost guarantee that if you pay based on the inverse of the number of bugs created, it will be a LOT longer before any code sees the light of day.

You'll be better off creating a culture and processes that fosters fewer bugs. Break the build, you get a bit of good-natured, public shaming -- and get to fix it. Use TDD to drive design and code development so that you know you have tests that cover the features you're working on. Use code reviews/pair programming to get more than one set of eyes on the code. Things like this foster good practices but don't weigh people down and cause paralysis like "you'll get docked money for each error you have."

Reward the team for doing good work, for accomplishing things - getting projects out the door quickly with happy customers. Reward individuals for going above and beyond in improving their skills and contributing to the team.

2

This is the only Q that matters: Is it good enough?

In any industry, perfection is expensive. Look for code that can be easily maintained. It's okay if there's bugs, as long the coder's style made it easy to fix.

2

Just to add to the list of what will happen:

  • Programmers will hide the fact that they have a bug in their own code.

It often happens than on a second look, perhaps during next feature enhancement, same programmer will find their own bug. I suppose you can protect against it, by not charging them points.

  • What happens if they don't want to tell-tale on their friends?

In my opinion, you do not want to stop programmers from exposing bugs. And I think that is all you're going to achieve with this type of approach.

This is coming from a programmer, with 12 years experience, but only worked in 1 company. Competent managers shouldn't have an issue with identifying good and bad performers, and this system seems like an attempt to automate management.

1

Empower your developers, instead of measuring them with random yardsticks. Look into Agile. Bugs are not a metric of good work, bugs happen. Happy customers are a metric of good work, but you can't really measure happiness.

The basic idea of Agile is that the developers have their objectives, they've divided the work among themselves, and they work towards workable features in small increments. They're responsible, directly, for what they do. And the best part is you don't have to keep tabs on the points!

There are many books and websites on Agile, and your developers will like you for taking on that approach, if it's well implemented.

1

Wait, the original poster didn't say which way this would go. Perhaps more bugs is better. Obviously the more bugs a developer produces, the more productive he/she is! If they have the average bug production rate, then the more bugs, the more code they wrote.

Of course, incentivizing more bugs is not a good idea either, since it would be rather easy to deliberately include bugs.

I know a fun one. How about you incentivize some of your developers to create devious bugs that can make it through as many levels of code checking, inspection, and testing as possible? Then you can expose flaws in that process ...

1

For starters, as others have suggested, be careful if you are considering tying compensation to "performance." Before you go down that path, watch Dan Pink's TED talk on the surprising science of motivation: http://www.ted.com/talks/dan_pink_on_motivation.html

Not that metrics are useless. However, these sorts of metrics are more useful for teams than individuals. The team here is the entire team involved in delivering a product: this includes developers and testers (and product managers, project managers...). Development and testing should not be adversarial. Measure a lot of things, and use them to get a rough idea of your team's progress. Don't kid yourself that all or even any of your metrics will be truly objective. Bugs are each subjective, so a number of bugs is even more subjective. Your metrics will be more accurate if there are incentives to report them truthfully. Such incentives may be intangible and ideally will be intertwined with a company culture of openness, cooperation, learning, and improvement. Most metrics will also represent tradeoffs with cost or ultimately profit. Reducing customer incidents to zero is likely cost-prohibitive, as is 100% test automation coverage. Usually expert opinion is more valuable than a metric. Goodness, ask your team how they're doing -- if the company culture is good they will report their performance truthfully. If you have an agile process, hopefully you do this already in your retrospective meetings. If you use metrics in combination with expert opinions and in a context of open communication and collaboration, I believe they can be valuable.

A product/service development team's metrics might include:

  • Velocity (average features/story points per unit time)
  • Monthly cost
  • Frequency of release or downtime
  • Cost per release or duration/cost per downtime
  • Customer incidents
  • Average cost/severity of customer incidents
  • Automated test code coverage
  • Manual test code coverage
  • Manual test feature/story point coverage (more subjective)
  • Average time from QA bug report to resolution
  • Average time from customer incident report to resolution
  • Software performance benchmarks
  • Users
  • Sales
  • ...
3
  • But you can game all of those metrics as well.
    – CaffGeek
    Mar 29, 2012 at 21:41
  • Absolutely. But if compensation is not tied to performance (see 1st point), and the team is doing this for its own benefit (see main paragraph), why would it cheat itself? This might seem laughable, but I've seen this basic approach work great in 2 companies (1 in California, 1 in Texas). Metrics won't help a hostile culture though. Culture first, metrics second.
    – Will
    Mar 29, 2012 at 22:34
  • yes, if they aren't tied to compensation, then yes, they are useful. As long as the metrics are kept within the team. As soon as management starts seeing them, they will start comparing with other teams, and start using them to evaluate
    – CaffGeek
    Mar 30, 2012 at 13:53
0

This is about the worst way to establish any kind of camaraderie. What if someone could judge you on, say, the number of poorly written requirements or missed milestones?

1
  • How about developer morale and productivity? Mar 29, 2012 at 16:26
0

It's impossible to measure performance just only from bug rates from developers work.

Why? Bug free apps doesn't mean the developer had built easy to maintain code, and it does not guarantee there is no hard-coded script in their work.

From my point of view and what I've done, I measure developers' performance by using status when he/she submitted their works to testers, how many the code being returned to developers and how long they submit their works after the related tasks is assigned to them.

Incentive? They will get it if they could exceed or meet the target assigned by me. I worked 8 years as a software developer, and every task I gave to my developers, measured first by difficulties and the complexity of business process.

2
  • Still, this doesn't seem like the best approach. Some tasks that you think may take 4 hours may take me 6 hours and vice versa, there may be a task that I think takes 3 hours that may take you 5 hours. Software development isn't an exact science, and I think it's a bit arrogant for us to think of our estimates as a 100% benchmark for other software developers. All developers approach problems differently, and sometimes that means estimates may differ.
    – jmort253
    Apr 1, 2012 at 2:54
  • As a project manager, you have the right to define your task which will be assigned to software developer. You need an experienced developer to weighting the tasks. Measure the task by developer (who get the task) skill. It's a subjective target, but it could be the best approach to create an estimation on overall project timeline Apr 2, 2012 at 6:23
0

Late to the party but this is what I'd do in your place:

  1. base a metric on desired outcome. I'd prefer more frequent releases. The shorter the time interval between releases, the higher the KPI. The side-effects are smaller releases (meaning fewer bugs), continuous delivery, automated testing and an environment where bugs can be fixed in minutes.

  2. Make it a team metric. If you do reward persons, ask each of the team members to distribute 10 points over their collegues, ranging by who attributes the most to the performance of the team. Distribute rewards accordingly.

  3. Ask the team what they would consider a good KPI, considering that you care about the performance of the whole (the value you deliver to your customers), not internal personal performance. Ask them how to reward total optimization instead of sub-optimization.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.