I am trying to implement KPI for employees of my company. One of the KPI have listed is the quality of code, as measured by bugs created by developers.
Now my problem is: I am planning to give 40 points (my scoring system) to a developer, if he/she creates bug free code.
There can be bugs which are severe, medium and low in nature. How do I rate my developers and give points based on number of bugs that they create?
Please be careful when using these kind of measurements as KPIs.
If you do this, I predict the following:
This article by Esther Derby suggests alternatives to using KPIs in pay reviews. There's a growing body of evidence that suggests that using performance as a way of measuring pay is destructive. My experience is certainly that it encourages heroism and a blame culture rather than team work, particularly when the KPIs are measuring unwanted items.
I think you're also missing a few KPIs...
Negative points every time a Product Owner changes a requirement. They should have know what they wanted before development started.
Negative points every time a tester puts in duplicate points. Perhaps they get positive points ever time they find a bug?
Negative points to HR every time a "good" developer leaves to go work somewhere else. And positive points every time they can talk a developer into coming to work for you b/c that's gonna be a tough one ;-)
This is a bad idea on so many levels.
Lets list the ways:
The behaviors you want to encourage are:
So:
Continuing down this path will do 3 things for you:
Reduce the morale of your development teams
Increase staff turnover as they look for work in more sensible development environments
Reduce the speed of delivery for your clients.
You now need to choose how to proceed.
Performance reviews based on bugs is a pretty bad idea imho.
Even in best case scenarios where testers are assumed to be doing perfect work, it's very hard to implement some sort of bug tracker that doesn't produce side effects.
What you might consider is using some sort of metric for re-work hours. That tends to work better because:
Move the emphasis from bugs to better performing software, it just makes more sense.
I am planning to give 40 points (my scoring system) to a developer, if he creates bug free code.
How about asking the programmers how much (money) they will fork out if somebody could show them how to create bug free code? Professional coding is no easy task and there are much better ways of judging good programmers.
Here is some reasons one ends up with buggy code:
1.Management decides to have some new features that would contradict some other features previously implemented. I mean who doesn't want new features?
2.Management would want these in the next release, but no time to refactor and clean up the existing code, I mean if the customer is not gonna see anything new, what is the management gonna present? We have no new features in this version but now everything works as it was suppose to? yeah try saying that in front of an audiance
What you really need is called code review, it is done by professional programmers going through the code in pairs and fixing the issues, but for this to work you need to have at least one good programmer. These a days anyone who can type is considered a programmer.
PS : The above comments are made from the view points of both a prgrammer and a manager,
Setting a target like number of bugs is something easy to trick, and it is easy to find somebody who the others can blame. This will kill your team cohesion. Please don't measure people by the bugs they introduced. If you do so, they won't commit anything until they are 100% that it is bug free, and you'll never have a release. While I wrote this answer I came up with 7 different ways to trick this KPI, so I'll look like a pretty good developer, while I'm doing nothing good for the organization.
I have a different proposal. Look for reoccurring problems. When you evaluate employees focus on their progress, and award those who are constantly improving, let those go who aren't improving.
Let's say we work together, and I do a lot of coding mistakes. We agree on that I learn more about the language we are using. You'll check my progress and when my coding style improves you award me. If not, then we have something to talk about.
The objections all of the contributors are on target and these are true with all KPIs. You establish a metric to increased desired behavior, that desired behavior is "paid for" by the removal of other behaviors, some of which are also desired. This is why establishing your KPIs is very challenging and you need to do so with care.
A balanced approach is called for here, where if you are measuring the reduction of defects in quality, you need to also measure, and reward, those other behaviors that will likely be affected, too. For example, if I am being measured on quality, I am going to slow down and double check my work. So you have to counter that with a measurement on time. I'd also take less risks, which in many environments you do not want. So you have to counter that. You have to also be careful with cause and effect, the relationship between the independent variable (what you manipulate) and the dependent variable (the result). What seems intuitive, logical, and even "common sense" is too many times not true. You would think increasing someone's wage would have positive effects on morale and work effort. In fact, morale is not touched, or it actually drops, and work effort will eventually decrease as the labor supply curve suggests.
It might be that scoring defects is not the solution to increasing quality but rather scoring those behaviors that lead to quality, i.e., leading indicators. We know increased skill should mean higher performance, so perhaps there is a KPI or two that will motivate skill acquisition and mastery.
To be honest, I believe you shouldn't. In my opinion there is no fair way to measure this. No programmer, no matter how experienced he is, will create bug free code. Sometimes bugs are obscured by other bugs. Sometimes bugs evolve from interaction of two separate pieces of code that on their own are free of bugs. Maybe really severe bugs will stay undiscovered for months or years.
I believe this is as faulty as trying to measure performance by number of lines of code produced per day.
I once worked in that kind of environment and guess what? I left this place with rage and in very bad terms. And that is what you're going to suffer from if you indeed try to measure your developer's proficiency based on their bugs.
You're doing it all wrong. Here's how we fixed it back in the days:
Make a complex yet short printed list of the most important points.
If someone does NOT check that list before releasing THEN you can blame him or her.
This is how things should go. Because if you start measuring bug, you'll gain very bad "karma" from your employee, everyone will hate you and act with you as if you were a dictator. And also you will give credit to those who does absolutely sweet-nothing!
Everyone makes mistakes and there's always THAT little thing that we don't remember before releasing, and this is why hotfixes exist!
Quality is a process and not a simple measure of "defect" rates. You improve quality by creating a better process, from start to finish, stretching across all levels of the organizations. Penalizing one role in the value chain will not help you accomplish your goal.
This is a prime example of "measurement disfunction".
Any attempt to reward or penalise an employee based on a hard metric will fail. The employee will be smart enough to game the metric, and usually to the opposite effect that was intended.
It's been covered very well by Joel Spolsky in these blog posts:
Joel's articles includes many examples, including the topic of this question:
Suppose you decide to pay a bonus to the developer with the fewest bugs. Now every time a tester tries to report a bug, it becomes a big argument, and usually the developer convinces the tester that it’s not really a bug. Or the tester agrees to report the bug “informally” to the developer before writing it up in the bug tracking system. And now nobody uses the bug tracking system. The bug count goes way down, but the number of bugs stays the same.
Developers are clever this way. Whatever you try to measure, they’ll find a way to maximize, and you’ll never quite get what you want.
If you tie someone's compensation to any numerical scoring scheme of any kind, you push that person to work to the scoring system rather than working to actually achieve business results. "I get full points if I don't write any bugs? Cool! I won't write any code!"
The corollary to this rule is that you can't come up with a scoring scheme that can't be "gamed" in some non-useful fashion.
For a system that is sufficiently complex to do real work of value, you will never, ever have bug-free code. You'll have less-buggy and more-buggy code. But there's going to be a bug in there somewhere.
I think you are just going to aggravate your colleagues and make some enemies.
Bugs should be picked up by tester. If that tester feels that certain employee is breaking things constantly, then it makes sense to speak to that employee and see how his performance can be improved. Rather than making everybody paranoid and insecure about their job.
As with everyone else so far, I'm pretty much against this. A couple reasons come to mind:
Outside Pressure
Most companies allow business to pressure/control software development. What I mean is very few places ask a developer how long it will take to do something, and then let him have that long. You give a timeline that the product owner doesn't like and you're likely to face pressure from them to get it done earlier.
While it's still the developer writing bugs, there are a lot of forces at play. He doesn't want to get dinged for not meeting business demands. But if they keep pressuring him to write a feature in 1 day that should take 1 week, you can bet there will be bugs.
Multiple Tiers
With multiple tiers or layers of software, a bug could be found in the upper layer but actually caused by something lower down. Additionally, "bugs" could be introduced because the layers misunderstood what the other was doing.
Decrease in productivity
Actual productivity will tank as you now not only have to QA the project, but also take time to figure out the true root cause and person to blame. While this might seems simple, I'd wager that it will blow up into something bigger. If my performance is based on me not getting any bugs and you're trying to assign a bug to me, you can bet I'm going to argue that it's not. This isn't going to be a simple email that says "This was John's fault not mine." Because now John is going to respond. In the end, you'll be sitting in hours long meetings at least once a week trying to sort through all the bugs.
Developers need to be professional
Approaches like this tend to diminish the professionalism of developers. Think of CEOs, they're often evaluated on the overall growth of the company, right? If they lost a $100k a year customer, that might be bad, but it's often downplayed if they still grew the company by $500k that year.
Plans that try to tie developers to metrics like these (number of bugs released, or number of lines of code written, features released, tests written etc) take developers from the realm of a professional employee and place them in the category of hourly worker.
Imagine you have an hourly worker mailing letters. His goal might be 100% accuracy, and you decide to dock him 1 point for each letter he screws up.
That's essentially what you're doing to developers. You're taking a group of people who are typically:
And you're turning them into factory workers. They'll start "punching the clock." They'll start looking quickest easiest way to do something RIGHT NOW with no thought to how it will play out in the future. After all, why build an entire sum routine when I can just hard code the values for 1*1 to 12*12. That way I know there are no bugs.
Caveat
All that in mind, however, I am NOT saying that developers should not be held accountable. I absolutely think they should, and honestly, any developer worth keeping around isn't going to want to release bugs. If you have developers that don't mind releasing bugs, you should fire them right now (seriously, do it before lunch.)
TL;DR
I cannot think of any other professional environment or occupation where people try to apply these types of metrics to one another.
If a manager cannot tell who his top performers are and who is bottom performers are then that person should not be managing software developers, because he/she is clueless.
Before you apply a metric, you need to spend a LOT of time getting the right one, or else you risk running off your top performers since they don't want to be treated like little kids or assembly line workers (and in the end you might decide to not have metrics.)
I've seen this happen. A typical new PM that came from the business (in this case retail multinational) had thought every department is the same - if there is a problem then slap the wrong doer and keep everyone lese on their toes. Never works with IT. Most projects that require a PM at all mean multiple coders, and sys an, bus an, user support, architects, etc etc etc. All with their oars in the water. Scope creep, task movement, delays, milestone constraints, upstream/downstream system changes, staffing/sickness/leave/reassignment/contract changes, external system issues, hardware issues, domain/envirnment issues, set up issues, legacy code/systems, poor code coming in to the project as the base (i.e. initial code requiring ammendment) the list is almost endless - so many things cause things to change at the coal face. Good dev teams will compensate, do the extra hours, go the extra mile - and throw their contingency out the window. This means bugs. This is not a problem in itself, it is part of the process - its not like a chair manufacturer making a crap chair that goes in the bin, bugs can be fixed. If you want to run KPI's do it on the number of bugs fixed as a team, not raised. Otherwise, bugs will be hidden or "merged" - blame will be fired in all directions (including upwards!). It will also make for a restrictive environment. As a Dev lead, if you came to me mid project and pushed for a change I'd tell you to wait until the next phase - there would be no scope creep at all unless you are willing to write off all bugs and take the KPI hit instead.
This appraoch will back fire on you big time - think again!
A good developer will be the kind of person who cheerfully tells the testers and QA team, "Please - I want you to find bugs with my code". He'll actively encourage testers to try to break his code, so that he can improve it. His motivation is to find all those bugs before it's released to Production.
If you motivate the developer to minimise the number of bugs reported, do you think he'll be so keen?
I'm thinking the only thing that might be worse is paying developers based on the number of bugs they fix. I can almost guarantee that if you pay based on the inverse of the number of bugs created, it will be a LOT longer before any code sees the light of day.
You'll be better off creating a culture and processes that fosters fewer bugs. Break the build, you get a bit of good-natured, public shaming -- and get to fix it. Use TDD to drive design and code development so that you know you have tests that cover the features you're working on. Use code reviews/pair programming to get more than one set of eyes on the code. Things like this foster good practices but don't weigh people down and cause paralysis like "you'll get docked money for each error you have."
Reward the team for doing good work, for accomplishing things - getting projects out the door quickly with happy customers. Reward individuals for going above and beyond in improving their skills and contributing to the team.
This is the only Q that matters: Is it good enough?
In any industry, perfection is expensive. Look for code that can be easily maintained. It's okay if there's bugs, as long the coder's style made it easy to fix.
Just to add to the list of what will happen:
It often happens than on a second look, perhaps during next feature enhancement, same programmer will find their own bug. I suppose you can protect against it, by not charging them points.
In my opinion, you do not want to stop programmers from exposing bugs. And I think that is all you're going to achieve with this type of approach.
This is coming from a programmer, with 12 years experience, but only worked in 1 company. Competent managers shouldn't have an issue with identifying good and bad performers, and this system seems like an attempt to automate management.
Empower your developers, instead of measuring them with random yardsticks. Look into Agile. Bugs are not a metric of good work, bugs happen. Happy customers are a metric of good work, but you can't really measure happiness.
The basic idea of Agile is that the developers have their objectives, they've divided the work among themselves, and they work towards workable features in small increments. They're responsible, directly, for what they do. And the best part is you don't have to keep tabs on the points!
There are many books and websites on Agile, and your developers will like you for taking on that approach, if it's well implemented.
Wait, the original poster didn't say which way this would go. Perhaps more bugs is better. Obviously the more bugs a developer produces, the more productive he/she is! If they have the average bug production rate, then the more bugs, the more code they wrote.
Of course, incentivizing more bugs is not a good idea either, since it would be rather easy to deliberately include bugs.
I know a fun one. How about you incentivize some of your developers to create devious bugs that can make it through as many levels of code checking, inspection, and testing as possible? Then you can expose flaws in that process ...
For starters, as others have suggested, be careful if you are considering tying compensation to "performance." Before you go down that path, watch Dan Pink's TED talk on the surprising science of motivation: http://www.ted.com/talks/dan_pink_on_motivation.html
Not that metrics are useless. However, these sorts of metrics are more useful for teams than individuals. The team here is the entire team involved in delivering a product: this includes developers and testers (and product managers, project managers...). Development and testing should not be adversarial. Measure a lot of things, and use them to get a rough idea of your team's progress. Don't kid yourself that all or even any of your metrics will be truly objective. Bugs are each subjective, so a number of bugs is even more subjective. Your metrics will be more accurate if there are incentives to report them truthfully. Such incentives may be intangible and ideally will be intertwined with a company culture of openness, cooperation, learning, and improvement. Most metrics will also represent tradeoffs with cost or ultimately profit. Reducing customer incidents to zero is likely cost-prohibitive, as is 100% test automation coverage. Usually expert opinion is more valuable than a metric. Goodness, ask your team how they're doing -- if the company culture is good they will report their performance truthfully. If you have an agile process, hopefully you do this already in your retrospective meetings. If you use metrics in combination with expert opinions and in a context of open communication and collaboration, I believe they can be valuable.
A product/service development team's metrics might include:
This is about the worst way to establish any kind of camaraderie. What if someone could judge you on, say, the number of poorly written requirements or missed milestones?
It's impossible to measure performance just only from bug rates from developers work.
Why? Bug free apps doesn't mean the developer had built easy to maintain code, and it does not guarantee there is no hard-coded script in their work.
From my point of view and what I've done, I measure developers' performance by using status when he/she submitted their works to testers, how many the code being returned to developers and how long they submit their works after the related tasks is assigned to them.
Incentive? They will get it if they could exceed or meet the target assigned by me. I worked 8 years as a software developer, and every task I gave to my developers, measured first by difficulties and the complexity of business process.
Late to the party but this is what I'd do in your place:
base a metric on desired outcome. I'd prefer more frequent releases. The shorter the time interval between releases, the higher the KPI. The side-effects are smaller releases (meaning fewer bugs), continuous delivery, automated testing and an environment where bugs can be fixed in minutes.
Make it a team metric. If you do reward persons, ask each of the team members to distribute 10 points over their collegues, ranging by who attributes the most to the performance of the team. Distribute rewards accordingly.
Ask the team what they would consider a good KPI, considering that you care about the performance of the whole (the value you deliver to your customers), not internal personal performance. Ask them how to reward total optimization instead of sub-optimization.
