Allowing PMs to grade/rate developers

Question

We're trying to introduce some new/updated ways to quantify developer productivity, and we want to include our Project Managers' thoughts. What do you think about allowing PMs to "grade" or apply some sort of "rating" to a developer's task? The grades would be based on things like the number of bugs that were found, code performance (does it run slow vs. fast), adherence to requirements, etc.

Would developers get upset about PMs grading them on a seemingly arbitrary scale? How can I do this in a way that won't rock the boat?

Answer 1

First, every appraisal, no matter who does it, is arbitrary to some point. If someone isn't fine with that they should find a place where there's no appraisals at all.

Second, for someone who is looking for feedback, more feedback should be always better than less feedback. I mean, even when I don't fully agree with someone's opinion it can still be very valuable because it shows how others see my work and, arguably, what I can do to change their views.

From these perspectives finding a way of incorporating feedback from pretty much anyone somehow connected with developers' work to their appraisal is good. And PM is definitely a person whose work is "somehow connected" with development.

Talking about that, PM has a pretty different perspective of a project, thus their feedback will be even more valuable. If we keep feedback just within a functional group, here: developers, and developers' functional manager we get pretty homogeneous view of a project. Bringing other point of view, especially from someone working closer with a client, would definitely help to make it better.

Note: In my answer I'm focusing on feedback in general and not on a specific grade, rate, mark or however you call it.

This is because in terms of appraising people any rating system is tricky by default.

My advice would be to incorporate feedback delivered by PMs but avoid any direct rating of developers. If functional manager isn't all defensive when it comes to their people they should be able to deliver such feedback to people in a valuable way. Such approach however requires trust between pretty much everyone around: PM, a manager and a team member. PM trusts that their feedback will reach a developer. A manager trusts they get honest, straightforward, possibly fact-based, feedback. A developer trusts no one tries to run them down, etc.

Depending on your organization's standards it may be a standard way that people are rated on a scale and if it is so I don't say you should avoid that at all cost. Do remember that PM's feedback is just a part of the whole judgment on one's work.

If people (here: PMs) never delivered feedback to other team members, rating system may be a good way to standardize the way they construct their feedback.

On the other hand please avoid any automatic measures, like a number of bugs, lines of code etc. It has nothing to do with feedback and it will only drive measured numbers down or up (depending on a goal) with no direct influence on quality, e.g. if you measure a number of submitted bugs, you may be pretty sure that people will stop submitting bugs; it doesn't say much about a project quality and even fogs your visibility more.

Answer 2

Bugs:

Programming generally involves taking many vague or abstract concepts and then tying them together to build something great. Judging developers based on the number of bugs in the code perhaps one of the worst ways an organization can shoot itself in the foot.

In a world where everything is so pro-Agile, there is no blueprint to follow. We're not building houses -- the purest form of waterfall model possible -- where we know every stud must be 16 inches apart because that's how we built the last 15 houses. Every software project is fundamentally different in some significant manner from other software projects.

Thus, bugs are just a fact of life. All software will have bugs, because many times developers are tying things together that have never been tied together before. Don't base performance on bugs, unless your goal is to demoralize and beat down your team.

Speed of Coding:

Some developers can code very quickly, but then 6 to 12 months down the road; suddenly, the progress comes to a grinding halt due to all of the shortcuts and bad decisions made in the code. Programmers aren't typists. You don't measure success by how quickly they can spew code into their editor.

Other developers are methodical, patient, and detail-oriented. They take the time to think about what approach will ensure the success of the product. Their decisions ensure that the product can still be supported years from now. They make sure that when one part of the system changes there isn't a domino effect created that reverberates through the entire codebase, knocking everything else out. Their code is well-commented, readable, engineered, and easy to maintain.

I understand why we as project managers want to measure this, but we must understand that this is a very delicate, possibly immeasurable balance, and it involves good judgement and decision-making by the technical people on the project.

Developers are generally paid on salary because their jobs involve the ability to make decisions and judgement calls, not produce little round widgets on a minimum wage production line.

Adherence to Requirements:

Software development is a creative process. It's also an engineering process. The engineering process comes into play when the developer has to make sure he or she understands what it is that should be built. Using logic, good documentation, sometimes formal methods and planning, a developer should be able to have a clear picture of what he/she must build.

The creative part comes into play with the actual solving of the problems that will ultimately arise during the development process. This sometimes occurs in other fields as well, like in construction. Perhaps there is some reason why the last two studs can't be 16 inches apart, so some creativity must be put into finding a safe and sustainable solution. With software, sometimes developers run into walls, and he/she must find a way around that wall by taking the things he or she knows about several different programming concepts and then bring them all together to come up with a great solution.

Now, the creativity part of software development doesn't mean the developer should go off an build whatever he or she wants to build; instead, the developer must still adhere to the requirements. The creative part is in the approach to solving the problem, not the actual problem itself.

With that said, sometimes engineers come up with creative ideas that can enhance the product. However, whether or not those items are implemented depends solely on who the product is intended for. At a startup, the engineer is likely to have a lot of freedom to innovate and create something that will instill passion in people. On the other hand, in a big corporation building custom software for a client, the goal is to deliver what the client wants, and that may leave little room in the requirements for unsolicited change.

Summary

In summary, the first two points are items that just don't make sense to measure. Bugs are a fact of life. Just look at them as unfinished features or part of the enhancing process. Speed of coding may not be a function of project success because coding is an engineering process, not a manufacturing process. Finally, requirement adherence may change from project to project and may not be measurable or important in some projects.

Instead, focus on evaluating the people. Is he easy to get along with? Does she help motivate other team members? Does she help collaborate with the team and generate problem-solving ideas? Do you trust this person to deliver what he or she says he's going to deliver. When bugs are found, does he take responsibility and passionately dig in and fix the unfinished feature (bug).

By it's very nature, software development is a subjective process, and you cannot use objective evaluation criteria to evaluate positions such as this without undermining yourself, your team, and your organization.

Answer 3

I do not think being a developer is relevant in answering this question. We live in a world where we are measured for our performance, no matter the activity or work we are doing. Being measured and compared with our peers are consistent with setting goals and achieving them, healthy competition, and weeding out those who do not belong. How one feels about it is immaterial.

The measuring stick you use should be a decomposed version of the stick used to measure the organization's performance.

There will always be unintended side effects to a metric. In many cases, those side effects are of no consequence to the organization. In other cases, the metric you choose will cause a behavior shift you did not want. So you approach it smartly. Don't measure something simply to say you are measuring. Measure it because you have goal you need to meet and that you have an eye on unwanted behaviors it may also produce.

Answer 4

Project managers should focus on the management of time an resources to meet the deadline. I would think no matter what the discipline, a PM should be able to rate how team members participate in project initiation (did they participate), how well they plan (did their lack of planning or lack of skill in planning adversely affect schedule) did they execute (did their estimates match their actual output or burn down rate?) and how well they monitor and control (correct work within their area of influence) and did they close out the project properly (commit lessons learns, submit final contractual deliverables, etc.).

However the premise and specifics of your question to the area of software development (number of bugs, etc.) is, I think, wrong minded. PMs should focus on the areas of discipline they know. While I think as a team there could be metrics defined that match the PM knowledge areas (e.g., classic PMI knowledge areas mapped to whatever the programming team thinks makes a matching metric for that area) I would avoid influencing and managing a programmer or programming team unless you were teaming and part of the team affected by the measurement(s).

Answer 5

You say you are looking to "apply some sort of rating" based on:

1. number of bugs
2. speed of code
3. adherence to requirements

only the last one of those has any sort of subjective part to the measurement and even then depending on the rigor and granularity of your requirements tracking that may be up for debate.

So my question is why make this something the PM's rate? why not just make it automated? look up the number of bugs, the performance metrics on the nightly tests etc. for each developer's code and keep a running average or something.

Of course if you make it an automated metric you risk having the developers game the system. If you make it a purely subjective measure of the project manager's opinions of the developers code then it's a rather worthless measurement unless your PM's are all developers as well.

I'm a developer and I would not take kindly to a non-technical person rating my code on some arbitrary scale. It's hard for me to offer a better solution without knowing what you are trying to accomplish or if there is an underlying problem you're attempting to address.

Answer 6

Project managers should absolutely have input into the performance reviews of the people on their teams.