9

We're trying to introduce some new/updated ways to quantify developer productivity, and we want to include our Project Managers' thoughts. What do you think about allowing PMs to "grade" or apply some sort of "rating" to a developer's task? The grades would be based on things like the number of bugs that were found, code performance (does it run slow vs. fast), adherence to requirements, etc.

Would developers get upset about PMs grading them on a seemingly arbitrary scale? How can I do this in a way that won't rock the boat?

13

First, every appraisal, no matter who does it, is arbitrary to some point. If someone isn't fine with that they should find a place where there's no appraisals at all.

Second, for someone who is looking for feedback, more feedback should be always better than less feedback. I mean, even when I don't fully agree with someone's opinion it can still be very valuable because it shows how others see my work and, arguably, what I can do to change their views.

From these perspectives finding a way of incorporating feedback from pretty much anyone somehow connected with developers' work to their appraisal is good. And PM is definitely a person whose work is "somehow connected" with development.

Talking about that, PM has a pretty different perspective of a project, thus their feedback will be even more valuable. If we keep feedback just within a functional group, here: developers, and developers' functional manager we get pretty homogeneous view of a project. Bringing other point of view, especially from someone working closer with a client, would definitely help to make it better.

Note: In my answer I'm focusing on feedback in general and not on a specific grade, rate, mark or however you call it.

This is because in terms of appraising people any rating system is tricky by default.

My advice would be to incorporate feedback delivered by PMs but avoid any direct rating of developers. If functional manager isn't all defensive when it comes to their people they should be able to deliver such feedback to people in a valuable way. Such approach however requires trust between pretty much everyone around: PM, a manager and a team member. PM trusts that their feedback will reach a developer. A manager trusts they get honest, straightforward, possibly fact-based, feedback. A developer trusts no one tries to run them down, etc.

Depending on your organization's standards it may be a standard way that people are rated on a scale and if it is so I don't say you should avoid that at all cost. Do remember that PM's feedback is just a part of the whole judgment on one's work.

If people (here: PMs) never delivered feedback to other team members, rating system may be a good way to standardize the way they construct their feedback.

On the other hand please avoid any automatic measures, like a number of bugs, lines of code etc. It has nothing to do with feedback and it will only drive measured numbers down or up (depending on a goal) with no direct influence on quality, e.g. if you measure a number of submitted bugs, you may be pretty sure that people will stop submitting bugs; it doesn't say much about a project quality and even fogs your visibility more.

  • 1
    +1 - This highlights what I was getting at. "Automatic measures" are not effective. It would be like measuring developer performance based on keystrokes per minute. Not good for anyone. – jmort253 Apr 10 '12 at 19:27
  • 1
    Thanks Pawel. I've actually been a follower of your blog for a couple of years now. Ultimately I thought your answer was the best because it outlines the problems and possibilities holistically... and just makes sense. – Matt Apr 10 '12 at 21:31
12

Bugs:

Programming generally involves taking many vague or abstract concepts and then tying them together to build something great. Judging developers based on the number of bugs in the code perhaps one of the worst ways an organization can shoot itself in the foot.

In a world where everything is so pro-Agile, there is no blueprint to follow. We're not building houses -- the purest form of waterfall model possible -- where we know every stud must be 16 inches apart because that's how we built the last 15 houses. Every software project is fundamentally different in some significant manner from other software projects.

Thus, bugs are just a fact of life. All software will have bugs, because many times developers are tying things together that have never been tied together before. Don't base performance on bugs, unless your goal is to demoralize and beat down your team.

Speed of Coding:

Some developers can code very quickly, but then 6 to 12 months down the road; suddenly, the progress comes to a grinding halt due to all of the shortcuts and bad decisions made in the code. Programmers aren't typists. You don't measure success by how quickly they can spew code into their editor.

Other developers are methodical, patient, and detail-oriented. They take the time to think about what approach will ensure the success of the product. Their decisions ensure that the product can still be supported years from now. They make sure that when one part of the system changes there isn't a domino effect created that reverberates through the entire codebase, knocking everything else out. Their code is well-commented, readable, engineered, and easy to maintain.

I understand why we as project managers want to measure this, but we must understand that this is a very delicate, possibly immeasurable balance, and it involves good judgement and decision-making by the technical people on the project.

Developers are generally paid on salary because their jobs involve the ability to make decisions and judgement calls, not produce little round widgets on a minimum wage production line.

Adherence to Requirements:

Software development is a creative process. It's also an engineering process. The engineering process comes into play when the developer has to make sure he or she understands what it is that should be built. Using logic, good documentation, sometimes formal methods and planning, a developer should be able to have a clear picture of what he/she must build.

The creative part comes into play with the actual solving of the problems that will ultimately arise during the development process. This sometimes occurs in other fields as well, like in construction. Perhaps there is some reason why the last two studs can't be 16 inches apart, so some creativity must be put into finding a safe and sustainable solution. With software, sometimes developers run into walls, and he/she must find a way around that wall by taking the things he or she knows about several different programming concepts and then bring them all together to come up with a great solution.

Now, the creativity part of software development doesn't mean the developer should go off an build whatever he or she wants to build; instead, the developer must still adhere to the requirements. The creative part is in the approach to solving the problem, not the actual problem itself.

With that said, sometimes engineers come up with creative ideas that can enhance the product. However, whether or not those items are implemented depends solely on who the product is intended for. At a startup, the engineer is likely to have a lot of freedom to innovate and create something that will instill passion in people. On the other hand, in a big corporation building custom software for a client, the goal is to deliver what the client wants, and that may leave little room in the requirements for unsolicited change.

Summary

In summary, the first two points are items that just don't make sense to measure. Bugs are a fact of life. Just look at them as unfinished features or part of the enhancing process. Speed of coding may not be a function of project success because coding is an engineering process, not a manufacturing process. Finally, requirement adherence may change from project to project and may not be measurable or important in some projects.

Instead, focus on evaluating the people. Is he easy to get along with? Does she help motivate other team members? Does she help collaborate with the team and generate problem-solving ideas? Do you trust this person to deliver what he or she says he's going to deliver. When bugs are found, does he take responsibility and passionately dig in and fix the unfinished feature (bug).

By it's very nature, software development is a subjective process, and you cannot use objective evaluation criteria to evaluate positions such as this without undermining yourself, your team, and your organization.

  • Regarding bugs: I understand what you're saying, but I'm not talking about the normal amount of bugs that inevitably happen in software development. I'm talking about the kind of bugs that should not get to QA, or any other person that is reviewing the work. I imagine we've all seen those kinds of bugs. Re: Speed of Code, I was actually talking about code performance (does it run fast vs. slow)... sorry about the confusion. I have updated my question. These are my initial responses, I'm still digesting the rest. – Matt Apr 10 '12 at 17:26
  • 1
    You just need to make sure you make that crystal clear whenever you talk about rating developers based on bugs. As a developer, I've seen a lot of crazy stuff, so when I see someone state they want to rate a dev based on bugs, and there isn't a clear explanation, I can't help but think the worst! – jmort253 Apr 11 '12 at 7:28
2

I do not think being a developer is relevant in answering this question. We live in a world where we are measured for our performance, no matter the activity or work we are doing. Being measured and compared with our peers are consistent with setting goals and achieving them, healthy competition, and weeding out those who do not belong. How one feels about it is immaterial.

The measuring stick you use should be a decomposed version of the stick used to measure the organization's performance.

There will always be unintended side effects to a metric. In many cases, those side effects are of no consequence to the organization. In other cases, the metric you choose will cause a behavior shift you did not want. So you approach it smartly. Don't measure something simply to say you are measuring. Measure it because you have goal you need to meet and that you have an eye on unwanted behaviors it may also produce.

  • 1
    +1 - "Don't measure something simply to say you are measuring. Measure it because you have goal you need to meet and that you have an eye on unwanted behaviors it may also produce." – jmort253 Apr 10 '12 at 14:57
1

Project managers should focus on the management of time an resources to meet the deadline. I would think no matter what the discipline, a PM should be able to rate how team members participate in project initiation (did they participate), how well they plan (did their lack of planning or lack of skill in planning adversely affect schedule) did they execute (did their estimates match their actual output or burn down rate?) and how well they monitor and control (correct work within their area of influence) and did they close out the project properly (commit lessons learns, submit final contractual deliverables, etc.).

However the premise and specifics of your question to the area of software development (number of bugs, etc.) is, I think, wrong minded. PMs should focus on the areas of discipline they know. While I think as a team there could be metrics defined that match the PM knowledge areas (e.g., classic PMI knowledge areas mapped to whatever the programming team thinks makes a matching metric for that area) I would avoid influencing and managing a programmer or programming team unless you were teaming and part of the team affected by the measurement(s).

0

You say you are looking to "apply some sort of rating" based on:

  1. number of bugs
  2. speed of code
  3. adherence to requirements

only the last one of those has any sort of subjective part to the measurement and even then depending on the rigor and granularity of your requirements tracking that may be up for debate.

So my question is why make this something the PM's rate? why not just make it automated? look up the number of bugs, the performance metrics on the nightly tests etc. for each developer's code and keep a running average or something.

Of course if you make it an automated metric you risk having the developers game the system. If you make it a purely subjective measure of the project manager's opinions of the developers code then it's a rather worthless measurement unless your PM's are all developers as well.

I'm a developer and I would not take kindly to a non-technical person rating my code on some arbitrary scale. It's hard for me to offer a better solution without knowing what you are trying to accomplish or if there is an underlying problem you're attempting to address.

  • There's nothing arbitrary about number of bugs found or adherence to requirements. – Scott C Wilson Apr 10 '12 at 1:08
  • Correct. My point was that if you're rating on a hard metric, it can be measured and tracked automatically thus it's not the PM doing the rating. If it's not an automated metric then it is likely arbitrary. – Ryan Apr 10 '12 at 1:11
  • Some things are subjective but not arbitrary. Is a person cooperative? Does he work well with the team and do his fair share? A lot of qualities which are evaluated in a performance review are not subject to automated testing. – Scott C Wilson Apr 10 '12 at 1:14
  • I agree. It appeared that Matt was saying the PM would be rating technical things like number of bugs, performance etc. which doesn't make sense to me. If this is just about normal performance reviews then of course his team-members and PM's have valuable input. – Ryan Apr 10 '12 at 1:18
  • 2
    Remember: you get what you measure. If you goal is to minimize the number of bugs submitted to bug tracker - count them, etc. Also see: pm.stackexchange.com/questions/5289 – Pawel Brodzinski Apr 10 '12 at 8:01
0

Project managers should absolutely have input into the performance reviews of the people on their teams.

  • 2
    How do you suggest they provide that input? Is there a quantifiable method that you've seen work? – Matt Apr 10 '12 at 17:28
  • I would suggest they provide the input directly to the supervisor of the person on their team (if it's a matrix environment where the team doesn't report to the PM). If the team reports to the PM, well, obviously the PM is going to review the team. – Scott C Wilson Apr 10 '12 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.