How to report performance of a software development project?

Question

What metrics do you use in order to report performance of a software development project? And how do you collect them?

Answer 1

I like Agile specifically because of this -- tracking and reporting on performance is intuitive, detailed, and also based on actual team results.

---

Some background:

in agile, you break a project into "stories," small pieces of work that, if implemented independently, would make sense to a customer. For example, for the PM.SE site, you might say "Posting a question (title and content)" is one story; being able to edit (by the original author) is one story; adding tags is one story; and so on.

Stories are estimated in a vague size unit (story points) which are relative. A story that's two points is roughly half the work of one that's four points. Having an iconic, standardized example for a one or two point story, you can easily estimate everything relative to that.

In typical agile/scrum, you track your "velocity" (number of story points per day). Your "product backlog" is the list of all stories over the life of the product; at any given time, you can say "we have X points of stories to do for this project, of which, we have completed Y points." That's your performance.

---

Above and beyond standard agile practices, you can do fun stuff, like:

• Budgeting (eg. we only do up to 20-30 points per two-week iteration)
• Statistics (over the last ten sprints, our velocity has been rising from 30 to 40 points per sprint consistently)
• Earned Value: I personally really like this part. Track the "expected points done today" vs. "actual points done today" value. It tells you if you're ahead or behind.
• Product planning: You're two weeks into an eight-week project. You've completed 20 points so far (10 points per week); the entire project is broken up into 160 story points. There's no way you're going to make it.

On the flip side, some people consider points insufficient, and require developers to further break stories into tasks and track tasks on a task-by-task basis and estimate those in hours. Whatever gives you the visibility you need.

Answer 2

The best metric for showing progress of a software development project is working software. Let's say that you report status weekly or monthly to your customers or project sponsor / stakeholders. You can report that 22% of features specified have been completed (or more likely, 13% of features are complete, another 4% have been started, and another 5% are 50 to 90% complete). But what do those numbers really mean? In reality, those numbers mean almost nothing to a customer. Even for a feature you think is "done", the customer may have had a different expectation, and then your "done" is something other than "done".

On the other hand, if you have a demo of working software on a set day and time, the customer/stakeholder can see exactly what is done or not done, to what extent what has been done meets their expectation. This is the heart of the agile software development philosophy. But even if you aren't "doing agile", you can still "report progress" this way.

Answer 3

While not disagreeing at all with ashes at all, I have a slightly different take on the most valuable metrics and tend to report the following:

• Lead Time: Also used predictively as "your wait from this point is approximately N days". This is the average time in working days from the customer's request to delivery for a specific class of service.
• Cycle Time: Really the same as lead time, but with a different starting point for measuring. This is the average time in working days from the team accepting it as "ready" to delivery for a specific class of service. The difference between Lead and Cycle time is the time it spends sitting in a backlog being ignored.
• Throughput: How many work items are delivered on average in a given time period for a given class of service. This is essentially velocity.

Many people like the Cycle and Lead time measurements as much as the throughput because it explicitly tells you how reactive your team/organization is. I tend to avoid velocity of stories because it includes the teams estimate, and is thus easily gamed by the teams, especially if they're expected to show progressive improvement.

For the second part of your question, it depends entirely on the context of the team. I can't imagine running a distributed or large project without some sort of electronic tooling support, and have personally had good success with both Rally and Microsoft's Team Foundation work item tracking. For small projects, it's less than 5 minutes effort to capture it from a card wall each day, allowing easy graphing with Excel or on big visible paper.

Answer 4

I suggest you to track following metrics: - velocity: number of story points you team delivers during one iterations - number of escaped defects: defects that were escaped from your QA efforts and found after release by the customer - time to fix defect: average time to fix one defect

We collect those metrics using http://www.programeter.com built by ourselves to track performance and quality of our product development efforts.

Answer 5

I would slightly disagree with thesis that agile is exceptional with relying with tracking and reporting on actual team results.

I could imagine non-agile teams reporting real work progress, and I have seen agile teams reporting fiction (95% of work completed and 1 out of 5 story completed during review), it is all matter of rules and personal responsibility.

What really can help on this area is agile concept Definition of Done - shortly speaking set of conditions that must be fulfilled to treat piece of work as really, really completed that immediately can be reported and completed. I think this is one of the most important thing that need to be defined before any measure of performance start. Only then there is a chance that measurement will be objective and informative.