How can the economic efficiency of a scrum team be measured?

Question

I'm given the task to develop a measurement on how efficient our team is within our current development process (scrum). The measurement has to be done for the complete team not for individuals.

My idea was to use the count of team members and velocity. But because velocity is a per team measurement the result will not be something useful.

Any ideas?

Answer 1

I would argue that there is no clear way of defining "efficiency" in any meaningful way.

Assuming that efficiency is somewhere along the line of "value/time", you have two insurmountable problems:

1. Value does not depend on the team exclusively: a very good team working on a very bad product will produce less value than the same team working on a very good product

2. Value is relative to the user and not necessarily an absolute number that can be measured exactly, for example think of "brand value" -- companies often ship features for free in order to build brand value: how would we estimate it?

Furthermore, maximizing value is the purpose of the Product Owner, not the team: in other words, the team cannot be determinant in choosing which stories are prioritized first. In other words, it is a myth that value delivered is proportional to time:

1. Assuming a "perfect" PO, the most valuable stories are developed first. Efficiency will therefore go down as time increases because the stories will be increasingly less valuable

2. Assuming an "imperfect" PO, sometimes the team will not develop valuable stories, and this is natural as the PO learns better what is valuable as the product is delivered incrementally. The efficiency measured this way will not be a measurement of anything intrinsic.

Lastly, using something across the lines of velocity is flatly wrong. Velocity can be easily doubled by halving the "size" of a point! Of course, you can ask teams not to do that, but you can't prevent teams from inadvertently changing the size of a point across iterations -- it just happens, sizes "drift".

In conclusion, my advice is that you take a step back: why are you trying to measure "efficiency" in the first place? Do you not trust your team to be efficient, or are you trying to measure some other quantity and using "efficiency" as a proxy?

In the first case, I would advise you to figure out a way to trust them, because clearly trust is a prerequisite in most cases. In the second way, you need to find a different way to measure what you need. "Efficiency" won't be a good proxy for anything because of the reasons above.

Answer 2

Maximize business value delivered

For a given team size (investment) you want the highest returns by way of delivered business value. Ideally, you should ask the Product Owner to assess this. However, this assessment by the PO will be subjective.

So, you need an objective measure. In your comment, you said you wanted to compare performance for the same team over a period. For this purpose, it is OK to use velocity as a measure of value delivery. However, you should apply these two conditions:

1. You use documented reference stories for various story points so that your estimations don't veer too much off track over a period.
2. If you have multiple teams, you use the same common set of reference stories.

In this push for higher velocity, the following are potential casualties. So, measure these also and make sure they are not compromised:

1. Quality: Number and severity of bugs found in production.

2. Team Morale: Minimize number of fire drills.

3. Technical Debt: Makes the code hard for the team to change. If the debt is not repaid, then it will keep on accumulating interest, making it hard to implement changes later on.