If SAFe must be continuously calibrated and measured in ideal days, why does the framework say recalibration is unnecessary?

Question

TL;DR

SAFe says you don't need to recalibrate estimation, but requires estimates and other metrics to be normalized across one or more Agile Release Trains. Context, question rationale, and the question itself (stated in a more answerable form) are below.

Context of the Question

One of the core practices of SAFe appears to be the use of normalized story points on which to base planning and costing decisions within the framework. Version 4.5 of the framework says:

SAFe uses a starting baseline where one story point is defined roughly the same across all teams. This means that work can be prioritized based on converting story points to cost...In this way, story points are somewhat comparable to an ideal developer day, and all teams estimate using a common method.

Functionally, this results in estimations being done in ideal working days rather than strictly relative level-of-effort scaling. No aspect of the framework appears to offer any alternative to ideal-days estimation, or define a point at which estimation is ever unmoored from man-hours. Furthermore, in the comments section of a now-deleted post, Dean Leffingwell has directly stated that normalized estimates are not a one-time thing. Specifically, he said:

Velocities are NOT normalized across teams. Estimating is. If you don’t normalize estimating, then there is no meaningful economics; you can’t figure ROI if you don’t know that the “I” is. If you want to scale agile, and there is no meaningful way to bid work across teams, and within a program, you will be blocked before you even try.

Why Ask This Question?

The point of the question isn't to debate whether or not this is an anti-pattern, but rather to understand this rather vague note on the page which says:

Note: There is no need to recalibrate team estimation or velocity after that point. It is just a starting baseline.

Authorial intent here seems unclear. Does it mean no recalibration within a PI Planning increment, or at any point within the process? Given the preceding context, it seems like normalizing of the estimation process (and therefore SAFe-specific application of measures like story points, load, velocity, and cumulative flow) is foundational to the framework.

The Question

Is there any mechanism within SAFe to estimate in level-of-effort rather than ideal days? In either case, how does the admonition that recalibration is unnecessary fit in with the framework's ongoing requirement to normalize metrics across teams?

Answer 1

Starting with the basics:

1. We all know that relative estimation is more accurate than absolute estimation.

2. We all know that effort based estimation is gamed! It creates multiple ill-effects of comparison of between teams, comparison between team members, etc. No matter how much one tries to keep managements from doing this, this comparison is inevitable. As managements practice these undesirable practices, teams game them even more.

So, to answer your first question "Is there any mechanism within SAFe to estimate in level-of-effort rather than ideal days", I would strongly recommend that one stays away from that temptation! It has been the bane for many of our ills in the domain.

SAFe's current guidance is: There is no need to re-calibrate team estimation or velocity after that point

If we stick to these basics, then my interpretation of the above SAFe guidance is as follows:

1. Use normalized SP to begin with in the absence of any data! It gives you a starting point. Further, since we have the same logic for all teams across ARTs, the starting point is common. It does help to associate a cost number for each SP (to begin with).

SAFe is making a debatable observation in that "While teams will tend to increase their velocity over time—and that’s a good thing— in reality, the number tends to remain stable. A team’s velocity is far more affected by changing team size and technical context than by productivity variations." IMHO, many managements would question this assumption.

Team productivity does change dependent on whole number of factors - from team member profile to team dynamics to technology changes, etc. Hence, in reality, it becomes important to keep doing planning and costing based on your most recent (2-3 PI) data.

Therefore, my second point below:

2. Once you have velocity data for a 2-3 PIs, you would use that for all future planning and costing. Instead of thinking of team capacity in terms of "ideal developer days", one would use the recent average velocity achieved to plan the same. Since you know the cost incurred in an ART per unit time (quarter/year), you would also be able to compute cost per SP delivered for that ART.

Hope this helps explain why one must not recalibrate but base your planning on recent data that the system is telling you.