How to make Release Planning more accurate?

Question

I had an interview with one company. They use Scrum, but don’t like rough estimates during Release Planning. So, they asked me, how to make Release Planning more accurate (they don’t want more than 30 percent of deviations from the plan).

I suggested they do following:

1. Gather requirements.

2. Filter requirements and create scope.

3. Decompose scope to "nouns" (deliverable items). - i.e. WBS structure

4. Decompose "nouns" to "verbs" (work needs to be done to implement "nouns").

5. Estimate "verbs" considering risks.

6. Find "verbs" dependencies and put them in a timeline. - i.e. GANT diagram

7. Calculate release date.

As you can see, it is more complicated than initial story mapping, and looks more like “heavy waterfall” style, than “agile” style. As I understand Agile philosophy, accuracy of estimation of final release date is not high priority, because "responding to change over following a plan".

Going back to the planning list: in Scrum we implicitly did 1-3 steps during initial Release Planning and steps 4-6 during each Sprint Planning. I suggested to do all these steps during Release Planning explicitly.

Despite the fact that this was the first (and only) way I figured to improve the accuracy of a release plan, I don’t really like it because (in my opinion) this kind of planning does not conform to the ideology of Agile.

I understand, that accurate estimates demand more effort and don’t believe in “miracle” high quality estimation without effort. But maybe you can suggest to me other ways (or some tips) on how to make Release Planning more accurate without having complicated heavy waterfall-style planning?

Answer 1

Not surprisingly, I hear that question a lot. The basic problem with the question is that Agile disagrees with the fundamental idea of a fixed-scope/fixed-timeline project. In the question you were asked, there is the assumption that the end date of a set scope is knowable and the problem is that we are bad at knowing it (estimating). That's not really true. The end date of a given scope is not a knowable point in time. Rather, it's a shifting point in time.

Improving team capabilities, understanding of user needs, and more sophisticated understanding of the implementation as the project progresses actually moves that date around.

Take a look a this Henrik Kniberg video from 11:25 -> 14:45 (the whole video is great, but this part is more relevant to this question). It does a great job talking about forecasting and you can use this with the results from Release Planning.

You can see that I can make your project end date whenever you want if the deadline is what's really important or we can extend the date to whenever the scope is done if that's what's important, but setting both is just a guess and a gamble.

Answer 2

Sergey, the steps you describe will make your release planning more precise, but not improve the accuracy. Don't fall into the trap of thinking that having a detailed, explicit process during release planning will improve accuracy.

It's a trap that we are biased to fall into because, as humans, we seek to create a sense of control where there is none. The only way to make release planning more accurate is to do the planning closer to the release date. In fact, if you do the planning on the day of the release, you will have very high accuracy ;).

Like Daniel does a great job of explaining, Agile disagrees with the concept of fixed time/fixed scope and respects the fact that we just can't estimate with a high degree of accuracy at the beginning of a project. You should ask your management what is so special about the 30% deviation mark.

The best you can do during Agile release planning is to:

• Make sure you have the right people in the room
• Plan often in smaller, focused bursts
• Accept that the plan will change
• Be explicit about any assumptions

Answer 3

It sounds like your interviewers were defining a "release plan" as a fixed scope and fixed date scenario. One simple way to improve the accuracy of those plans by at least 30%.

Simple Answer

• Slash quality standards as needed to meet the deadline

---

Seriously though, this sounds like a trick question for a shop running Scrum. Fixed date and scope projects are all but doomed to failure for projects of any appreciable complexity (see Stacey Matrix below for illustration of 'complexity').

You might be able to make a case that predictive processes or methodologies can be overlayed onto more simple or even complicated projects; however, most software development is definitely in the complex range. For further reference, check out the Cynefin Framework for good definitions of simple, complicated, complex, and chaotic.

Here's how you might have answered that question

---

First, I would break up the fixed date and scope dysfunction; we would either release on a particular date but flex on the scope of what we released or release when a particular scope of work was completed. Additionally, the development crew would be charged with developing high quality, low technical debt software so future releases would not be slowed or made increasingly unpredictable by poor, technical decisions and corner cutting.

The Scrum Guide says:

Scrum is founded on empirical process control theory, or empiricism. Empiricism asserts that knowledge comes from experience and making decisions based on what is known. Scrum employs an iterative, incremental approach to optimize predictability and control risk. Three pillars uphold every implementation of empirical process control: transparency, inspection, and adaptation.

To ensure the reliability of release plans improves by 30%, I would insist development teams remain as unchanged as possible over the course of the release. This increases stability in the system, and will potentially increase productivity greatly when compared to teams that re-form often.

Next, I would work with the team to form forecasts based on their observed progress towards the release scope (or date) after 2-3 sprints of the new release cycle. This 2-3 sprint period ensures there is some meaningful and empirical data used to form the forecast.

Last, I would maintain meaningful visualizations of the teams' progress, taking into account their collective "cone of uncertainty" range to provide up-to-date estimates to stakeholders as new data emerges. I'd use basic math like that in Mike Cohn's "Velocity Range Calculator" to accurately estimate what scope would be complete given a date or at which date a given scope would be complete.

Answer 4

I will avoid going into the comparison with a fixed date release plan and just throw some suggestions how to improve the estimates of the user stories and related tasks (once you know how much they take and the team velocity is pretty straightforward to see which scope can be achieved in a given release).

• Split the bigger stories; smaller stories/tasks are easier to estimate
• you can use story points instead of time units (man-hours, man-days, ...), i.e unit-less numbers and if you make them relative it will be easier: you can compare similar stories (that is, a two-point story should expect to take twice as long as a one-point story) - people are more comfortable making relative estimates than they are making absolute estimates.
  you could start by finding out which one is the smallest story, one that can be done by one person in about a day or half day, and you estimate this as a 1. Then proceed with all other stories in a relative way.
• ​ Even more sophisticated is to use numbers derived from the Fibonacci series: 1,2,3,5,8,13,21, … (each number is the sum of the previous two). This special series has the advantage that the numbers increase exponentially and this enforces the concept that bigger is a task more complex will be to be implemented and should be split.
  You can even come out with your own (simplified) series, like 1,2,3,5,10,20,40,100. ​ The lower range (until 10) is designed to help teams to estimate small things they understand well with greater precision. ​ However, the upper range has increasing numbers, reflecting greater uncertainty. ​ If the estimates reach this range, it means that the story is too big for an iteration, it represents substantial risk and needs to be split.
• ​ Usually who volunteers for the task gets to estimate it. This assumes that the volunteer knows what is talking about but very often you get a biased result, that can be quite off at the end.
  ​Rather than place all of the burden and responsibility for an estimate on the shoulders of an individual estimates are owned by teams.
  ​The accepted approach for doing this is to use 3 point estimates and to converge these as a team. Techniques such as Wide Band Delphi provide techniques for combining multiple estimates with associated imprecision.
  ​A bit more sophisticated would be to ask everyone in the team and make an average. There could be variations on this like removing from the average the highest and the lowest
• Another possibility would be the so-called planning poker: for each task, everyone in the team decides on an estimation value and then all together present their value. The lowest and the highest gets to explain why they think so and then the process is repeated until the members converge on a common value.
  The big advantage from this process is that the value comes as if it was from the whole team.
• Keep an historical set of completed items for the review
• Dedicate part of the retrospective focus to the difference between the release targets and estimations and understand where it is coming from.
  Focus the estimation effort on the area in which there is the clearest need for it (such as where the difference between expected and outcome was bigger or areas particularly uncertain).
• Plan spikes (dedicated research iterations) for areas of uncertainty
• In any release plenty can go wrong to invalidate the estimate. This variability is magnified in the large scale (more or bigger team, longer cycles). Estimators can model stochastic (probabilistic) systems, such as development work by using Monte Carlo simulations. With such a model for release duration, you can include the impact of many elements of variability or risk, including scope change, attrition and long sickness. It is especially useful in very large and long-duration development and in fixed-price, fixed-duration projects.
  Tom DeMarco created an excel file for the Monte Carlo simulation called Riskology that is freely accessible.

But beware of the discussion: too much precision can harm.
​Spending too much time does not generally increase the accuracy of the estimates (and it bores the team!). ​ Spending too much time estimating is truly a form of waste. ​ At the end, teams must be careful to time-box their estimating and not attempt to turn this heuristic into a pseudoscience.

Answer 5

Making release planning more accurate means considering following cautions: But, you can not avoid considering 15-20% cushion.

1. Understand resources and their efficiency levels [This is possible only if you are working with team for long time]

2. Keeping a tab on iterations and Quality control.[This is possible if stake holder communication and discipline is maintained.]

3. Keeping an eye on upcoming technology [You never know what can make your life difficult or easier]

4. Reviews and Development keeping them hand in hand. [Keeping optimal QA turn around and review cycles]

5. Close watch on competitor moves [Assessing continuously competitor developments will keep you ahead in product vision. I know some products which went to white boards after competitor released awesome content.]

6. Make a scope-able split of features. And, prioritize the most important ones on TOP.

Answer 6

It sounds like you are in a low trust environment.

If you are in a low trust environment you need to do as much planning as required to get started. Once you start, and deliver, you will begin to create trust and erode the need for such detailed planning up front. Your answer is the only viable one.

This is so typical of large organisations at the beginning of an agile transformation, or a stalled one, that there is a whole section on it in the Professional Scrum Training.

There is ideology, the ideal scrum environment, and then there is the reality of your organisational dysfunction. As this organisation looks to need serious work and effort to keep them moving towards agility you should ask yourself if that is effort you are willing to contribute.

I for one love a challenge...

Answer 7

Completely agree with what @Daniel said and could add a little bit more: if the environment (such as unplanned distractions, approximate man hours available, ..) is constant then with time the team will realize their sprint velocity that could only help estimating how quickly a story can be delivered. This is not a contract but just an estimate.

And the estimates we are talking are maximum of days of up to a sprint period, not months as can be seen in waterfall.

When you add continuous delivery to Agile, you don't even need (or better - there is no point) creating estimates as stories fluidly come and go from the backlog with almost minimal lead time. With continuous delivery you can have multiple releases every day. This however works with software based products and might not directly apply to any other kind of products.