12

This question is related to this question but I couldn't find this part answered there.

I am part of a large project whose entire product backlog has been sized in story-points. The estimate is roughly based on effort and assumes a certain amount of effort per SP.

My goal initially was to establish average velocity (in SP) per sprint of the team and then use this as a forecasting metric against the rest of the backlog (assuming that is also sized in SP).

Since the entire product backlog is not fully groomed there are several items that are not fully defined.

Is it possible to use the above method to arrive at an accurate plan/ forecast ? What potential pitfalls could this run into ? What other methods, if any, can be used to forecast completion date for the entire product backlog.

Improve this question
3
  • How large is the backlog relative to your velocity? If you're talking about projecting more than 5-6 sprints out, you'll basically be guessing.
    – Andrew Clear
    Aug 11, 2015 at 7:16
  • The product backlog is several tens of times the "recent" sprint velocity.
    – koushik
    Aug 17, 2015 at 12:10
  • Then you are effectively guessing. The accumulation of variation over that large of a time period is very difficult to determine with any amount of precision.
    – Andrew Clear
    Aug 17, 2015 at 18:18

6 Answers 6

Reset to default
6

If you can get 2-3 sprints of velocity, as DrewJordan recommends, this is hands down the best way. That being said, reality often finds us facing management who asks "When can you be done", before work has even started. If this happens, there is a method I learned from Agile Learning Labs that works well.

Caveat- To do this, you must do a full estimation of the backlog. If you are trying to estimate an entire year's worth of backlog, this won't be accurate. Nothing can estimate a year plus product well. It is still better than current models.

Step 1: Pick a reasonable release time frame. If you're doing agile, then you should be looking to release every three months at the most. If not, you lose a lot of the value of agile and will also find it hard to do agile when there is no real feedback coming in. This will also directly address the challenge of the Caveat listed above.

Step 2: Estimate the backlog. As mentioned in the Caveat, this needs to happen to be able to forecast a release date.

Step 3: Plan a theoretical first Sprint. Now this is not management assigns stories to a Sprint. This is honest to goodness team self-organized planning. Take the first story from the backlog and the team asks themselves, "Do we think we can get this done in the sprint (they are two weeks long, right? :) ). Then repeat this. Keep doing this until the team doesn't feel even close to comfortable that they can complete a story. This is KEY. If even one person thinks it can't be done, then stop and don't add the story. It is better to under estimate than over at this stage.

Step 4: Projection. Add up the story points of the sprint planning. Now divide the total story points in a release (remember, your goal is no more than about three months). At this point you have how many sprints it will take to complete the work. If that's more than three months of work, the release is probably too big and you should look at what is really needed.

Step 5 Reinforce this is a FORECAST. When you give the time estimate give it as "We have a 60% confidence in this schedule." Then follow this up with "After our first Sprint that should go up to 70% and after our second sprint 80%. By our third sprint we should have 90% confidence on either what our final ship date will be or what we will have done if the ship date must be fixed to a certain calendar point."

You'll never have more than 90% confidence, don't try. Mother nature and Murphy love to mess with anyone who says "I have 100% confidence".

Improve this answer
3
  • Thanks for this great and detailed answer. Regarding step #5 above, how can I determine the confidence level ? - e.g., what does 60% confidence level mean ? Is it 60% probability that this timeline will be met and 40% that it will be exceeded ? How will doing more sprints increase the confidence level ?
    – koushik
    Aug 9, 2015 at 17:55
  • Your confidence will be related to the variance in sprint velocity, and the likelihood of unknowns arising during development. The more sprints you've done, the better you'll understand those variances.
    – Andrew Clear
    Aug 11, 2015 at 7:19
  • At the start, your confidence is literally a guess. I tend to lean towards 60% because anything less than 50% and many senior execs won't even let the project go forward. Your initial estimate is an absolute guess, that's the point. The goal is to make sure people know that and don't hold you to it.
    – Joel Bancroft-Connors
    Aug 11, 2015 at 20:21
6

You can estimate a completion date, date range, completion sprint, etc by dividing the current backlog size (in SP) by the teams current velocity but...

  1. Definitely remind anyone looking at your forecast that it is an estimate
  2. Quickly check out how variable your velocity is. What is the team's last 3 iteration velocity? Current iteration? Total project duration? A highly variable velocity should prompt you to lower your confidence in any long-term forecasting results.
  3. List all your assumptions and the method you used to derive the forecast before presenting your number(s). Make sure your audience understands the impact of the assumptions you had to make.
  4. Be honest about your confidence in providing the forecast
  5. Try and provide a date range rather than a specific date with best, middle, and worst case numbers. You can go all kinds of fancy with Monte Carlo simulations and tools like Crystall Ball if you want.
  6. Use your scenarios as a chance to escalate risks
  7. DON'T make any long-term timing commitments based on your forecast numbers. The forecast should be used primarily as a tool to discuss and highlight risks/deficiencies that should be addressed to meet the customer's high level needs.
Improve this answer
4

It is worth remembering that Scrum is an agile framework and agile is all about responding to change.

It is fine and often useful to use the team's velocity to forecast when a backlog will be completed. But keep in mind that during the development process you will be expecting and welcoming change. At the end of every sprint you do a sprint review where you elicit feedback from your Product Owner and stakeholders. The team should hope that this feedback will help bring the product development closer to what the stakeholders and users really want.

The upshot of this iterative process is that long-term forecasts are highly unlikely to remain the same. It may even be necessary to revise the long-term forecasts after every sprint.

Improve this answer
1
  • +1 for the reminding that agile is all about responding to change
    – Ooker
    Dec 31, 2016 at 10:05
3

Yes, that's exactly what you should do. I'd recommend using at least 2 sprints, and preferably 3, to determine an average velocity.

It's also important to frame it in terms of probability: If you're velocity is 5, and there are 20 points, then you'll probably be done after 4 sprints. Unless, scope changes, something unforseen comes up, a risk materializes, etc.

What I'd do is, for the items that aren't fully defined do the best you can. Provide a range of forecast (i.e. say 4-6 iterations). This gives you some room for scope changes or whatever might come up, while giving your stakeholders an estimate of completion. Reinforce the ideas that they can change the completion date in either direction based on changing requirements or scope if they feel the need.

If you need something that's more of a science, you can have a look at PERT formulae. This can give you a statistical idea of when the project will be done, and how likely that estimate is. It's useful when your executives demand more specifics than "4-6 sprints" but also allows you to keep reminding them that it's a probability, not a given.

Improve this answer
1
  • Avg velocity can be really misleading if there is a lot of variance from sprint to sprint.
    – Andrew Clear
    Aug 11, 2015 at 7:20
2

You can plot your teams velocity against the backlog and get optimistic and pessimistic release dates.enter image description here

The wavy black line is your team's burn-up line - it fluctuates because velocity often fluctuates. The green line is an estimate of their best burn-up/velocity and red is an estimate of their worst burn-up/velocity. Where they intersect the fixed scope line provides an estimation of the release date. A video explaining this is here.

Improve this answer
1

I have good experience with Agile Monte Carlo estimation e.g. https://agilemontecarlo.com . Just factoring in the average velocity is not enough to get the risk profile of the project. Especially for communication with stakeholders the 25% and 75% simulated delivery date is great to know.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.