7

Our scrum team has provided high level estimates for the project we are working during release planning meeting.

The team provided guesstimated hours as well as Story points for each of the stories in the release backlog. Based on this we arrived at an end date and communicated it to the client.

As we got into Sprints, we got into a situation where the stories in the backlog needed to be decomposed into multiple stories and as a result of that our end date got pushed out and we communicated the same to the customer and he was OK with it.

We were having regular product backlog grooming sessions as well. But, when we get into Sprint planning meeting, the team's estimate on user stories are again changing and again as a result of that the stories are getting decomposed and the size of the stories also increasing - with this the end date for the release backlog also having an effect. And this happened more than couple of occasions for our release backlog.

Where are we getting it wrong? How could we rectify the situation so that the team can finish what was committed to client during release planning OR at least come close to what was promised?

Thanks

  • 1
    As you have more context than I have, I will act as the coach. Where does our gut feeling tell you the problem lies in? DougB has some good points based on his experience. Have you posed this to the team to solve, i.e. ask them where they feel they can improve. May I ask why you are re-estimating again in sprint planning and what value are you getting out of it? What opportunities do you see to that could help change the team? – Brett Maytom PST Jul 18 '13 at 4:08
  • 1
    I'm not sure that you are failing, except insofar as you aren't re-baselining your project plan. A more detailed answer awaits you below. – Todd A. Jacobs Jul 18 '13 at 17:39
6

As soon as you said "The team provided guesstimated hours" I recognized an issue here. When you do high level release management, you have to realize that it even applying story points to epics is a really inaccurate tasks and can only be used in broad strokes. Trying to actually estimate hours is bound for failure.

As some others have said, yes, it sounds like your team wasn't great at estimating, and that they changed the estimates many times. However, as a product manager myself, I think one of the great things about agile is that it tries to alleviate the pain points that come from trying to plan far in advance.

I have run release planning meetings with my team, both at the high level of breaking the large project into epics and estimating those (20, 40, 60, 80, 100 were the options), and at the lower level in the few months running up to release of using story points and team velocity. Agile teaches us that you will never know every little thing that has to be done for a release until you are close to it - that is the nature of software and working with humans. Some complexities will be missed, you may find a month before launch that the feature you thought was perfectly designed really needs some additional criteria, etc. I recommend when you do the high level planning more than 2 months in advance, you only communicate to customers what quarter you expect to release in, and if you feel really confident (the team has done this type of project many times and therefore can be expected to have a good handle on what's needed), perhaps the month.

3

This could be due to a number of things:

  • The requirements. The less detailed and clear your requirements are the more you can expect there to be significant error in your estimates. Garbage in yields garbage out
  • The estimating process. If your requirements are good enough to provide reasonable estimates the estimates can still be off if the process isn't rigorous enough or if enough thought isn't put into the estimates.
  • The corporate culture. I've seen estimates being fudged in order to tell a client what they want to hear because the corporate culture was one of "Get the contract and worry about how to deliver later". Depending on your organization there can also be consequences for speaking unpalatable truths.
  • The estimator. Generally speaking people are optimistic and will give you estimates accordingly. This is made worse if you get estimates from somone who isn't actually doing the work. When you get an estimate you should ideally challenge it and give the estimator the opportunity to ask for more time.
  • The expectations. If you originally generated high level estimates it is reasonable to assume that there will be variance, perhaps significant, between those estimates and what comes out of more detailed planning. As a PM you should manage stakeholder expectations by providing them with a range of delivery dates that will narrow as the planning becomes more detailed. For example, if you have a high level estimate that the work will take 20 weeks you may present to the client that the work will take 18 to 35 weeks. As requirements are fleshed out and you know more about what needs to be delivered and how, maybe the estimate gets revised to 25 weeks but you present it as 24 to 30 weeks.

Based on the description you've given, I suspect that you were expecting too much in terms of accuracy from a high-level estimate. The only solution to this is to go back to the team and ask them to re-estimate the remaining work giving you a most-likely-case and a worst-case estimate. Present those to your client and if they are good with it hit the Project Reset button and carry on.

  • 1
    The team had estimated the stories during the release planning meeting at the begining of the project based on their best understanding at that point of time.The team had to do that because product management had to commit to the client on when the sofware will be shipped.As a Project Manager gave an end date to product management with 75% confidence level. In Sprint meetings with more information the team estimation of story points is bloated up because the stories were split into multiple stories and story points shooted up for each story which resulted in pushing the end date – ramu Jul 18 '13 at 10:47
  • 1
    In hindsight it might have been better to have said "We are 75% confident we can hit date X, and are 95% confident we can hit date Y", or something to that effect. At the end of the day you can avoid a lot of problems if you under-promise and over-deliver rather than the other way around. – Doug B Jul 18 '13 at 12:14
3

TL;DR

Release planning is an estimated management target, generally based on level-of-effort estimates of large themes or epics. Sprint Planning is a team commitment based on iteration-sized user stories. They serve different functions, but the key is to remember that estimates are not guarantees.

Schedule changes do not necessarily imply that your estimates are inherently wrong. Your goal should be to improve the accuracy of your estimates, not to meet arbitrary scheduling targets. However, improving the accuracy of your estimates should reduce variations in your estimate-based scheduling to acceptable levels.

Release Planning in Iterative Development

Release planning is primarily about defining milestones and expected shipping dates. The cone of uncertainty is always largest at the start of a new project, and therefore the accuracy of the initial release plan should be a range with a defined confidence interval.

Because Scrum is an iterative development process, release planning should be revisited every iteration. Based on what is currently known at the start of each iteration, the Product Backlog can be modified to reduce scope or change priorities to hit fixed release dates, or the release dates can be moved to more-accurately reflect the team's actual progress towards the planned release.

Releasable Increments

There are a number of core ideas behind agile development, but one of the core concepts is the idea of iterations that produce "potentially releasable increments." In other words, while any given sprint might not contain the complete set of features planned for a given release target, each sprint should produce a working, functional, and (ideally) user-visible feature that could be released.

Put another way, every sprint should result in a product that has generated some amount of earned value. While the product might not be feature-complete, it is nevertheless functional and potentially shippable at the end of each sprint. This gives stakeholders the ability to convert earned value to market value at the end of every iteration, although they are certainly not required to do so.

Continuously Re-Estimate

The key to successful Scrum implementations is to continuously re-estimate. This re-estimation obviously includes epics, themes, and user stories, but it also includes re-estimating project timelines and the scope of target milestones.

Deviations from initial estimates are to be expected, and are handled by the inspect-and-adapt process baked into Scrum. However, if you find that your estimates are not growing more accurate over time, then you need to:

  1. re-evaluate your estimation process to see where you might improve, and
  2. determine whether your original scheduling-baseline was based on faulty assumptions.

Management Needs to Embrace Change, Too

"Embracing change" is not the sole province of the development team in Scrum. It is also the responsibility of the Product Owner and stakeholders to embrace all necessary changes to scope, schedule, resources, market forces, and other realities that are needed to ensure the project either succeeds or fails early.

Sticking to a plan made months or years ago, without taking stock of where the project is now, is one of the most common ways that projects fail. Agility doesn't guarantee success; it just expects the visibility and transparency of the project to enable stakeholders to make well-informed decisions.

  • 1
    Thanks for the response. I completely agree to everything you said. In an ideal Agile world, client side product owner is probably part of Scrum team. In our case the end client only cares about what as a vendor we can provide in terms of scope and timelines. So, frequently revisiting plans will have an effect on either scope or schedule or both - So, my concern is how do i satabilize a plan and estimates in such a way that we as an organization deliver to client what they need based on our initial estimated promise – ramu Jul 19 '13 at 12:03
  • 1
    @ramu There is no known methodology that will allow you to keep a static release date with an iron-clad guarantee of zero changes to scope or resources during the lifetime of the project. If that is the expectation, then the failure is in effectively communicating the Theory of Constraints to the client. Good luck! – Todd A. Jacobs Jul 19 '13 at 15:05
2

In Scrum, we teach teams not to take on "Risk" where it is not theirs to take. Your team did their best estimate at the start and as time progresses, you and the client realise that the team's ability to estimate is very bad.

The first question I am going to ask the PO, is what risk mitigation did he/she have should a team likely estimate incorrectly? I think it is very well accepted that technical teams are bad at estimation, regardless of method.

The next is have you been tracking the teams rate-of-work (velocity) versus the original estimate. Regardless of technique you use, velocity is ultimately a ratio that is applied to original estimates. For example, the trend indicates the team takes 72% more time than estimated. Thus, applying a velocity to the the original estimate should give you an indication of the "Real" target date.

Scrum does not officially dictate how teams should estimate and how release planning should be done. However, there are several proven patterns that work.

Basic Principles

  1. Groom your product backlog to a level where the team have a gut feeling of what has to be done. This should be done very quickly and without full analysis.

  2. The team should then estimate all the PBI's with a relative scale using story points or t-shirt sizes (I medium may be twice as big as a small). Theoretically, a PBI that is a 2 should take roughly double the effort/time that of a 1.

  3. When new items are created then the team should estimate those too. The team should still estimate against the original scale and not factor in their improved performances and tooling; doing this results in a ratio applied to velocity which will give you weak results.

  4. Create tasks in sprint planning to help the team focus on how to deliver the PBI. If you really want to estimate tasks, you can but I often find this does not add much value.

  5. Simply track how much work the team has delivered in a sprint, i.e. the team managed to complete 42 story points in the sprint. This is your velocity.

  6. Thus if you have 450 remaining points in your product backlog, it is going to take you eleven sprints to complete (450/42 = 11 - rounded up).

  7. Educate the management and PO that the team give a ESTIMATE and NOT ACTUAL times

As a Scrum Master, your challenge is to help the team first stabilise their velocity and then improve their velocity by reducing waste.

1

It sounds to me like you're doing a very waterfally scrum and that's where you're running into issues. Your "end date" should be the end of your sprint. Nothing should be committed to past that. It sounds like you're taking a giant set of stories across multiple sprints and committing to far out dates to have them all completed. That's waterfall and without waterfall-style up-front planning and design you'll see the exact problem you're running into.

In perfect world scrum, time is fixed, stories are adjustable. That "end date" shouldn't change, only the number of stories committed to and only during sprint planning. If you're in the middle of sprint and it turns out your estimates were off, you bring in your product owner and negotiate the sprint backlog based on the new understanding. That could mean moving things back to the product backlog, swapping out stories or, in extreme circumstances, aborting the sprint and starting over. If you're following scrum, a complete abort of a sprint should only set you back a week or two. Not bad compared to waterfall where similar issues can set you back months.

One of the basic premises of agile methodologies is to realize that we are bad at estimating and that the further out the estimate, the more incorrect it usually is. That's a major point behind doing short (2 to 3 week) sprints. As soon as you start giving committed estimates that are months out, you're back to waterfall and you need to plan and design like waterfall.

  • 1
    I do not agree with the statement of "very waterfally scrum" and that this is jumping to conclusions without facts. There are obvious estimation challenges and likely understanding of estimating. – Brett Maytom PST Jul 18 '13 at 4:11
  • 1
    The question mentions estimates changing negatively during sprint planning and a release backlog with an end date. This shows they have committed to dates and features beyond the current sprint. You can have all the stand-up meetings, card walls and story points you want, but if you're committing to a release date of certain features several months down the line, you're not agile and that's not scrum and pretending like it is is just doing waterfall development poorly. – NightMan Jul 18 '13 at 13:55
  • 1
    For our project, the end client would not care if we use Agile or Waterfall, it is an endeavour from engineering team to deliver in increments to product management. Hence, as an organization we have to commit a release date, although we can commit at Sprint level to product management. So, the issue is as a company we cannot go to the end client and keep changing the scope/dates frequently. That is where the main issue – ramu Jul 19 '13 at 11:59
  • 1
    And it's a common issue. I've dealt with it on nearly all of my scrum-based projects. It's unreasonable to ask a team to commit dates on work that hasn't been fully analyzed. Scrum removes that up-front analysis by positing that things will change so much during the project anyway that those up-front estimates will not remain valid. The solution is that the team will only commit to work in the current sprint with that work being fully releasable by the end of the sprint. – NightMan Jul 19 '13 at 15:51
  • 1
    (Continued...) Once you start giving "release backlog" estimates beyond the current sprint, you're not accepting the scrum solution any more and need to go back to doing all that up-front analysis, waterfall-style. I just don't see any other way. Also note that incremental development does not equal scrum. You can do incremental development within a waterfall methodology. – NightMan Jul 19 '13 at 15:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.