Project Management predictions vs Agile/Scrum flexibility

Question

Scenario:

The business has gathered requests and ideas from customers and new product development.

The business and Product Owner decide they want to create and implement feature A.

The Product Manager works with a BA to define requirements and translate them into user stories that meet the team's definition of Ready.

The Product Owner and Manager work to create a Plan of Record with deliverables.

The during grooming, the Product Owner gives the team the scoop on the new product, and we have developers ask questions and assign Story Points.

Where I'm running into an issue, is from the standpoint of the business, they want estimates as to how long it will take to develop, test, and implement this feature from start to finish. To me, this request feels a bit waterfallish, and not in line with the purpose of Agile development, which is to pivot and adapt as complexity and the need to change are discovered.

From the business perspective, "as long as it takes" isn't really an acceptable answer. So my question is how do you balance the business' desire for some type of estimation for overall effort, and the desire to remain iteratively agile.

Answer 1

[...] how do you balance the business' desire for some type of estimation for overall effort, and the desire to remain iteratively agile.

Educate them on why a big upfront estimation can be completely useless.

The problem with business people is that they often lack project management skills and don't understand how software development happens. Consequently, they are unaware of the existence of the Cone of Uncertainty and its implications:

• Estimates (e.g. on duration, costs or quality) are inherently very vague at the beginning of a project
• Estimates and project plans based on estimations need to be redone on a regular basis
• Uncertainties can be built into estimates and should be visible in project plans
• Assumptions that later prove to be mistakes are major factors in uncertainty

When you give business people estimates, because they don't understand what's involved, they take it as your promise to deliver them a product, in a specified amount of time and at a specified cost.

If your project is small (e.g. couple of weeks or months), you have good requirements, you have good specs, you know how you are going to build it, you know the people who will build it, and market conditions are stable, then whatever you come up as an upfront estimate might even happen, and everyone is happy. Hooray!

But most projects are not small. Requirements even if good can't 100% cover everything, team members might change, market conditions might change, because you work for a longer period of time not just a couple of weeks or months you have more risks and you are forced to make more assumptions, etc., so an estimate provided today isn't worth much six months from now.

Unfortunately, business people take the estimate as a given and plan all sorts of other things around the product (launch events, using it as a dependency for other business activities, allocating budgets for other things, etc). When, eventually, you don't deliver on time, a lot of their plans go flying out the window. And it's your fault, of course.

You can obviously provide an estimate up front and give business what they asked for, but you will be keeping them ignorant and they will end up making plans based on something that might not happen.

So, educate them:

• on what an estimate is. Provide a range, not a number;
• on what risks exist, what chances they are to occur, and how that will affect the estimate;
• on what assumptions you made and what will happen if they prove wrong;
• on why estimates are almost always wrong (there has been a lot written about the topic);
• and on why, because of the previous items, an iterative and incremental approach works better. Mention also other advantages (there has been a lot written about this topic too).

If you educate them on why you prefer an Agile approach, and if they understand, they will then know what to expect and will collaborate with you when things go off track, and they can then adapt their plans and goals accordingly. If you just give them the estimate as they asked, they will just blame you when things go off track and will insist on you giving them what you promised, when you promised it, because other plans were made that depend in it.

Answer 2

This is a massive topic, but the highlights would be this:

1) It is waterfall, not Scrum. You're doing a design, creating requirements, and planning out the implementation. The fact that you're using agile-ish terms is just adding confusion. Now, that's not a condemnation. Maybe you're perfectly good using waterfall, but the Scrum and XP terms just add confusion.

2) Part of the point of Scrum is that you can ship the product after every sprint, so the conversation about when it will be done is very different. It will be done every sprint and will be more full-featured every sprint. The business can decide to stop funding the work as soon as they have enough or have discovered that progress isn't moving in a direction they can support.

3) You can sort of forecast in Scrum using burn-up charts. I say 'sort of' because part of the point of Scrum is to adapt. The burn-up chart is based on what you know today and it will change. In its healthiest cases, a burnup chart is usually used to answer questions like: how likely is it that the product will have these features by this date. That is subtly different than asking "when will it be done", but that small difference is very important.

Answer 3

It's common for business to have some level of distrust towards IT, especially if they were disappointed by IT before. If you gain their trust the situation will most likely change and they'll get off your back. For that you need to:

1. Understand them and their pain points, speak their language, feel for them. If they recognize that you're one of them they'll trust you more. For that you need to study the domain, learn their workflows.
2. Deliver and excel their expectations. Good UX skills together with strong dev team are necessary for this.

At first you'll have to play by their rules and provide estimates. But once you start to deliver and gain their respect things will change.

And when providing estimates don't forget to mention that after initial implementation you'll be ready to hear out their feedback. And if they change their minds about something or will have suggestions - you'll be happy to address them. They'll get their estimate and you'll get your agility.

This all depends on the organization of course. Some companies (especially large ones) may have strict policies and even if business wants to be agile they don't necessarily have means and authority to change the rules.

Answer 4

There is nothing about Scrum that stops you coming up with a long range estimate. The Scrum approach makes those forecasts easier in several ways. Backlog items are normally designed to be independent and estimate-able and the fact that in Scrum the delivery date is always the end of a sprint means that your estimates only need to be accurate to the nearest sprint.

Forecasts can be wrong of course. There is always the possibility of unanticipated work, delays or scope changes, but that's true whatever approach you use and is nothing to do with Scrum.