Managing scope creep in Agile

Question

I often get asked this question in the interview, and no matter what I answer the interviewers do not seem to get satisfied.

So, they start with asking how to manage scope creep in Project Management? I usually answer by telling that the requirements need to be clearly defined before the project starts, and if there is any change during the project then it has to go through a change control process which involves the change control board, etc etc

However, the next question they ask is, if you prevent scope creep from happening in the project how can you say that the project is Agile? Because agile is all about making changes to the project in an efficient way

How do you answer this question? Can you guys please give me some ideas on how to handle this?

Answer 1

TL;DR

Scope creep is a project risk, and must be controlled. However, in agile frameworks, scope is a variable constraint rather than a fixed one. To be an effective agilist, one needs to understand the differences between scope and change control, and how to properly apply a given agile framework to embrace change (which is a core value) without putting the overall project at risk.

This generally involves collaborative agreements, iteration-based change control, transparency, and effective stakeholder communication. Agile practices don't wish scope changes into the cornfield; instead, they adjust other constraints to make the cost of change visible so that the business can make informed strategic decisions.

Managing Scope

[I]f you prevent scope creep from happening in the project how can you say that the project is Agile?

In an agile framework, scope creep is really a problem caused by injecting new or unplanned work into the middle of an iteration, rather that adding scope to the overall project. All agile frameworks solve this through formal processes and ceremonies. In Scrum, for example:

1. New work should generally only be introduced during Sprint Planning.
2. New work that takes priority over current work required early termination of the current Sprint, and a return to Sprint Planning.
3. New work for the project must be prioritized by the Product Owner in collaboration with the stakeholders, so scope creep at the project level is managed through consensus that the proposed work is relevant to the project's goals.
4. In Scrum, iterations are a a fixed time-box with a relatively fixed cost (excluding capital costs), but scope is a variable constraint.
5. "Scope creep" in the traditional business sense of the term will extend the total run-time of the project. By adding scope to the project, you impact the schedule (generally making it longer). You manage this through transparency and effective communications with stakeholders.

Kanban and Lean have similar mechanisms for managing change. The point is that there's no free lunch. Adding scope adds cost or time to the project.

Trade-Offs

The project management triangle is generally understood to be a set of sliders consisting of scope, schedule, and cost. (NB: There are other models, but this one is the most common.)

The point of this metaphor is that you manage quality by adjusting the sliders, but they're interdependent. You can't fix cost, schedule, and scope at the same time, so increased scope will generally increase a project's costs and schedule as well, while often reducing overall quality as the organization attempts to increase scope without increasing costs or extending the schedule.

As a project manager, you need to be able to articulate the trade-offs for the project. In your specific case, you need to have a deeper understanding of how those trade-offs are made within an agile context, and what tools your framework provides for managing and communicating about those trades-offs.

Answer 2

I usually answer by telling that the requirements need to be clearly defined before the project starts, and if there is any change during the project then it has to go through a change control process which involves the change control board, etc etc

Your interviewers are right — this isn't agile.

The point of agile is that requirements change. Your understanding of the problem changes. Your understanding of the market changes. The thing you thought you needed to build in January is not the thing you know you need to build in June.

If you insist on defining your requirements up front, one of two things will happen:

1. you’ll build to spec, and find you’ve built the wrong thing
2. you’ll ignore the requirements, making the requirements-definition process wasted effort and the requirements themselves a dead document

(There are certain exceptions, involving extraordinarily well-understood engineering problems, extraordinary constraints, and extraordinary budgets. If they're asking you these questions you're almost certainly not on one of those projects.)

Now: scope creep. To a first approximation, there’s no such thing as scope creep in a smooth-running agile process. There's new scope — new priorities and new requirements, which lead to new work — but there’s no creep.

The first problem with scope creep in a traditional process is when the organization lies to itself. You’ve promised to deliver features F by time T at a certain quality level Q. (Q probably wasn't ever made explicit, and you probably were never going to make T anyway, but never mind.) But now you’ve silently redefined F to mean F+1, F+2, all the way up to who knows, 2F, 3F worth of effort, without ever explicitly making the tradeoffs to T or Q. So T is going to slip, and Q is going to slip, but no one’s admitting that no one outside the development organization even knows it’s happening, so somewhere after crunch time when you finally admit it to yourselves and finally give your stakeholders the bad news, they’re going to (justifiably) flip their lids.

This is the problem your change control board, etc., are designed to prevent—that cycle of denial and flip-out.

But, even with a change control process that keeps the organization honest, you have another problem. You set out to deliver F by T, and now you’ve increased the scope and pushed out the timeline and you’re on track to deliver 3F by 3T. From the point of view of the development team, everything’s fine.

The larger organization, however, is in deep trouble, because you haven’t shipped any software. You’re losing sales (or mindshare, on an open source project) to your more agile competitors who have, even if they’ve only shipped F/2 worth of features. You’re missing out on the opportunity to get feedback from users, so your assumptions about what features your users want aren’t being marked to reality, and if you do finally ship 3F you’re likely to find out it isn’t what they want; you’ve wasted 1F–2F worth of effort and you have another 2F–3F ahead of you just to catch up.

This is why the Agile Manifesto says Working software is the primary measure of progress. Fundamentally, agile development is pretty simple:

• you build a small thing
• you ship it
• you build the next small thing

The details are all about figuring out what the next small thing should be, and making sure that it’s built to an acceptable level of quality.

Where agile breaks down, it is often in the form of scope creep. That small thing becomes bigger, and the time it takes to ship it gets longer, and then it gets bigger still, and the next thing you know you haven’t shipped anything for three months and even if you’re still having daily standups (or scrums) and breaking work into iterations (or sprints) you can’t honestly call yourself agile any more.

Different agile practices defend against this problem in different ways.

Test-first design, test-driven development, continuous integration, and continuous delivery, when properly applied, ensure the software is always ready to ship.

High-level planning poker and low-level task estimation encourage developers to think features through and make sure they and the product owner have a common understanding before they start development.

Iterative development (allowing new tasks to enter the work stream only at iteration boundaries) limits the rate of scope change, incentivizes product owners to think through features before asking developers to deliver them, and minimizes developer thrash — the cognitive whiplash and context-switching overhead of being pulled out of your flow to drop one task and start another. Velocity tracking and evidence-based scheduling keep your iterations realistic.

Lean / kanban–type processes do away with iterations but “pull” features from the backlog only when developers are ready to work on them — which is to say, after their previous feature has moved on to QA and/or has shipped.

The real way to answer your interviewers is to turn the question around on them. When they say “scope creep”, what are they really worried about? Are they worried the product won’t ship on time? Are they worried quality will suffer? Are they worried about developer burnout? Are they worried about all of the above, and also that they won’t find out till it’s too late? These are related but different concerns, and a good process will address all of them in related but separate ways.

In the lean world, they say that if you want to find the root cause of a problem, you should always ask “why” five times. Ask your interviewers why they care about scope creep, and you’ll find your answer.

Answer 3

tldr; Agile shifts the focus to the discovering and creating a solution for the customer's needs. Agile values "Responding to change over following a plan" because one must "Welcome changing requirements" in order to "harness change for the customer's competitive advantage"

It can help to begin with an understanding of the classic Iron Triangle model, aka Project Management Triangle. The basis is that there are three attributes to control (scope, cost, and time) which all affect quality.

Let's look at classic project management first. In the beginning the scope is fixed then the PM hopes to properly estimate the cost and time. Once all of the requirements are written, then work can begin. As changes and issues arise, something needs to be adjusted. One common solution is to add more people, see The Mythical Man-Month, which adds cost. Realistically, once a project begins, the other two aspects (cost and time) became fixed as well; any increase to either is considered failure. The primary result is a lack of quality, often also over budget and late, sometimes failing meet scope agreements, very frequently not meeting the customer's need or missing the market opportunity.

Now let's look at the agile philosophy. Instead of creating a complete set of specifications (scope) at the beginning then attempting to predict cost and time, the core need is identified. A backlog is created to represent the desirements: desires which may or may not be actual requirements. This list is then ranked in order of value to the customer. Development begins with the core items of the highest value. One thing of importance to note here is that the focus has already been shifted to a solution (or product) mindset and approach over the classic project mentality. The team works in small time-boxes, fixed cost and time, to deliver the solution in small, quality, usable increments. The scope is flexible because adjustments are made as the customer and team understand more about what is actually needed; they add what was initially missing from the desirements and remove what is now realized as unnecessary or of little value. The work can stop at the end of any iteration when the proper level of functionality (scope) is achieved.

Answer 4

In the Agile manifesto this is described with this principle:

Simplicity--the art of maximizing the amount of work not done--is essential.

Product management

Scope creep in the backlog should be managed by the Product Owner. So this is any planning before the development team starts working on it. Delivering business goals, not just software features. Product Owners could practise something like impact mapping to find how new features have impact to prove that the feature is not just nice to have, e.g. scope-creep.

Development team

Developers can prevent scope-creep using Test-Driven Development. Tests could be unit-tests or acceptance criteria. Think first and "document" about what you want to build before building it, just like your requirements, but just-in-time.

One way XP’ers would keep themselves honest is to insist they write a failing unit test (demonstrating the need for complexity) BEFORE adding the extra complexity to the system.

http://www.agilenutshell.com/yagni

Then do not implement more functionality then needed to pass one of the tests.

In short YAGNI (You Ain't Gonna Need It). This should work for both functional scope as limit developer tendency to over engineer a system.

Answer 5

I think there are different answers to this question, depending on context.

CodeGnome has covered "out of sprint" high level scope well - basically add items to the backlog and either extend the project, or drop other features.

Within the sprint, this question is a bit more subtle. I think it is important to differentiate between "Scope Creep" and "Gold Plating", and "Change from learning".

Scope creep and gold plating should be pushed outside the sprint - and handled as above. The team has a set goal with stories with acceptance criteria, and due to the nature of agile projects the requirements are often vague when work is pulled into a sprint. Often during development as the work progresses lots of questions arise around "should we do X as part of this?" Where it is important to coach the team to follow the principles of maximising work not done, and working effectively to deliver the minimum to meet the story and sprint goal.

"Scope Change" however, is the essence of the value delivered by agile. Learning from a sprint might totally change the product backlog, allowing you to deliver much more value than was previously planned, in the same or potentially less time. By getting something out early and getting feedback the huge positive gains of agile are seen, and this is a positive rather than the potentially negative element of traditional scope creep.

Answer 6

First you need to determine exactly what they mean by 'scope creep', as the exact meaning may be taken as being different by different people.

Do they mean no scope changes, at all, whatsoever?

• Difficult to call this 'Agile', really. While it could still be possible to change time and/or budget, the fact that you're unable to accept changing customer needs would be a massive project smell in any sort of Agile project.

Do they mean no scope increases, but taking stuff out and putting it back is fine?

• This is pretty straightforward; when a scope change is needed, you just need to determine what to take out of the scope. Normally, another option would be to add time or cost (though, in many cases time is the only helpful one, as many ways of throwing money at a late project, whether through new team members or new technology, will only make it later due to initial sunk time costs). In this case, however, doing so would be 'scope creep', and thus your only really option is to manage the scope, both adding and removing from the scope for a net change of zero (or negative, if it turns out you really didn't need this huge feature X after all, so you can easily replace it with the smaller feature Y).

Do they mean any sort of scope changes are fine, but not too often or out of control?

• This is likewise straightforward, though in a different way. You simply need a rigorous change management procedure, where each and every scope change is properly/thoroughly evaluated, and responded to accordingly (whether that response is increasing time, taking something else out of the scope, finding some other solution, or just outright rejecting the scope change).

There might be another meaning they're intending when they use 'scope creep', but I think the above three are the most common. If all else fails, abandon any preconceived notions and just think about what sort of reaction, within the constraints given, would be necessary to bring value to the project, to the company, and to the customer. In the end, bringing value is what Agile is all about.

Answer 7

My opinion (IANASM) is that there is no such thing as "scope creep". In Agile, you should be working on the most valuable features to the business at the moment, for a short time period (the current sprint, in Scrum terms).

There should exist a backlog of work that is owned by the product owner, and they keep it in order of what will be the most valuable to work on next.

If some new information comes up, the product owner is free to reorder that list of work and reprioritize certain features over other features based on what they know at the time.

When the current sprint/iteration is done, the team picks up the next set of features they feel they can complete in the next iteration.

The concept is that those iterations should be long enough to get a set of features completely "done" (for your team's definition of "done" -- ie. deployed to prod) but not so long that changes to the business do not get reacted upon fast enough.

Scrum, at least, talks about being able to terminate a sprint before it is done, but it is made to be an expensive and disruptive operation, so that it should not be done on a whim.

Agile should balance committing to a set of work and a time period to do that work that doesn't make product owners nervous they will never get "the next thing".

Answer 8

Agile is all about the practices that work with continuously changing requirements. It is assumed that the requirements and the plan have a high probability of being suboptimal even three months down the road, because:

• The context changes
• Opinions and points of view mature or change
• Attempting a solution is a learning experience that sheds new insights into the problem and the possible solutions

This is a quote from Extreme Programming Explained:

XP is a path of improvement to excellence for people coming together to develop software. It is distinguished from other methodologies by:

• Its short development cycles, resulting in early, concrete, and continuing feedback.
• Its incremental planning approach, which quickly comes up with an overall plan that is expected to evolve through the life of the project.
• Its ability to flexibly schedule the implementation of functionality, responding to changing business needs.
• Its reliance on automated tests written by programmers, customers, and testers to monitor the progress of development, to allow the system to evolve, and to catch defects early.
• Its reliance on oral communication, tests, and source code to communicate system structure and intent.
• Its reliance on an evolutionary design process that lasts as long as the system lasts.
• Its reliance on the close collaboration of actively engaged individuals with ordinary talent.
• Its reliance on practices that work with both the short-term instincts of the team members and the long-term interests of the project.

Answer 9

There are lots of good answers here, but I thought I'd try giving a shorter answer to see if it would help.

There is no such thing as Scope Creep. There is only deeper understanding.

This presumes only a few things:

1. You and your client trust each other enough to collaborate frequently
2. If you have fixed delivery, you are able to flex on scope
3. If scope is fixed, you are able to flex on delivery
4. You build what you know is valuable for your customers
5. You don't ship $#^!

One is required for two and three. Four and five build up number one. It's a harmonious cycle that is true to the Agile Manifesto and one which Scrum incarnates.

I expanded on this here, if you would like to dig deeper.