Should we take into account scope changes to the Sprint to measure the team efficiency?

Question

My question is about scope changes to the sprint after stating it. The Scrum guide says this about adaptation:

A Scrum Team is expected to adapt the moment it learns anything new through inspection

What about a Scrum Team that agrees to start a new Sprint with 20 tickets. The team announces it to stakeholders and everyone is happy, but right after starting the Sprint the team learns that one ticket is infeasible (for example, technically it can't be completed) at least during the current Sprint.

In terms of Scrum, how bad is this? I think what can be improved is Sprint Planning and Backlog Refinement. The team should have noticed that this ticket cannot be done.

I am asking this question because the management of my company checks only the velocity chart in JIRA, and this chart doesn't take into account scope changes. At the end of the Sprint, we have the commitment bar with the story points we started with, including ticket that we've removed later, so the team is considered to be under-performing.

Answer 1

Velocity is not a metric for team performance. It's a measurable dimension you can attach to the work being performed in the sprint so that the team can forecast future work in future sprints (see the concept of Yesterday's Weather).

If you use velocity for team efficiency and communicate it to stakeholders, and then stakeholders evaluate how the team is doing or evaluate progress based on this number, then velocity will become a target and it will be a abused. Management will use it to put pressure on the team and punish them for low values, and the team will use it to game the system and inflate the number. In the end everyone looses and nothing improves.

Should we take into account scope changes on the sprint...

Yes.

... to measure the team efficiency?

No.

Velocity is a tool. Don't take this tool away from the team or things will become more and more about improving abstract numbers and less and less about the safety of the working environment or the proper features that need to be built to bring in the needed value.

As the other existing answer mentions, focus on the Goal of the Sprint, not on the velocity. And try to find a way to communicate with stakeholders that doesn't rely on ill understood concepts turned into metrics. Trying to manage velocity as a performance metric will just give everyone ammunition to use against the communication and collaboration that should exist between the team and its stakeholders (assuming you are indeed trying to use Scrum and not just doing traditional command and control development in sprints).

Answer 2

TL;DR

Your management team needs to be educated on how Scrum works, and to modify its key performance indicators to be outcome-based rather than effort-based. Its current approach to metrics is not only an agile anti-pattern, but is actively harming the Scrum Team's adoption of the framework.

Analysis and Recommendations

Problems with Planned Work

What about a Scrum Team that agrees to start a new Sprint with 20 tickets...but right after starting the Sprint the team learns that one ticket is infeasible...during the current Sprint.

There are at least three key problems here:

1. The Scrum Team commits to a Sprint Goal, not to completing a bunch of (potentially unrelated) tickets.

   CodeGnome's Scrum Tautology℠ says:

   Always remember that the goal of a Sprint isn't to complete lots of backlog items. The goal of a Sprint is to deliver the Sprint Goal.

2. The Scrum Team can adjust the scope of the Sprint with the cooperation of the Product Owner, but only if it doesn't jeopardize the Sprint Goal.

   If removing the item from the Sprint Backlog would make the Sprint Goal obsolete or incomplete per the Definition of Done, then it can't be removed without a return to Sprint Planning.

3. If the Developers have a Sprint Backlog item that isn't essential to the Sprint Goal or the Developers' plan for delivering it, then it doesn't belong in the Sprint in the first place.

   The Scrum Guide requires that the work selected from the Product Backlog form a singular, cohesive goal for the increment the Scrum Team will deliver by the end of the Sprint. While it's natural to have architectural runway, chores, in-Sprint bugs, or other items crop up during a Sprint and end up on the Sprint Backlog, the Developers should not be committing to work that isn't related to the Sprint Goal or necessary to deliver the planned Increment.

Problems with What's Being Committed, and to Whom

The team announces it to stakeholders and everyone is happy[.]

While Scrum values transparency, the Scrum Team is a self-managing unit. The Product Owner should certainly be communicating with stakeholders about the Product Increment being worked on, and the Sprint Goal and perhaps even the Sprint Backlog should be artifacts available for anyone interested in viewing (but not kibitzing about) them, but there is nothing in the Scrum Guide that requires the Developers to commit to specific Product Backlog items, and definitely not to commit to non-Developers about specific Sprint Backlog items.

In fact, committing outside the Developers to Sprint Backlog items is such a huge anti-pattern that I would urge the team to stop doing it right now. The Sprint Backlog is the Developers' plan to meet the Sprint Goal. The artifact's contents are entirely up to the Developers, and they can change the contents of the Sprint Backlog as they see fit at any time.

What you're describing is like an outgrowth of both per-person performance tracking, as well as an expression of the 100% utilization fallacy. Since the goal of Sprint is to meet the Sprint Goal, it doesn't matter whether the Developers have one ticket, a hundred tickets, or don't use tickets at all. The only correct measure of a Sprint is whether or not the Sprint Goal was met within the agreed-upon Definition of Done.

In other words, the Scrum Team should measure outcomes, not ticket volumes or the contents of Scrum artifacts. Ultimately, management needs to measure the same thing: an outcome. That usually means asking "Did the team deliver the potentially-shippable increment it planned, or not?" It's very binary: it's true or false, yes or no.

Problems with Non-Agile Metrics

While the Scrum Team is clearly still learning the framework, your leadership team doesn't seem to think there's anything new or different that they need to do to lead an agile organization. This is a prima facie fallacy.

As a purely pragmatic matter, most organizations adopt Scrum without doing any education at the middle-management or executive levels. This often leads to new Scrum Teams like yours trying to implement a new process while their leadership is still "holding them accountable" in very non-agile ways.

At a bare minimum, the Product Owner and Scrum Master need to work with stakeholders and the leadership team to explain:

1. How Scrum works.

2. The appropriate inspect-and-adapt inflection points for management review and input.

3. The need to adopt a whole-team approach to measuring outcomes, rather than trying to measure individual productivity.

4. Tools must support a process; they should never drive them.

   In this case, management is forcing the Scrum Team to build processes around legacy ticketing metrics rather than defining new and better metrics or better processes, and then finding or modifying the tools used to measure or support them. Unaddressed, this will lead to failure. Your mileage in this case will not vary.

5. You get what you measure.

   If management wants to use completed tickets as a primary metric, then the Scrum Team will inevitably turn each work item into 120 tickets. One goal of frameworks like Scrum is to create a predictable delivery cadence, not an ever-increasing one. By focusing on ticket volume instead of outcomes, you will get ticket volumes and not necessarily useful outcomes. Q.E.D.

Core Recommendations: Education and Communication

The Scrum Team needs to do a better job of educating the leadership team about how Scrum functions, and communicate more effectively with them. Collaboration between the Scrum Team and the rest of the organization is fundamental to success, both for the team and for the company. If neither the Scrum Master nor the Product Owner is sufficiently experienced with Scrum to improve the communications outside the team, then requesting an agile coach to help the company adopt Scrum more effectively may be a good solution.

If no one is willing or able to fix this underlying dynamic, then it's probably time for the Scrum Team members to brush up their resumes. Without effective collaboration with stakeholders and leadership, the project will fail, the Scrum Team will get blamed, and the executive team will reward each other with bonuses and stock options after firing all the people doing the actual hands-on work. This is more or less what they teach in business school, so expecting a different outcome without changing the underlying parameters is basically just wishing the problem into the cornfield.

Other Related Answers

• Is it possible to remove Stories during a Sprint in Scrum?
• Story points or number of stories?
• What to do when capacity is reduced in the middle of the Sprint?
• Does every Scrum story need to be completed at the end of the sprint?
• If we can’t finish all tasks, does this mean we are doing Scrum wrong?
• And search for my answers that mention "deliver the Sprint Goal"

Answer 3

In the Scrum framework, the team does not commit to a body of work. The commitment made by the Developers is to the Sprint Goal, which is created through a collaborative effort between the Product Owner and the Developers. By committing to a goal rather than a body of work and ensuring that the goal can be satisfied in an appropriate level of effort, the team is able to adapt to unplanned changes, such as unplanned work or unplanned decreases in capacity.

Stakeholders outside the Scrum Team shouldn't necessarily care too much about the sizing or scoping of Product Backlog Items. They may be interested in the current Sprint Goal, which Product Backlog Items relate to the Sprint Goal, and the current state of those Product Backlog Items and perhaps tasks associated with them.

If the team continually has issues with meeting their Sprint Goal or if they are feeling like the undiscovered work that emerges from a refined Product Backlog item is impeding their effectiveness, that would be a good topic of discussion at a Sprint Retrospective. Looking at refinement and Sprint Planning would be a good place to start.

Try to shift management away from velocity charts. Instead, move to a binary: Did we achieve the Sprint Goal? Use the Sprint Review to talk with the key stakeholders about if the team is achieving the Sprint Goal and what the different stakeholders think about the rate of value delivery, using this feedback in the Sprint Retrospective to come up with process improvements.

Answer 4

Scrum focusses on a learning organization. Discovering that you cannot complete a ticket after you agreed to do it, is a mistake. Sometimes this happens and you should try to avoid it. Use the retro to think about how.

It is quite normal for most teams not to complete everything you promise. Developers are human beings and optimistic. If you do over 90% you are doing very good.

I disagree with the responses saying you need to complete a sprint goal and that is your only commitment. In theory this may work, but it depends a lot on the type of development your team does. By stating you complete tickets it looks like you do a lot of functional management and bug fixing. Finding sprint goals every two weeks if that is your job is very complicated.

Velocity is nothing but a tool to help to estimate the amount of work that can be done during a sprint. As a product owner, I always start with a base velocity, but we take into account if people are ill, have a day off or attend a course. This results into an estimate of the number of storypoints that can be scheduled for the sprint. Then we make a list of things to do in order of priority. During a sprint that may or may not work out as intended. As said, we are humans, there may be a lot of external things affecting what really can be done. For each individual ticket, we always make big mistakes in estimating it. But over all tickets for a sprint it works quite well.

I am a very experienced product owner and I have worked with many totally different teams with a huge variety in tasks. Educate your management not to interfere a lot with this. A team works well if it is autonomous and putting pressure on each perception of possible failure prevents learning and it makes devs cautious in promising anything, causing under performance. Taking some risk is part of life.

Answer 5

I see a couple of really good answers so I'm going to keep this as short as possible and just let you know that we are trying to solve for this with minware:

As other's have mentioned your organization shouldn't be directly measuring productivity by just looking at the velocity report. Visualizing the impact that scope creep had on the team's work-in-progress during the sprint is one ways that we are trying to help!