Handle stories which can not be estimated

Question

Do scrum have a method for handling tasks which can not be estimated in any meaningful way? I'm going to give an example we faced recently. We sell some hardware devices and recently found out that after we upgraded our compiler the new compiler introduced a (subtle) bug on some of our older devices.

It is still unclear if the bug is in the old chip or the compiler but it doesn't really matter to us since we need to fix the problem - and the only way we can fix it is to locate the part of our code which is causing the problem and write it in a way so it compiles into a binary which runs as expected.

From our point of view it's a very black box problem. My initial gut feeling was that it would take 3 man weeks, but it ended up taking closer to 3 man months. If the developer had been lucky and started looking at the part of the code which turned our to be problematic from the start he could have solved it in 2-3 days.

We don't really know how to handle these problems. It is completely impossible to break down, and while working on it there is very little progression.

I like to compare it to making a mathematical proof. When you have the right idea it is fast and straight forward (or can be), but until then it can be extremely hard/impossible to say if you are close to solving it. You can sit for hours or days, and when the right idea suddenly pops into your head it's solved in 20 minutes.

Answer 1

In my recent practice, when we had an item the Development Team could not split nor estimate, we played a 'spike' i.e. an investigative piece of work with a time-box and agreed outcome (further items that can be estimated). If such an investigation came back with enormous estimate such as the OP alludes to, we would have a major problem and exceptional situation that has a high risk of disrupting or blocking short term Product development. Scrum wouldn't help us much, we'd be looking outside the box for a resolution:

• Can we get a contractor?
• Can we steal someone from elsewhere in the org?
• Can we compensate the customer?
• Can we defer and risk being sued?

Doing Scrum doesn't mean you can forget about all those traditional/boring things like riak management, business continuity, etc.

Developers on Scrum Teams can suddenly resign or require long term leave at short notice to leave you in a similar situation.

At face value, I would identify the pair of Developers with the experience to best deal with the bug, take them out of the Scrum Team at the first opportunity (right now if it is urgent enough) and plan to continue without them for a reasonable number of Sprints (you may have metrics if this kind of thing has happened before!). The current Sprint may fail but probably doesn't meet the conditions for stopping the Sprint; that's for the PO to decide.

Yes, velocity will take a hit and your ability to forecast will be hampered. But I'd prefer this situation to one where the the bug and the bug hunters are kept "within Scrum" with an open-ended forecast as regards fixing the bug within a Sprint (when everyone expects the Sprint to fail).

Answer 2

This is not unusual in Scrum and a common approach is to use a spike.

The idea of a spike is to spend some time investigating an unknown with the intention of learning enough to be able to estimate it. Typically spikes are time-boxed so that they don't impact too much on the capacity estimates of the team.

It is important to remember that a spike does not need to solve everything. It just needs to do enough, such that the team has a lot more confidence in estimating. Once you are at this point the work can be treated like any other backlog item.

Answer 3

Unestimable Bugs are a Scrum Implementation Smell

Your Scrum implementation exhibits a number of anti-patterns that are likely contributing to your estimation problem. These include:

1. Treating tasks as user stories. "Find the bug" is a task, not a story. It doesn't meet INVEST criteria, and is not inherently estimable.
2. Estimating time instead of effort. You can't estimate how long something will take if you have no constraints on scope, complexity, or effort. A time box is not an estimate!
3. Automatically carrying stories and estimates forward. Each Sprint is a time box, and stories that can't be completed within a single iteration should always be reprioritized, re-estimated, and (most importantly) reformulated if the team learns that the story presents a problem in scope or delivery.

Typical solutions include, but are certainly not limited to:

1. Implementing TDD/BDD practices to reduce the frequency and complexity of released bugs.
2. Writing bug-related stories that are small and testable, e.g. "As a developer, I want to debug the Foo module so that I can learn if it's the source of bug #12345."
3. Creating a time-box for scoping or root cause analysis, which can then be used to help estimate the level of effort to find or fix an issue.
4. Using methodologies like the Mikado Method to identify scope and complexity, and to provide some rigor and interim deliverables to your bug-stomping process.
5. Re-evaluating "failed" stories at the end of each Sprint, and during Sprint Planning, to ensure the right level of effort, resource allocation, and business value is assigned to each deliverable.
6. Swarming over stories as a team when the stories are important, or when not completing them threatens the Sprint Goal.
7. Stop the line and refocus 100% of team effort on addressing critical issues, rather than attempting to continue development on top of breaking issues or foundational process problems.
8. Continually adjust your delivery schedule based on the real-world status of the product. Scrum estimates are there to help you plan iterations and milestones, and the framework will not magically solve scheduling issues, process issues, or "unestimable" bugs.

It's also worth noting that Scrum is primarily a development framework, rather than a product support methodology. While mature Scrum adoptions with strong agile practices can manage maintenance issues quite successfully, not all business problems are well-suited to time boxed iterations.

If the business will not invest the time, money, or training in implementing a more robust Scrum process, it might consider adding a second, Kanban-based support team—or even spinning up one-off processes outside the Scrum team—to work on bugs in your delivered products instead. I would consider that an anti-pattern as well, and don't recommend it in your case, but it's certainly an option that's available to you.

Answer 4

This doesn't answer your question, but I feel that your root problem might be that you need to have more robust testing -and in general better codebase to identify problems quickly-.

Ideally, this errors should be detected before deploying or at least once found, there should be a mechanism that would make it easy to identify what part of the code is causing the problem.

The big smell that I see to conclude the above is that you mentioned the bug would've taken 3 days instead of 3 months if the dev knew where to start.

Going to your original question. I think it's sensible to start documenting the occurrences of this situations, how they evolved, how they are being solved to try to implement a proper solution for future situations.