How to plan for bugs, looking for advice

Question

I work in a software development company, where we use something which resembles scrum.

• We have sprints of 4 weeks
• We have 6 sprints / release
• We plan a development per sprint, with an estimate
• We have a debug factor, this is the time per sprint allocated for debugging (varies per developer)
• We have maximum levels for our bugs

We run into the issue that when we are working on our developments, and the bugs start coming, we need to drop our development work and focus on debugging. When the maximum levels are reached, we don't release our software anymore and we need to first fix bugs before we make new builds for our customers.

This all seems perfectly reasonable, since we only release every half year and we have customers waiting for the bugs to be fixed.

However, this creates the situation where we sometimes don't find the time to work on our developments, because we have to keep debugging for weeks sometimes.

How can we plan correctly for bugs? A couple of things I thought about, but I'm curious what your opinion is

• Plan bugs as regular developments, with an estimate. Don't give them priority
• If a developer spends it's debugging budget for a sprint, don't debug anymore that sprint and continue work on the development.
• Other options?

Answer 1

There are several ways of tackling this, I've tried most of them and none are ideal simply because you are dealing with what Scrum calls "unplanned work".

One of the most important things, in my experience, is having a good bug triage system in place. When a bug is reported, someone should make an informed decision, based on a company-wide-known policy, what the severity of the bug is and if it needs to be addressed immediately or can be put off until the next Sprint. Things you want to consider here are rate of reproduction, percentage of user base affected, risk of data corruption, etc.

Once you know what the severity of your bug is you can decide what to do with it and how to handle it. If the severity is high enough to pull it into the Sprint, you handle it as unplanned work with the highest priority in the Sprint Backlog.

How you handle unplanned work is something completely different; there are several ways of doing it. Below are the two techniques I've used in the past; both work equally well but will not suit all Teams.

• Use a "fast track lane". Any bugs that pop up in the Sprint Backlog that need immediate attention get put up on the "fast track" and need to be picked up before any other new work is started. The fast track is a separate priority "swimlane" or "expedite" lane that goes above all other swimlanes.

• Use a "swat team" if you Team is big enough. Introduce a rotation system in the Team, where every Sprint one or two people are the "swat team" that will immediately jump on any unplanned work that pops up on the Sprint. The lower availability of these developers should be accounted for in the Team velocity for the Sprint. Generally you want the swat team working on lower priority Sprint stories since they could be pulled off the tasks at any time. This obviously only works in large enough Teams.

Answer 2

we use something which resembles scrum

My suggestion to you would be to move closer to Scrum by looking to have a potentially shippable increment at the end of each sprint.

Note that the key word for you here is 'potentially'. You can still release every six months if that is what your Product Owner wants to do, but you should still target having something ready to ship every sprint.

To do this you will need to think of features only being done when they are bug-free. The work that you take in to the each sprint will be the capacity your team has to develop new work and correct any bugs that are raised by the new development.

Working this way will make it easier to plan, as your team will become aware of their real capacity to deliver work.

There is also a lot of merit in reducing the chances of bugs happening in the first place, by using engineering techniques like automated regression testing, continuous integration and behaviour driven development (BDD).

Answer 3

There are several ways that you can deal with bugs. But regardless of your bug policy, there is one thing that you really should do.

If your developers are spending a significant amount of their time on fixing bugs, you should give it a high priority to find out where those bugs are coming from and what can be done (on a technical level or on a process level) to prevent those kinds of bugs in the future.

For example, if your customers only see the new features when you release them and then complain that those features don't work how they expected it, then you should look at ways that you can involve those customers earlier in the validation of the new features.

As for bug policies, the two major ones that I know of are

1. "Zero bugs": Any bug is one too many and it should be solved with priority over picking up new development. This requires that you have a robust classification process to avoid that changes to correctly working functionality are classified as bugs. If there are many bugs reported, then this can mean that no new development is done for a significant time.

2. Prioritize bugs along with new development: For each work item the priority in relation to the other work is determined, regardless of if the work item is a bug or new development. During planning, the top N items are taken into the sprint, where N depends on the capacity of the team. At least for planning purposes, it is useful to have an indication how much effort would go into solving each bug.

Personally, I like the second approach, because it gives the Product Owner the possibility to decide on a case-by-case basis if that highly demanded new feature is more important than fixing that bug that hardly anyone notices.

Answer 4

While you are resolving the problem of how to allocate time for bugs, you may also want to look at ways of reducing the number of bugs you're finding. You may be able to catch them earlier in the process, leading to less rework later on.

There are a number of methods/tools to help do this - peer programming; code reviews; static code analysis tools; automated testing; ensuring user stories have good acceptance criteria agreed between Product Owner, Tester and Developer before work begins; close communication between testers and developers early in the development process.

Best of luck

Answer 5

The main problem I see is that you are releasing each 6 months, which is too rare. There's a reason why Scrum says a maximum of 4 weeks per Sprint. Because at the end of the 4 weeks, the increment needs to be potentially releasable. You don't have to do it, but it has to be in this state.

So, because the releasing cycle is too long, your customers raise their issues as bugs, which you are forced to fix before the new release. You can't convince them to wait for the release because it takes 6 months, which is clearly too long.

What you should do, as a company?

You should choose a Sprint length that your customers will accept to have the bugs fixed. Obviously there will be some critical bugs which need to be solved ASAP. Allocate to those 10% of the Sprint. Keep in mind, it's a team metric.

The rest of the bugs go in the Product Backlog and get prioritized along with the features you are developing.

Let's assume you do this and you start a new Sprint with some features and bugs in it. Now you have three types of bugs:

1. The bugs you've selected in the Sprint Backlog. Just fix them.
2. The bugs that appear during QA of the features you have selected in the Sprint Backlog. You need to fix them in the same Sprint to deliver your feature.
3. Critical bugs not in your Sprint Backlog. Use the 10% allocated time to fix these when they appear (eg. production bugs).

At the end of the Sprint you deliver a new version which has:

1. Some bugs fixed.
2. Some features developed (and the bugs found during QA already fixed)
3. Critical bugs fixed with priority. (10% of your time).

The customer can accept this because it's reasonable to wait 2-4 weeks for some minor bugs, while the critical bugs get fixed ASAP.

Now, if 10% of the Sprint length is not enough, then you have Technical Debt. You may increase to 15% or maximum 20%.

You need to anticipate these bugs. You need to refactor or redo some of your code if you know it has critical bugs that your customer will discover. Create some items in the Product Backlog to anticipate these bugs and talk your Product Owner into giving them high priority. Anticipate your customers' bugs and you'll have a better Sprint Planning and time management.

Lastly, you may have a customer which claims every bug is critical. When you receive it, have a look at it. If it's not critical (eg. some button in the UI which never worked), ask the Product Owner to discuss with the customer or do it yourself if you can deliver the message.

When assessing priority/criticity put yourself in the customer's shoes. It may be a small technical bug, but if the customer is losing money on it, then it has high priority. How high? Ask the Product Owner.

If you allocate 10% of your time to fix critical bugs and actually use more of it, your Product Owner should decide which of the Sprint Backlog items should not be delivered. It's simple: you put something in, take something out.