Is backlog refinement (grooming) waste?

Question

So, I have seen it happening, especially with online tools: The backlog grows till you don't see the forest for trees, important features drowns in a sea of issues, and duplicates appear because it is too heard to keep track of what is in the backlog.

Backlog grooming refinement to the rescue, and it seems to be an undisputed best practice. But I see it more as a cure for symptoms rather than cause. Beside, while time spent grooming refinement the backlog could save time elsewhere, the time is still not a value adding activity. Wouldn't it be a better approach to prevent the grooming refinement? That is, are there practices to follow that reduce the need to spend time on grooming refinement?

(I think I read something on this subject in a blog post or article. Can't find it though, so if anybody knows please tell.)

Answer 1

TL;DR:

I agree with you that, once you've got a grip on your backlog, that grooming is wasteful. Spending time breaking down requests that will never be implemented is waste and violates the spirit of "Maximizing the work not done." Delay breaking things down until the last practical moment (but no longer!) and you'll need to do very little grooming.

However, you have to get control of your backlog before you can get to a point in your journey where that's feasible.

---

The best thing to do is not create a user story until you're relatively certain that it's going to be implemented at some point in the near future.

First, I'll share my vision for my team, then the reality of where we started and how to got to our current stage of our journey.

Ideal:

• Epic/Feature/Story structure to the product backlog.
• Work one Epic per team at a time.
• Cap the number of Epics. Once you hit that cap, PO must remove a low priority Epic from the product backlog. (It's okay, if it becomes important later, it can be re-added. Truly important things will come back up naturally.)
• Epics are not broken down into Features until they're the next priority and the current Epic is nearly finished.
• Features are not broken down into Stories until they're the next priority.

This delays work and decisions as long as possible, while leaning out the grooming process. You can't groom things that don't exist and any grooming that does need to occur, can happen during the prioritization process. That feature isn't as important as you used to think? Scrap it. No big deal. We've no sunk cost in it yet.

Capping the number of Epics/Features in the backlog keeps it from ever growing large to begin with. It also highlights how limited a resource your developers really are and facilitates the prioritization process. It really forces the PO to think about what's really, actually important. You can't have every feature you come up with on a whim. We need to focus our efforts on what brings us and our customers the biggest value.

But like I said, that's an ideal my team hasn't actualized quite yet.

Where we started:

Huge, unorganized, unprioritized, garbage heap of every thing anyone ever asked for. All of it in a few hundred stickies stuck to my boss's office wall. I shit you not.

The steps we took:

Lots and lots and lots of backlog grooming.

1. Moved the mountain of stickies out of our manager's office and into a public space.
2. Did a rough grouping by area of the business. (Marketing, Quality, Shipping, etc.)
3. Determined who could answer questions and make decisions about each area. (Established product owners.)
4. Attempted to discover what each sticky was about at a high level. If no one could answer "What is this?", the sticky was thrown away. (Literally, crumpled and thrown away.) Again, don't panic, if it's actually important, someone will ask about it again.
5. The team made a "best guess" prioritization.
6. Individually, held initial prioritization meetings with each PO.
7. Made the prioritization meetings recurring.
8. Started entering new requests as stories in our electronic tracker (TFS, Jira, whatever you've got). They also got a sticky on the backlog with an issue number written on them.
9. Determined which items were bugs and tried to reproduce them. If they could be repro'd, they were entered into electronic system and its number written on the sticky. If it couldn't be reproduced, we threw the sticky away.
10. Started putting effort into entering remaining stories into electronic tracker. Numbers, again, written on stickies as they were entered. After several months of a couple hours a week, everything was digitized, but the physical board remained. Again, if no one could answer "What is this?", it was trashed.
11. Identified all current projects and entered them as Epics. Attached related stories to them.
12. Identified upcoming projects and entered them as Epics. Broke down top priority Epics (top 3) into Features only.
13. Started grouping loose stories into Features and Epics, starting with currently WIP Epics. This did involve a handful of placeholders for "miscellaneous, stories that aren't really part of a project" Epics & Features. Stories were "right sized" and turned into Epics/Features as the team saw fit during this process.
14. Discontinued use of physical board.
15. Team works with PO to break down current Epics into Features and Stories. POs prioritize Epics & Features. Team prioritizes and pulls stories within the top priority feature as they see fit.
16. Desperately try to convince POs that if an Epic is consistently their lowest priority, there's no sense in having it on the backlog, because it will never get done. It's a manufacturing company, so we've had some success explaining that a backlog is inventory and inventory is waste. Not a lot of success, but some.

The process I described has taken us 9 months. We've still a way to go, but we have been able to reduce the backlog's size and convince all of our POs that there is absolutely zero sense in breaking down and estimating Epics (even Features) that we may never actually implement. Added bonus: there's is infinitely better visibility for both the team and our stakeholders about how much work is in the queue, being done, and has been done.

Next Steps:

1. Convince POs that it's okay to remove things from the backlog that we'll never actually do.
2. Put a hard cap on the number of Epics & Features in the backlog (per area).
3. ???
4. Profit!

Answer 2

The backlog growing extremely large and overflowing like that is unfortunately inevitable in a lot of organizations, especially ones such as mine where they tried to turn a multi billion dollar company IT shop from a free for all into an Agile shop.

I agree with the premise Grooming is a not a value added task, and is much like deferring a payment until a later date - you are only pushing off the inevitable. There really isn't a "best practice" I hate that term, it's very HR/Business-centric. There are some solutions which may or may not work for you.

The only control measure is the WIP Limits from Kanban frameworks, setting a hard cap on work you could have in the Backlog can help your teams from overcommitting and having a giant backlog in place.

A lot of that will have to do with who is setting these projects, in my current organization we have 3 stacked levels of Business (who want to be IT) above us that shove Requests down to us, with no PMO or Delivery Team buffer in between us.

If your organization is also open to it, consider having a PM advisor getting attached to the Business (Governance/Practice, whatever it is called) to get them to stop burying your Teams, and make sure your Teams do not over commit to work that needs to be done.

Answer 3

Backlog grooming (or backlog refinement as it tends to get called these days) is the process of making sure the backlog is in a good enough shape to take into a Sprint Planning meeting.

It came about because teams realised that leaving everything to the planning meetings was a dangerous approach. By doing backlog refinement in the previous sprint you allowed yourself time to fix issues with the backlog before you entered in to a planning meeting.

A team might look at the top 5-6 stories on the backlog in the backlog refinement meeting. They will talk through them with the Product Owner and they may even estimate if they fully understand a story. But there will be times when the discussion on a story reveals a gap in knowledge. For example, say the development team looks at a story and explain the technical implications of doing it to the Product Owner. The Product Owner then realises that the story is more complicated than they imagined and so decides to break it down further.

They don't have to do this break-down during the backlog refinement session. They might take it away and fix the stories in the days between backlog refinement and Sprint Planning.

This is how it is described in the Scrum Guide:

Product Backlog refinement is the act of adding detail, estimates, and order to items in the Product Backlog. This is an ongoing process in which the Product Owner and the Development Team collaborate on the details of Product Backlog items. During Product Backlog refinement, items are reviewed and revised. The Scrum Team decides how and when refinement is done. Refinement usually consumes no more than 10% of the capacity of the Development Team. However, Product Backlog items can be updated at any time by the Product Owner or at the Product Owner’s discretion.

Note the cap of 10% of the team's time spent on refinement. This is to ensure that teams don't get sucked in to excessive fiddling with the backlog. They should do just enough refinement to make their Sprint Planning meetings go smoothly.