How should I deal with common dependencies when estimating stories in Scrum?

Question

There's something that has always bothered me with Scrum. Hopefully I can get some insight here.

With Scrum, we try to break the backlog down in vertical slices. Let's say stories A and B. These stories are supposed to be customer oriented and deliver value. Assume here that they have similar customer value.

Now, let's say that, to implement, A and B both need a shared infrastructure. So let's say we're at backlog refinement. If I ask the dev team to estimate in story points, they might estimate them each at 8 points, if they are estimated separately: A8, B8. This is good for me as PO because I can move the stories around in the backlog without breaking anything.

However, when forecasting, my backlog will appear heavier, because, if combined, they might be delivered with a lesser effort than the sum of the parts. If my team has a hypothetical velocity of 13, it will look like they won't be able to deliver in 1 sprint. If there are many items like this, it will be impossible to have any visibility of a timeline beyond the planned sprint.

If we have multiple stories with a same shared infrastructure, the backlog will seem very heavy, and forecast will be impossible.

I have seen 2 variants:

1. Split the infra work to a separate "technical" story: A3, B3, T5 This solves the estimation process, but it's a horizontal slice. The infra itself has no customer value. Also, the backlog now contains dependencies, because the stories are no longer atomic, which means that I can't reorder them without being aware of the dependencies. Without looking at the specifics, I would have chosen A and B to be done first, because they are low effort and high value. So it's going back to classic project planning.

2. Estimate the infra effort in one of the stories only, and estimate the second one as if the first one is already done: A8, B3. The downside here is that one story will seem more complex than the other, so if I don't look at the details, I will be inclined to prioritize story B, because it will have the same customer value as A, with less effort.

So the questions are:

• Are there any documented ways to handle these?
• Have you found ways to deal with situations like this that don't require you to pick between the downsides mentioned here?

I have dealt with these a bit with an ad-hoc way, by using one of the two options above and by marking dependencies, so that I don't put prerequisite tasks before subsequent ones. If I do, I redistribute the story points. But it looks bad for anyone else looking at the backlog, and I prefer the backlog to be a useful and transparent artifact.

Answer 1

(Too large for a comment)
I disagree with two of your remarks under point 1:

• "The infra itself has no customer value." So, get rid of the rigid requirement that "These stories are supposed to be customer oriented and deliver value.", change it to "These stories are supposed to deliver value".
• "The backlog now contains dependencies". There's software to record that; this will prevent you from picking up stories with dependencies. If they are clearly indicated you can pick up T5 in an earlier sprint. You don't have to cram it all into one, I assume there are more things you divide over sprints.

It sounds like you're blocking "let's design a system that works" ^(*) with theoretical "should be's". Why do you want to hold onto your vertical slices?

So yes, split into A3, B3, T5 and see how this works.
When you have done this a couple of times, evaluate it. Your biggest 'risk' is doing T5 and then later deciding that A3 and B3 are not necessary.

^{(*) Not 'true', since you take the effort to ask here}

Answer 2

This problem isn't unique to Scrum. It's quite common across the Agile methods, which call for frequent delivery of valuable, working software to customers and users. I've come across two ways to handle this situation, both of which have worked out well.

The first way would be to define the infrastructure that is needed as part of both A and B. Then, estimate both A and B as if the infrastructure would be created and tested as part of both. The second approach is to split out the infrastructure into a dependent item that must be completed before either A or B is completed, ensuring that the sizing for A and B is based on the assumption that the infrastructure is in place.

I would avoid the proposed solution of estimating A and B in a way that requires one to be done before the other. This adds an unnecessary and potentially unclear constraint to the way that the team must work. If the wrong one is started first, it will be significantly underestimated.

I have to agree with Jan Doggen's point that defining each Product Backlog Item in terms of direct customer value isn't always the best way to approach it. Some work - such as work to design, build, and validate shared infrastructure - doesn't have direct value to a stakeholder, but removing this shared dependency opens up opportunities for the development team to deliver any number of work that is directly valuable to stakeholders while maintaining the integrity of the system.

I like to think of each Product Backlog Item as an independently designable, implementable, testable, and deliverable entity. You can design the infrastructure, stand it up, perform testing on it, and put it into production. This gives you an opportunity to account for infrastructure configuration, performance testing and configuration tuning, security testing. Having the infrastructure already in place before deploying the software that uses it often eases the pressure of deployment. Considering all of this, there is value in ensuring that your infrastructure choices are correct to meet non-functional requirements prior to delivering software that uses it.

The choice between a technical dependency in your Product Backlog and including the work in both Product Backlog Items depends more on your context than anything else. Things like the number of teams supporting the product, technical savvy of the product manager, the tooling used to track your Product Backlog, the time delay between identifying this work and starting this work, and the pressure on the whole team all would affect my decision one way or the other.

I don't think that concerns about the heaviness of the Product Backlog should weigh into the decision at all. After all, you aren't making a firm schedule and committing to it. You're forecasting the amount of work. If you're estimating, all of your estimates have noise. If you happen to have overestimated the amount of work, you can always use that time effectively, by either taking some additional time for Product Backlog Refinement, skill-building, paying down technical debt, or starting some other work early.

Answer 3

This is a common dilemma. The team should decide how they want to estimate in each case but it can be perfectly OK to split the difference and give each story the same estimate even though you know one of them will take longer. Treating them as equal means you don't need to track any hard dependency and although you may have a slightly lower velocity in one sprint you'll see an uptick in subsequent sprints when you complete the other items. This is reasonable enough, especially since such situations tend to occur near the beginning of a new piece of work where you might expect velocity to be lower.

Velocity and points are not the only things - or even the main things - that teams use to decide what can go into a sprint. During Sprint Planning the team need to gain a decent understanding of what has to be done. At the planning meeting they should be well aware that a story needs some infrastructure which isn't yet in place; they can take that fact into consideration when deciding what items can fit into the sprint.

Answer 4

This is one of the reasons why doing your estimating and planning close to when you start the work is valuable.

The conversation can go something like:

"We know we have that bit of infrastructure work, shall we add it to Story X as that looks like being the first one we work on in the next sprint? If we change our minds about the order of the stories, we can always switch it to another story and re-estimate during planning."

To answer your question: The best approach is to be aware of the potential problem and incorporate it into your planning and refinement sessions.

Answer 5

TL;DR

Don't conflate "end-user value" with "product value." They are neither fungible nor orthogonal. End-user value is simply a much smaller subset of the overall value that a Product must deliver, and over-constraining your Product Backlog based on end-user value is a very whiffy set of project- and product-management smells.

You are Misusing the "Value" Metric

You are laboring under a common miapprehension about Scrum. The current Scrum guide has some general advice (but little to say that's prescriptive) about the way the Product Backlog is ordered, except to say that it should be ordered according to product-specific criteria.

Nowhere in the Scrum Guide does it manadate that "value" be customer-facing value. Even the section on the Product Backlog artifact simply says:

Product Backlog refinement...is an ongoing activity to add details, such as a description, order, and size. Attributes often vary with the domain of work.

While it also says that a product is a vehicle for delivering value, and other sections of the guide speak about the Increment (e.g. the cohesive Sprint Goal) delivering value, nowhere does it mandate that the value be solely for the customer, or define what SAFe calls "architectural runway" as lacking in value.

Use Value as a Multi-Valued or Aggregate Metric for Prioritization

Rather than trying to shoehorn all your Product Backlog items into an artificial definition of value, you should be treating value as an aggreate metric, or at the very least the result of a set of sliders, to determine whether items on the backlog add value to the product you're building.

Infrastructure work that's a prerequisite to building a product increment clearly adds value to the product, since you can't build the increment without it. Likewise, features that add value for stakeholders rather than customers have value, too. As a random example, building a feature in a way that your legal department can patent has zero have for end users, but may have value for the company. Excluding that sort of value-oriented thinking from your Product Backlog is most definitely an anti-pattern.

Value as a Proxy Resource Allocation

Think of "value" as a holistic metric, but also think of it as a way to identify resource priorities. All projects have constraints, including time, budget, scope, quality, and people. By prioritizing activities that deliver value, however you define that, inherently directs a project's limited resources towards specific Product Goals.

In the abstract, perhaps a company would make more money in the long run by embiggening a widget. On the other hand, perhaps the company can realize more short-term value with its limited resources by ensmallening a therblig. Which is a better use of the project's limited resources? The aggregated-value metric embodied by the Product Backlog answers that question for the Scrum Team, that particular Product, and the business objectives of the stakeholders.

There's no single answers to this sort of question, which is why Scrum is not prescriptive about it. Instead, the framework provides some guidance, stringly implies that stakeholders should be consulted, and then explicitly says that the final decision is solely the province of the Product Owner. In short, it's up to the Product Owner to define how "value" should be applied within a given project, with the intent of maximizing product value and indirectly directing the project's resource allocations.

Answer 6

"What good are all those concrete blocks at the base of the walls of my house? They're sitting directly on the dirt, they aren't pretty, and I never look at them ... of what 'value' are they to me?"

You don't think about these things until cracks start showing up in the wallboard ... then you've got a pretty-expensive problem to deal with.

I recommend separating out these shared, "infrastructure" tasks as a separate story. Then, by some means that will be clear to whoever might be looking at the metrics, link these stories to all other stories which depend on them, and also make it clear how they depend on them. Is this a prerequisite? A corequisite? Is it shared-but-user-visible functionality?

Documentation of these dependencies is extremely important: you need to be sure that you identify every such dependency. In the text of the "shared story," list every story that share it and briefly explain why. Likewise, in the text of the "sharing stories," list everything that is shared. You need to be very careful about this so that the reader of every story fully understands the total picture – especially when it comes time to develop the "shared story" into an implementation plan.

Finally, it might be the case that the shared material should anticipate future needs. You should very carefully and deliberately explore that in the description of the shared story(ies). Consider whether this anticipated functionality ought to be built-out now or merely "stubbed."

In my view, a good "story" should accurately describe an upcoming unit of work from a meaningful point-of-view, so that an objective reader could have a reasonable chance of scoping it and planning it based only on what the story contains. And, as noted, it should explicitly call-out every dependency or potential side-chain impact.