How do you streamline the budget process in Agile?

Question

When you are doing Agile, I heard you need to also budget in an Agile way. Apparently you need to move away from project-based budget to a streamlined budget process?

How?

There seems to be two ways.

1. One is to fund teams instead. Which makes sense. You pay for the team that does the work and then they work from the top of the backlog until you finish the entire thing or stop when it no longer makes sense to keep incrementing the product. I don't understand how this is any different than project-based budgeting?!
2. Again, instead of funding projects, you fund value streams instead. This confuses me. Isn't this the same as point 1? The value stream won't build itself, you need a team, thus you actually fund the team when you fund the value stream.

Can someone help me with my confusion?

How exactly do you move from project-based budgeting to Agile budgeting?

Answer 1

The goal in agile is to deliver incremental value. When you achieve this goal (and measure the value delivered) your budgeting can be much simpler:

We know how much value this team is delivering.

We know how much this team costs.

Is the team delivering sufficient value to warrant their cost?

In some organisations it can be a challenge to measure the value delivered by an individual team. In those situations it might be easier to measure the value delivered by a value stream, i.e. a collection of teams.

As you hint at in your question, this isn't ideal. The preference would be to measure value at the team level.

For me the goal of agile budgeting is to be able to produce statements like the following:

Alpha team costs us £40k per sprint. In the last two sprints they delivered a new search feature and a sort feature. These new features have driven up sales and reduced churn. I think we can say with confidence that the value they delivered exceeds their cost.

Answer 2

TL;DR

Most agile frameworks don't really address budgeting directly. Like most other aspects of such frameworks, budgeting is usually an emergent property of a given framework's implementation. That said, estimating an agile budget is often most effective when each outcome-based milestone defined in an agile release plan is budgeted based on a fixed, per-iteration cost multiplied by the expected number of iterations needed to reach the milestone. Then sum all currently-planned milestone costs for a rough whole-plan estimate.

Next, ensure sufficient slack between milestones, and apply any necessary fudge factors for milestones further in the future that are likely to change in scope or may involve either known-unknowns or unknown-unknowns. This is most easily represented as a factor of your iteration-based budget, and often increases with project length. Factors of 1.2 to 4.0, or 1-2 standard deviations from the mean of the schedule range, are pretty typical values. However, the actual values are often heavily influenced by organizational politics, sales/market pressure, and the optimistic or pessimistic bias in the planning process. (Pro tip: pessimistic plans fail less often.)

Since scope is usually the variable constraint in agile frameworks and methodologies, you should try to tie the budgeting process to one or more outcomes rather than a fixed scope of work. Well-run agile projects can forecast budget and schedule within a reasonable confidence interval so long as scope (i.e. how the outcomes are achieved) remains flexible.

How to Visualize Budgeting for Business Agility

In most agile frameworks, budgeting is essentially the intersection of two things:

1. An agile release plan. (See Also: more related answers on PMSE)
2. Your per-iteration run cost for the team or project.

Here is a Venn diagram that may help with visualizing this:

Additional Considerations

Your agile release plan can and should change throughout the lifetime of the project, although there should be some relatively stable objective to achieve. In other words, even when implementing an agile framework, a project is never expected to be open-ended. However, unlike more traditional methodologies that require "big, upfront planning" (BUFD), within more agile frameworks the "what" and "how" can evolve over the life of the project enabling pivots, managing opportunity costs, and reducing sunk costs.

In a properly-implemented iterative framework, after some initial kick-start period you should always have a working product (or at least a working product increment) at the end of each iteration regardless of whether or not the project is completely "done." That is what allows such projects to take advantage of emergent design and just-in-time planning, extract earned value by shipping viable business increments at almost any time, and succeeding or failing early whenever the product is deemed "good enough" or the project's goals can't be achieved without chasing sunk costs.

Agile frameworks like Scrum rely on scope being the flexible project constraint. If you lock scope, then either schedule or cost will need to become more flexible. A fixed scope makes realistic budgeting extremely difficult since cost and schedule are tightly intertwined in such frameworks, so think in terms of a set of acceptable outcomes rather than "scope" (especially in the sense of highly detailed or predefined design specifications) during project initiation.

Successful Outcomes Don't Always Mean Delivering a Successful Product

Also, try to remember that no project plan or budgeting process can provide an actual guarantee of success. Budgeting, like the rest of project management, provides an estimate rather than guaranteed accuracy. Be sure to clearly communicate that the budget is an estimate based on a set of assumptions. Even a project that is roughly identical to a previous project is subject to force majeure, changes in the market, and other unforeseen or unforeseeable circumstances.

Even a literal money-back guarantee only provides a financial stop-loss; it can't actually guarantee the success of a given project as originally planned. That's why agile frameworks focus on adaptability and meaningful outcomes. Historically, 68% of IT projects fail. From that perspective, even failing at a responsible moment can be viewed as a positive outcome. Agilists often call this "failing early," even though it's more accurate to describe it as successful cost savings rather than failure.

Failing early is technically a success if:

1. The project has avoided prospective costs already budgeted for the future when the project can't achieve the desirable outcome. All money saved by ending the project early is thus converted to cost-savings.
2. The project doesn't chase sunk costs by adding more budget to the costs already incurred, although not all cost overruns are necessarily a sunk-cost fallacy.

Not wasting budget on an outcome that can't be achieved, with or without changes to scope or schedule, is actually a positive outcome. Changing scope or schedule, or even increasing the budget once the cone of uncertainty has narrowed sufficiently, might make sense if done mindfully and if the changes provides sufficient confidence that the project could then meet an updated definition of success. However, this is called the sunk-cost fallacy for a reason. The cognitive biases that drive the continuation of existing projects beyond any reasonable utility and lead to significant cost overruns that don't provide sufficient return on investment are commonplace.

Apply Agile Principles of Transparency and Flexibility to Budget Estimates

Put another way, just be sure your budget is provided as an estimate, not something written in stone. Don't treat budgeting any differently than the other estimation processes you apply to your project. Whether or not you're using an agile methodology is actually a secondary consideration here. The primary consideration is that a budget estimate is an estimate, and the role of project management is to ensure the most positive outcome possible within the original planning values, and to work with the right people to modify the plan when actual values deviate beyond acceptable levels (however your project has defined "acceptable") from previous planning values.

This is just as true in agile frameworks as any other type of framework. The only functional difference is that applied agile principles make this process more flexible and transparent to everyone involved.

Answer 3

With projects you come from the mindset of, I have this much work, how much will it cost. This means you can only come up with a total cost, once you actually know how much work it will take, and in a complex environment, you can usually only know that for certain at the end. You may attempt to make predictions, but these predictions will have large uncertainty, causing decision making based on those estimates to be risky.

With teams/value streams you are approaching from 'I am willing to spend this much on it in a given timeframe, what can I get for it?'. This allows you to know the total cost upfront, even when there is a large uncertainty about the size of the work. Instead you find out over time what in fact you are getting for it as you learn about the product.

If you expect to continuously be spending on some improvement, this allows you to control your rate of expenditure, and ensure it is prioritized to appropriate returns, without needing to know upfront how much any given piece of functionality will take.

Answer 4

Although Todd A. Jacobs gives one (very good) perspective, there is another approach.

If you have a (relatively) fixed cost per unit of time, considering the cost of the team and other resources (such as facilities, infrastructure, and tools), you can budget in bets. Instead of fixed-length iterations, the team can define small (weeks, perhaps a couple of months) goals or steps and estimate how long it would take to achieve that goal or make that next step. The bet can be funded if the perceived value is less than the cost. If the bets are too expensive for the value obtained by completing them, the work doesn't have to be completed.

Depending on the situation, the bet could range from a proof-of-concept to an incremental improvement to the system. When building a new or novel technology, the bets could prove the concepts on which the rest of the system will be built. The bets could be incremental improvements when evolving an existing product or system.

The use of the term "bet" communicates that there is a level of risk for the funder. The team works to estimate the time and effort needed to implement the goal, but that estimate could be wrong. The value is often also an estimate, and the estimated value may not be achieved. All stakeholders are making their best guesses based on the limited information that they have to decide if and how to proceed with the work.