How do you plan for risks in your project's estimates?

Question

I have been working on the risk-management topic in software development for a while. According to the different literature there could be knowns knowns (typical risks), known unknowns (product related risks), and unknown unknowns (black swans).

• known knowns

  You have some industry statistics and can use some simulations (e.g. RISKOLOGY) so you can add say 20% to scope and schedule.

• known unknowns

  Here you can brainstorm with the team and come up with the list of project related risks, and come up with preventive actions and estimate them, so these can be included into the scope. But what about the remainder of the known unknowns? Should you plan for contingencies? How much? 10%?

• black swans

  Just plan for contingency? How much? 20-30%?

How do you plan for these risks at the project initiation or planning phase, especially when customer asks for estimates? How do you usually present all these different types of risks to the customer, especially if the risks double the budget or schedule estimates?

Answer 1

Risks are typically classified into these two categories: Known Unknowns and Unknown Unknowns. You can break the latter down into Unknown Knowable Unknowns and Unknown Unknownable Unknowns. Known knowns are not risks since this implies certainty.

You have two types of drivers when it comes to your budget and schedule risks. The first driver is aleatory in nature, or random variables that will drive your results. The best visual is to establish a 3-point estimate--e.g., best case, $1,000; most likely $1,500; worst case, $2,800--and draw a triangular distribution connecting these three values. For simplicity, let's say you establish your budget at $1,600, $100 above the most likely in your estimate. This leaves the range from $1,601 to $2,800 as your unfavorable exposure and everything below $1,600 as your favorable.

Your second driver are more discreet type risks, or epistemic risks. These are specific threats that you can identify and calculate the likelihood and impact. Because of the specificity of the threat, you can calculate the mitigating costs as well as what value you want to propose for contingency, maybe based on expected monetary value.

Your last question is not easy to answer. This depends on how you are contracted with your customer. If you are going in with risk, i.e., a fixed price contract, your customer does not need to know what you built into your proposal for risk. None of his/her business. If T&M or cost plus, then it becomes a joint discussion about your threat assessments, the costs you calculated, and a joint decision about what to put into reserves. Your target values is what you put under contract. The reserves are not in contract but held by the customer to use in case bad things happen.

Either way, the presentation of your proposal is not easy. Customers want to hear a guaranteed success at near zero cost. That's your challenge as a seller of services.

Answer 2

TL;DR

Risks can be accepted, controlled, or transferred. Your project plan should certainly include risk analysis and recommended controls, but the risks properly belong to the project stakeholders. As the project manager, you should focus on documenting the risks and providing reasonable project controls for those risks that aren't accepted or transferred.

Targets vs. Estimates

Every project has a target. In most cases, the management targets for projects are unrealistic. For example, a project that is estimated with best-case scheduling and no slack in the process is unfortunately common, and will represent your best-case scenario for the project in question.

An estimate, on the other hand, measures what you think you know and then factors in your cone of uncertainty and other risk analysis figures to arrive at a more realistic estimate. The more you know about the problem domain, and the more you know about the risks you may face, the more accurate your estimates will be.

In general, you may find it helpful to offer your estimates as a range rather than a set of fixed numbers. The size of the range inherently implies uncertainty, which is really the point: you're making an educated guess based on a set of assumptions.

Fudge Factors

An alternate method is to apply a "fudge factor" to your estimates. This can be based on historical measures (e.g. your projects typically come in +/- 20% of the realistic estimate) or on risk tolerance (e.g. your project can tolerate schedule slippage <= 10%).

I'm a big proponent of transparency in planning, so if you go this route I'd certainly make a point of acknowledging that this figure is a fudge factor, rather than the result of a quantitative analysis. In Scrum, I typically apply a fudge factor of 0.6 for new teams, and 0.8 to established teams, but your mileage may definitely vary.

Explicitly Document Assumptions

Risk is never zero, and estimates are not guarantees. In general, I've found that the best way to handle this reality is to simply document them as part of your project assumptions.

For example, your project documentation may include a formal or informal risk analysis, as well as a target schedule or set of estimates. By documenting the assumptions that underpin your project plan, you improve communications and provide for contingencies at the same time.

Consider an imaginary project where you assume:

• a schedule variance of plus/minus ten percent,
• a budget variance of plus/minus ten percent, and
• a supply chain that will not deviate outside of your schedule or budget variance.

If any one of these assumptions turns out to be false at some point during the project, then stakeholders can re-evaluate. The organization may decide to take any number of actions if the project is out of tolerance, including changing the plan or canceling the project.

Your Role: Not a Guarantor

If you're selling fixed-price services, then you need to bake that risk into your estimates, too. Sometimes your organization comes out ahead, and sometimes it doesn't. If your organization can't tolerate that risk, then don't do fixed-price contracts.

Remember that your role as a project manager is primarily to apply reasonable controls to the project, and to routinely communicate the status of the project to stakeholders. Politically, project managers may get blamed if a project slides off the rails, but try to keep in mind that your role is to track the project and apply project controls. You can't actually guarantee an end result, nor should you try.

If you want to sell customers a pig in a poke, switch to the sales department. Otherwise, use your position to educate stakeholders about the scope and assumptions contained in your project plan, and be prepared to explain how you came up with the planning numbers that management and clients must approve to move the project forward.