As a project manager how worthwhile is it to have 50/50 estimates of certain metrics?

Question

I was reading Steve Tockey's book Return on Software and he mentioned it's quite beneficial to have such estimates. Let's say I have an oracle that spews out 50/50 estimates of cost/schedule/budget/size (function-point, story point etc.,) for a software project. You are equally likely to overshoot as you are to undershoot the estimate.

The question is, is such an estimate really worth it? Why or why not? Intuitively it seems to make sense, but upper management would want more confidence for their planning - but you can only get that from widening your range and maybe from historical data! But that is only a 'sense of security' and maybe totally false, since you can't really say if you overshot it was 10% likely and hence it's justified since you really don't know what that 10% even means, to be honest (I've seen people claim only 10-15% risk and don't even know where the percentages came from or what do they signify).

I could increase my confidence by widening the range substantially but it doesn't give anyone much information.

So if I were to put forth a 50/50 estimate of things would my life be any better?? How would/should a PM manage for such estimates?

UPDATE: I would like to clarify a point: It's not that we are just using the 50/50 estimate and basing everything on it. You could say that is our 'mean estimate' i.e., based on past data of actuals vs. estimates. We then specify a range using n-standard deviations depending on our required confidence. So the point is that our 'initial estimate' is the 50/50 point (the mean) - we are equally likely to be under/over it (it's an estimate!). Now having such a point and the +/- range around it how worthwhile is this estimate?

Answer 1

Having a general rule of targeting at the 50th percentile of your probabilistic density is not helpful. First, the 50th percentile may not be the likeliest result. If your density is skewed, then your mode average is not equal to your median (50/50); therefore, targeting at the 50th will not be your highest probability.

Second, where you target within your probabilistic density is more a function of your risk appetite. If you were pursuing a firm fixed price contract, for example, targeting more to the right of the curve would be a smarter move so that you are providing contingency for uncertain events. On the other hand, if it is a project that you have performed many times before for this customer in a rather stable environment, moving to the left of the curve, i.e., a more aggressive and risky target, would be a reasonable business decision.

Answer 2

I've run into this type of dilemma more than a few times in my career and unfortunately it’s next to impossible to speak directly to a generalization like the 50/50 scenario you've articulated. I would say that assumptions; such as your 50/50; have little or no value. Instead, I prefer to try and embrace the reality that this is stuff we don’t know and stuff we do know. What we really want to do is give everyone the same picture (mental or otherwise). As the picture starts to crystalize amongst the larger group, we often find that it changes. This is because various stakeholders can; ideally; help each other fill in the gaps; reassure issues related to risks, etc…

For example... What were your assumptions (what’s known/unknown), things you’re worried about (risks), approaches you’re considering (patterns/designs), resources you feel you may need (people/things/money), etc… I like to use Planning Poker as one tool to shed light on things like this.

The more details you can shed light on openly (and early), the better your chances will be of having the larger group of stakeholders better understand what to expect, what to watch out for, and/or how to mitigate the bad stuff. In the end, that's really the essence of what any estimate process is trying to accomplish. Sure, there are budgets to work through, schedules to define, staffing issues, milestones to plan, etc... But those can only be accomplished 'after' the earlier bits are well underway.

I also written a few things on my blog about estimating which may provide some other ideas.

Answer 3

It depends on the goal for the estimate.

As David points out for a FFP contract or for defining your critical path you will likely want more confidence in your estimates.

On the other hand, 50/50 estimates are one of the premises of critical chain management of projects. If you assume all the tasks in a project are equally likely to be over/under the planned duration/cost they should balance out in the end. Tasks running over time/cost just consume the feeding and project buffers that you've put in.

Answer 4

Off the bat, it's not that useful. I haven't found it to be true that you're as likely to be under as over. More often than not (almost enough to make it a rule), you are going to be over.

Upper management wants you to deliver what you say you will deliver within the schedule and budget you said you would. Your best path is to go with your best estimate of what it will take, plus a large margin of error (erring on the side of needing more time and money) then making that your baseline schedule and budget.

Projects have their own life, regardless of your best plans and estimates. During project execution focus on keeping upper management in the loop on how things are going. Experienced managers know that plans change when the rubber meets the road.

Answer 5

The real question is do you have a documented track record (statistical basis) to believe the 50/50 estimates? In other words did 50% of the projects in your firm come in under budget and ahead of schedule last year?

If to the contrary you find that the actual outcome bear little if any relation to the estimates (of whatever kind) spewed out by your oracles, the first step is to inform them that you would like statistical evidence of the quality of their estimates prior to accepting them.

Take a look at Joel Spolsky's blog post on Evidence Based Scheduling to see how easy it is to use historical data to get much better estimates.

Answer 6

The whole issue is that you'll never know until the project is done if you were indeed on a 50/50 balance.

To solve this issue, I like to use project methodology with gates, where you refine the cost at each gate, thus reducing the uncertainty as you go.

I used to work with a method where the beginning of the project would split in inception phase, concept phase, then final conception phase, and only at the end of this phase PM were expected to give an accurate budget.

So to answer OP directly : I think this is useless, as you can't manage (ie take decisions) anything with that level of uncertainty, and your management will never accept it either. Iteration is the key here.

YMMV, so I'd be interrested in reading how this kind of estimate helped to take any kind of decision