Discerning estimate validity

Question

I've had experiences in the past where different developers will give different qualities of estimates--which then have to be consolidated into a single project plan--only with a twist.

• One developer will look at a task, figure it sounds like an easy problem, and estimate a couple of hours.
• Another developer will look at a task, figure it sounds easy, but recognize that there could be additional complexity or clarification along the way, so planning for contingencies, estimate a week.
• A third developer doesn't want to do the task and in an effort to discourage its inclusion (or his assignment to it), will estimate a month.

How does one go about discerning which estimates fall into which category? Do you just go by feel?

Answer 1

Planning Poker is one of the options and is a very easy way of giving an 'average' estimation of the task.

It is explained on Wikipedia and a lot of other Web sites.

Planning Poker is a consensus-based technique for estimating, mostly used to estimate effort or relative size of tasks in software development. It is a variation of the Wideband Delphi method. It is most commonly used in agile software development, in particular the Extreme Programming methodology.

What I found great advantages is that everybody that is included in this has his estimation, and the extremes, like the one in your question are automatically removed during short discussions among developers.

If an estimation should take a long time, or are done quickly, by doing planning poker you get an insight view thanks to the opinion of all developers in the room.

Answer 2

Actually any method which involves discussion between three estimators should do well. A few ideas you might find useful:

• Planning poker which is pointed by Kennethvr.
• Similar technique, which works for big number of tasks to be estimated at the same time, is magic estimation. Read more here: http://campey.blogspot.com/2010/09/magic-estimation.html
• If you have some historical data you may approach the problem trying to base on that. One of my favorite techniques here is Evidence Based Scheduling. See: http://www.joelonsoftware.com/items/2007/10/26.html
• If you look for something simple plain old discussion between estimators should bring you to results which aren't spread that wide.
• Another trick that may improve quality of estimates in general, and this situation specifically, is using short estimation scale. One idea can be T-shirt sizing (S, M, L). Another, and the one that is usually an instant hit, is: 1, 2, 3, TFB, NFC (where TFB stands for Too F*king Big and NFC for No F*king Clue). The latter usually introduces fun factor to estimating which makes the process smoother.

As a rule of thumb: if results differ that much it is a clear signal people understand the issue differently, so if you want to get reasonable estimate you should get people at the same page in the first place and only then focus on getting some number which makes sense.

Answer 3

How valid is any estimate? And what is the track record of each of the people providing previous estimates? Those are two big things to take into consideration. Given the productivity swings from developer to developer (I think DeMarco or Capers Jones said is was as large as 20 to 1 - dependent on ability/experience) it makes sense to have the varying ranges. You said about one finding an easy solution.

I think the best way is to get the person/team actually doing the work to provide the estimate, check their earlier estimates against delivery and have one senior developer check. It might require a few rounds to determine any major variations in specific sub-components.

Answer 4

Such an output from your estimators is a clear indicator of problems with the input. You should make one step back and refine your requirements. Then again request an estimate.

Answer 5

The title of this question 'Discerning estimate validity' raises the question of the validity of estimating full stop.

I recommend this post by David Anderson on the subject:

http://agilemanagement.net/index.php/Blog/a_taste_of_lessons_11_agile_practices/

Here is my personal experience. I have stopped using estimates for two reasons:

1. Variation in the estimates due to the 'non fungible' nature of developers - See Tom De Marco's Book 'Slack' for more on the myth of fungible resource.
2. Time taken to produce the estimates which could be spent doing more valuable work.

You can mitigate against the first point by using team estimating methods such as Planning Poker as suggested in the answer above but this this will still leave you with an average estimate. If the actual work is done by a developer whose capability to do the work falls below the estimate then the estimate will be misleading.

To the second point, lets say you average a throughput of 30 stories a week. Each story needs five minutes to estimate You have now spent two and a half hours of the whole team's time estimating.

If the estimates the team produced in this time could be demonstrated to be accurate and so provide a useful metric for steering the project then that would be time well spent. My experience has been that they are not.

Instead of estimating I prefer to take the Kanban approach which is to make probabalistic forecasts based on the historic throughput of the team.