As described here planning poker is a group estimating process. It seems that it would work.But is there any data to show that it is better than individual estimation?
Edit 4/19/11: I am interested accuracy in terms of time-planning. I am interested in planning poker in particular, but will look at the Wideband Delphi references since they are better than nothing.
Better at what? Do you mean better accuracy in terms of time-planning? Better at understanding your product? Or something else?
Do not overlook point #5 from your linked description:
a group meeting, specifically focusing on having the experts discuss points where their estimates vary widely.
This is like a homing missile for areas where the requirements are not clearly understood, and this is the real benefit of planning poker. Individual estimates may be as good in terms of time-planning-accuracy but they miss out on team interaction and discussion, which (like most agile techniques) is designed to find problems early on where they are cheaper to fix.
Secondly, reflection on disagreements allows team members to think more critically about the estimation process and to become better at it. Marcin Niebudek's linked paper does not seem to discuss this long-term aspect of group estimation (although I can only read the abstract).
You may find interesting the paper titled Inconsistency in Expert Judgment-based Estimates of Software Development Effort. The purpose of that paper was to check the consistency of estimations made by experienced individuals, but one of the conclusions is that collective estimation may reduce such inconsistency.
You will also find some references in this paper. However I think I've seen somewhere the paper or article saying that there is no evidence that collective estimates (like those from planning poker) are more accurate. Maybe someone here may point it. I'll try to find it and add in the comment.
Basically it will always much depend on who is doing the estimates. Don't expect inexperienced team to come up with accurate estimates (no matter if they play poker or not). Planing poker has its strength in encouraging discussion around the most optimistic and most pessimistic estimate within a team, but itself does not provide any magic solution. It's just that you will sometimes discover that some people find some parts of the task more problematic or see some risks that you would not think of if doing the estimation alone.
You may also want to have a look at this post by Mike Cohn where he analyzes the relationship between story points and hours as it sheds some light on the accuracy of group estimates and why there is a tenuous-at-best relationship with hours:
If the one-point stories are centered around a mean of x, ideally the two-point stories will be centered around a mean of 2x. This will never be exactly the case, of course, but a team that does a good job of estimating will be sufficiently close for reliable plans to be made from their estimates. What these two figures show us is that is the relationship between points and hours is a distribution. One point equals a distribution with a mean of x and some standard deviation. The same is true, of course, for two-point stories, and so on…
Ergo, the "accuracy" of group estimates is really - and this is key - continually refined over successive iterations so that the effective distribution centres on a mean. This could be extrapolated by a single individual, however, without group ownership and gut-checking, it has no context and is not refined.
Planning Poker is not really about "estimation." IMHO the word was ill-chosen in Mike Cohn's book, Agile Estimating and Planning. Planning Poker is really about "sizing."
The difference is that an "estimate" is a guess or prediction about how much time will be needed to complete a task, while "relative sizing" is a unitless comparison between the level of effort of two tasks.
The purpose of sizing tasks is not to try and make "accurate" predictions of time. It is to get a general sense of how much work will fit into a given time-box (iteration, sprint, etc.). It's not meant to be highly accurate.
Data to show Planning Poker is better than estimation. Hmm. Are apples better than oranges? Some people would say so, I guess.
IME general task sizing has been at least as useful for purposes of short-term planning as traditional time-based estimation, and it takes less time and effort. I'm pretty sure I know whether I prefer apples or oranges.
There's an insidious problem with task-level estimation. People start to worry about whether reality happens to line up with their predictions, instead of focusing on building the right product. At best, a distraction. At worst...?
Planning poker focuses the team discussion on what matters, and allows quick movement on from what doesn't. It gets the entire team on the same page about what the stories are, and in general agreement about their effort cost.
Picture: team around the table, story one, everyone shows the same number. Great, put it in the bag and move on. No more wasted time on that one.
Next story, 5 different estimates. Okay, that's weird - discuss.
Oh, you mean x? I really thought this was dealing with y. Well, I don't know, maybe it is y. What do you think? Okay it is. What about x? We need another story for x, then. Okay, break it up - this is y, that is x.
Estimate y, similar numbers. Bag it. Estimate x, similar numbers. Bag it.
So, where you might have had one guy estimating a story incorrectly due to a misunderstanding of what it actually dealt with, now you have the whole team with a common understanding of what the story is, and agreement on what it will cost in effort. The important part is that you've headed off potential downstream troubles related to poor understanding of the features at the pass. Consequently, your estimates are probably much better.
However, no, I do not know of any data to substantiate any of this.
This study shows that Planning Poker provides better estimates.
K Molokken-Ostvold, NC Haugen (10–13 April 2007). "Combining Estimates with Planning Poker—An Empirical Study". 18th Australian Software Engineering Conference (IEEE): 349–58. doi:10.1109/ASWEC.2007.15. Retrieved 1 February 2008.
Planning Poker combines a number of techniques, one of most important being the use of relative estimating.
You start with a reasonably well understood task (Task 1) and estimate that. Subsequent less well defined tasks are then easier to estimate, because they can be compared to already estimated tasks. Are they easier or harder than Task 1?
It's akin to looking out of your window and finding two reference points. In a traditional estimating approach you would try to estimate your distance from each reference point in metres. With relative estimating, you say which one is further away and by what magnitude relative to other reference points.
Planning poker as whole probably doesn't have much data associated to it - but the component techniques it uses may do.
