What is a reasonable strategy for estimating story points for new stories?

Question

The project I'm running is very exploratative in that I don't know very much detail about features in advance. My boss will tell me that he wants feature A and that after feature A is finished, he'll come up with say a dozen refinements, A1 - A12.

I'm now trying to plan our second 3 month release. The first release went very well and we learnt a ton about running an agile project, but this refinements mess things up in terms of a longer term plan.

My thoughts for this next release follows some ideas from Mike Cohn's Agile Estimating and Planning whereby, say 30% of a release plan will a feature buffer. So as refinements come in, they we will decide whether they are more important than things in the feature buffer and either swap something out with the equivalent story points or leave the refinement out of the release.

My problem is that before we can decide whether a refinement goes in the release or not, we need an estimate of its story points. So far we'd only been estimating new stuff in the interation planning meetings, but this means:

• We can have up to 2 weeks delay before we know the estimate
• If we estimate in the same meeting as the iteration plan, I don't have any time to work with my boss to figure out the priority based on the estimate.
• The refinements start adding up and we could end up spending way too long just estimating for hours on end.

I had two ideas to deal with this:

1. Put some time aside to estimate each day for any new stories that might have come in. Possibly during the standup, but perhaps as a separate event. I like this because planning poker has been working really well for us, but I don't want developers to spend any more time in meetings than they have to.
2. Have a planning poker system, i.e. a web page or something where developers can put in their estimates for a story in their own time. This would only work for very simple tasks where everyone agrees. As soon as people's estimates diverge, we'd do it in person. I like this because it removes the need for more meetings. But I don't like that we're not talking about stories in person, so there's much more potential for misunderstanding etc.

So my question is, how and when do you guys estimate story points (or whatever measure you use) after you receive a new story?

Answer 1

From what you write you estimate features because you need predictability, namely you want to know how many and which features you can pack into the next release. At the same time you understand that requirements are changing over time which means that your estimates are going to change as well. You try to cope with it estimating and re-estimating in details as you learn more about features or changes in them.

Another, different approach would be to measure just a coarse-grained size of features and then track down how much time you've spent on a feature in reality. For example, you can start with T-shirt size estimation (S, M, L, XL, etc) and then look how much of variability there is in real effort invested into building these features. (see 2 and 4 in sources)

You should quickly learn that there are ranges of effort spent on building each size of features. Now you may use this historical data to roughly say how many features of each size you can pack into the next release. You will probably end up with a bunch of scenarios: optimistic (most features are smaller than average of their size bucket), average (basing either on average or median size of features in each of size buckets) and pessimistic (most features are bigger than average). Actually multiple-point estimate which you're going to get this way is better than a single-point estimate. (1)

Basing on this you may come of with a list of features that are going to be, likely to be, not likely to be and definitely not going to be in the next release. (3)

From your perspective it should significantly decrease amount of effort you invest into estimation, as you're just going to assign every new feature to one of general sizes instead of busily estimating story points for each of them. What more, in retrospect you can either choose to assign time spent on feature and its refinements separately or jointly. I would probably go with the latter as it means you're going to estimate how much time you need to finally get a feature ready to release instead of getting the first version of a feature which is going to be adjusted several times. Anyway the choice is yours here.

What more, in the long run you should get better results this way as instead of basing on guesses (entry mechanism for planning poker is guessing) plus discussion you will use your track record to improve quality of data. (5)

Note: it may be difficult to kick off without having historical data at hand so you may want to use the current method for some time while you're gathering data and then eventually switch to a new approach. By the way, it would allow to compare results from both methods too.

UPDATE: Sources/further reading:

1. For some basic ideas on estimation I strongly recommend Steve McConnell's Software Estimation: Demystifying the Black Art
2. For understanding how estimation based on historical data works I recommend Joel Spolsky's article on Evidence Based Scheduling. Although I don't propose to apply the concept directly it will help you to understand why and how it can improve the quality of your estimates.
3. Deciding what gets, what can get and what doesn't get into a release is pretty popular concept and is covered for example in Mike Cohn's book Agile Estimating and Planning, which by the way you probably know.
4. Variability of work and how it influences planning is pretty well described in David Anderson's Kanban: Successful Evolutionary Change for Your Technology Business. Note however that the main goal of the book is introducing Kanban and not discussing variability. Nevertheless the book is worth recommending anyway as it can help you with other areas of organizing software development as well.
5. I described a bit similar approach I used in my team in a post on Coarse-Grained Estimation. A related topic was standardizing feature size which you may or may not find useful or even feasible in your case.

Answer 2

I have successful used what I call a "Piling" exercise to do a one time establishment of story point sizing and to size a backlog in one pass without having to argue over each individual story.

1. I have the team print or write out each story/feature onto a card/piece of paper
2. take some post it notes and spread them across the table and label them XS, S, M, L, XL, to identify 5 possible story piles (you can obviously have more than this if you want)
3. Ask the team to put each story into one of the piles, with discussion and clarification if needed (sometimes some good things are discovered)
4. after the piling is complete, label each pile with one a story point value (3,5,8,13,21,ect.)
5. Rejoice!

From here I found that the team easily starts to level set on what a "5" means and what an "8" means and so on.