5

As far as I'm aware there's no definition of Story Points and how to compare them. Each person in a team may have his personal understanding of the correlation between an effort and Story Points. Isn't Story Points estimation just a fallacy?

Isn't it just a belief. For example, it's assumed that all tasks have a specific property - the difficulty, the amount of effort. But they maybe don't. And even if they do, it is just a belief that we can adequately estimate it as a number. The amount of time a task will take is intrinsically indeterminate.

For example: During Planning Poker all teammembers agree that a PBI should be estimated as 10 Story Points and they go to the next PBI. This 10 Story Point estimation actually means nothing because everybody understands 10 Story Points differently (different amount of effort, time, risks).

I just want reliable arguments (a research, comprehensive surveys) that SP is really a tool, and not just a belief.

Improve this question
11
  • 8
    The question and the comments on the answers makes it seem like you just want to argue. Story points have no fixed tie to time or money. Consistent teams working in the same context often observe a strong correlation after they've been working together for a period of time. The technique was designed this way. It's just a tool you can use or not.
    – Daniel
    Commented Jan 27, 2020 at 2:10
  • 1
    Chris Brettini you got arguments! Thing is you have a BELIEF that only your BELIEF is right and you cannot provide reliable arguments for that.
    – Tiago Peres
    Commented Jan 27, 2020 at 11:23
  • 3
    The request for information is good - although I rather thought this had been addressed before on PM:SE. The distinction between "tool" and "belief" seems like a false dichotomy and seems to be deployed in service of a rhetorical goal.
    – MCW
    Commented Jan 27, 2020 at 16:00
  • 1
    Yes, i also think what Chris Brettini wants was already answered here.
    – Tiago Peres
    Commented Jan 27, 2020 at 16:04
  • 1
    I'm not sure what a belief is defined as. What Taigo posted may help. Also, in one of Mike Cohn's keynotes he talks about his organization running multiple estimations methods against each other and story points came out on top as most useful. Hundreds, if not thousands of teams use them successfully every day. They are a well-priced tool, regardless of published studies. They do, however, challenge many traditional assumptions about estimation, so you can't use them like you use hours.
    – Daniel
    Commented Jan 27, 2020 at 20:21

5 Answers 5

Reset to default
15

Story points are a relative measure of effort rather than an absolute one. However, each member of the team should have the same understanding of the size of a points estimate. A common understanding is achieved when the team estimates repeatedly together and when they agree common baseline stories against which to measure. This is really no different to estimating in hours or days where people also measure things against remembered baselines. Planning poker is one way of making sure that teams have a common understanding of the size of items.

Relative estimation with story points has a few advantages over absolute estimation. It seems that many people come up with more accurate relative estimates than absolute ones. Velocity, as measured by story points completed per iteration, is an evidence-based measure whereas hours based estimates tend to be more subjective. If you measure things in hours then you can still retrospectively measure how many estimated "hours" you actually completed but that will inevitably differ from actual hours of work put in, so the reality is that "hours" tend to become a relative measure too.

Improve this answer
9
  • Thank you! The advantage of hours is that everybody undestands them. How do we know that we have the shared understanding of story points? Should we include risks in SP-estimation? How to estimate complexity in SPs?
    – Chris Brettini
    Commented Jan 26, 2020 at 10:40
  • 5
    You know that you have a shared understanding of story points estimates the same way you know that you have a shared understanding of hours estimates: by agreeing them as a group. Team estimating techniques like Poker or Bockman's Game ensure that if anyone has a difference of opinion then that difference is obvious to everyone and the group has a chance to agree a consensus.
    – nvogel
    Commented Jan 26, 2020 at 10:53
  • Your argument has a flaw - you already assume that every body has a shared understanding. Assuming this being the case you then describe the Planning Poker process.
    – Chris Brettini
    Commented Jan 26, 2020 at 14:19
  • Planning Poker is about achieving a shared estimate for a PBI - not about achieving a shared understanding of what 1 SP means.
    – Chris Brettini
    Commented Jan 26, 2020 at 14:22
  • 1
    @ChrisBrettini I often start with 8 points = what can be done in a day, but it doesn't matter. Your first few iterations will be wrong, but increasingly less wrong as people adjust their estimates based on data. Tried to do 40 points last iteration, got 20 done, next iteration's capacity is 20. Your iterations should be short, a week or two, for fast feedback and correction. Daily tracking of task completion with burndown charts will quickly tell you if you're on track or not. This leaves nowhere to hide bad estimates. Long term estimation is an advanced topic.
    – Schwern
    Commented Jan 26, 2020 at 23:03
12

Let's be serious, people don't usually care how you do estimates. What they care about is how much it takes and/or how much it costs. Time and money. That's what they want. The estimates is just something that helps you answer those questions. It doesn't matter what you use for estimations as long as people can get back a time or money value. It can be estimating directly in hours, or man days, or it can be story points, T-shirt sizes, puppies or vegetables. Nobody cares. Seriously now. It's about time and money.

So you need to have a way to convert from an estimation to time and money, right?

Everyone understands what time is. Everyone understands what money is. And we like to think about them as absolute. One hour is one hour. Ten bucks is ten bucks. But not really. They mean different things to different people. If I am rich and you are poor, ten dollars for me might be useless but for you might be difference in having food on the table or not. If I am a busy person and you are not, then one hour for me means a lot and I use it wisely, while for you it might mean spending it online whatching cat videos on YouTube. Although we perceive them as absolutes, they are not.

From the discussions on the other answers I see that you are asking why not estimate in hours directly instead of story points, since story points are abstract and not absolutes. Everyone understands one hour, but story points mean different things for different people, right? But from what I said above, you see story points are not so different than hours. They mean different things for different people. One hour of development for a senior developer doesn't mean the same thing as one hour of development for a junior developer. The senior can build an entire feature in one hour, the junior might use that hour to figure out how exactly to approach the feature. If the senior developer estimates a feature to take one hour, that estimation is subjective. It depends a lot on skills. The senior will build feature F in one hour, but the junior might take four hours to build the same feature. So what good is a one hour estimate for feature F if it will have to be the junior who needs to work on it? (if the senior developer is unavailable for example).

Estimating in hours is a way to lie to yourself and give you false confidence. You understand hours, so when you estimate a project and get back 1078.65 hours then you have some absolute information there, right? You know what you are dealing with. But you don't. Software development doesn't work like that. That's why we are no longer doing Waterfall all over the place but instead trying to be more Agile. There is a lot of complexity in building software, there is a lot of effort that goes into building the right thing, and a lot of risks. Hour estimations don't reflect these and thinking hours are absolutes is simply delusional. History has shown us that. People suck at estimating, and they suck at attaching hours to those estimates. But it seems we can better estimate things relative to each other. If you have two features, you can estimate pretty well which one is larger than the other, thus which one will need more effort or take more time.

Story points are a way to highlight the difference in sizes between features. A 5 SP feature is more than a 3 SP feature, and less than a 8 SP feature. People might not agree that one hour or ten dollars are the same for everyone because a lot of subjective things influence that, but they can agree that one feature is more complex than another. A 5 SP story is a 5 SP story for both the senior developer and the junior developer. It might take the senior one hour, and the junior four hours to build it, but that doesn't change the fact that in relation to the things they both worked on so far, this is a 5.

Initially people have different understandings about what a 5 is. The senior might think 5 is easy, the junior might think 5 is hard. So when estimating you will get different values for the same feature. But there is a discussion. People dissect the feature and explain why they think it's a 5 or a 1 or a 13 or whatever. In time they figure out, relative to the other features, what is a 5 and a 1 and a 13. It doesn't matter how they subjectively reached that number, relatively speaking they learn to attach the same numbers to similar sized features. Once this happens people will know how much to pull into the sprint and the velocity will start to become relevant. Then you can attach hours to the story points per team as you know how much they can deliver per sprint. But just remember that it will still not be an absolute. There isn't a coincidence why you use Fibonacci to estimate. The higher the SPs, the higher the unknown. In fact, it's not even Fibonacci. A Fibonacci sequence is 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, but most planning poker cards are 1, 2, 3, 5, 8, 13, 20, 40, 100. Things get rounded of. The number 89 is absolute, 100 is an approximation. Does it really matter it's an 89 or a 90 or a 95? It makes no difference. It's a lot. So just say 100 and call it a day.

Enough rambling... to get back to your question. The definition of a SP is that it's an abstract measure for the difficulty of a feature and the effort needed to build it. With time, the people in the team figure out what SPs mean for them (this is why, for example, you can't compare story points of one team with the story points of another; 10 SP in one team might mean 40 SPs in another).

See also if this provides extra insight: Why use story points instead of hours for estimating?

Improve this answer
12
  • Thank you very much for a detailed answer! But it's still just a belief. For example, you assume that there's such a thing as a difficulty of a task/feature. How do you know that such a thing exist at all? You are sure that all tasks have a specific property - the difficulty. But they maybe don't. And even if they do, it is just a belief that we can adecuately estimate it as a number.
    – Chris Brettini
    Commented Jan 26, 2020 at 14:44
  • 2
    All features have difficulty and effort needed to build them. That's not a belief. That's just how it is. Things can be easy to do, or hard to do. Things can take a small or a large amount of time to be done. And all things carry risks. When you estimate, you take all these factors into account, no matter if you estimate in hours or in SPs. I didn't quite understood your comment, but the point with the SPs is not to adequately estimate the difficulty as a number. You should consider SPs as boxes in which you can put your features. Smaller boxes, larger boxes, etc. All estimations are...
    – Bogdan
    Commented Jan 26, 2020 at 16:38
  • 2
    ... most likely wrong. When you use SPs to estimate, the team discusses the work enough to figure out, understand, and agree in which which box a feature should go into. Like many things in Agile, SP estimation is "just enough" estimation to understand the effort needed (i.e. similar effort as other similar features from the past). A 5 SP feature doesn't mean that the 5 is accurate, it just identifies the box with 5 written on it. You can later figure out time for SPs depending on how many SPs the team delivers per sprints. Estimating directly in hours gives you false absolutes...
    – Bogdan
    Commented Jan 26, 2020 at 16:38
  • 1
    ... that trick you into thinking that you get palpable useful information out of the estimate, but in fact you don't. If you estimate something as 10 hours and it actually takes 11 hours to do, then you are late on your work. If a 5 SP feature takes 10 hours or 11 hours to do, then that's how much it took. You are not late. That's just how 5 SPs translated to hours. After a few sprints, a few rounds of SP estimations, a few values of velocity, you can get an idea on how much things take. But nothing is absolute when it comes to estimates. SPs better express that fact than hours could do.
    – Bogdan
    Commented Jan 26, 2020 at 16:38
  • 1
    ... were said on this post, an advantage of estimating in SPs is that upper management has less room for messing with the estimates. If developers say 10 days, some might be tempted to try to pressure developers into giving a less value ("Can't you guys make it in 8 days? It's really important, it's needed for such and such, bla bla bla"). If something is 5 SPs, you can't say "Can you make it a 3 SP?" because you can't decrease the actual difficulty of something the same way you can do with decreasing the time you work on it.
    – Bogdan
    Commented Jan 27, 2020 at 9:02
5

Each person in a team may have his personal understanding of the correlation between an effort and Story Points.

Initially, in a new team, that may be true. That is why an estimation based on Story Points is more than each team member just giving a number and then taking the lowest/highest/average/whatever as the final estimate.

When doing a Story Point estimation, that should also include a discussion in which the team members can explain what they considered when coming to their points value. It is important that at least the people with the highest and lowest estimates a heard, because they are likely to have specific insights into the topic at hand. This can also include insights into risks and/or uncertainties associated with the work item at hand.

Through these discussions, the team members will also get a more common understanding of the combination of effort, complexity and risk that goes into a Story Point.

To underline that estimation is not an exact science and to avoid endless debates if a work item should be 40 or 41 points, estimation techniques like planning poker (that are commonly used to estimate story points) have a granularity of estimates that can be given that increases with the size of the estimates themselves.

Improve this answer
3
  • How do they come to a common understanding of risks? Risks estimation depends on personal experince. Why should we bother ourself with these obfuscated SP instead of just using hours?
    – Chris Brettini
    Commented Jan 26, 2020 at 10:42
  • 1
    @ChrisBrettini, how do you gain experience in estimating (end recognizing) risks? I hope not exclusively from hitting your nose, but also from others who can point them out before you hit your nose. And that is where the discussions come into play, because that is where someone can point out the risk to the other team members.
    – Bart van Ingen Schenau
    Commented Jan 26, 2020 at 11:00
  • 1
    @ChrisBrettini The team comes to a common understanding by experience. The longer a team exists as a team, the more their own personal ideas of what a story point is worth will tend to sync up. When you change the members of a team, there will be a period of adjustment where their individual estimates differ more widely, and they have to discuss and agree on which number to use. Over time, they'll start to suggest the same numbers, they'll require less discussion, and their velocity of points per sprint will become more consistent. I've seen this happen to me at multiple companies. It works.
    – anaximander
    Commented Jan 27, 2020 at 11:23
4

Mike Cohn has a great article on Story Points. Some of the highlights are

Story points are a unit of measure for expressing an estimate of the overall effort that will be required to fully implement a product backlog item or any other piece of work.

...

Because story points represent the effort to develop a story, a team’s estimate must include everything that can affect the effort. That could include:

  • The amount of work to do
  • The complexity of the work
  • Any risk or uncertainty in doing the work

...

A story point estimate must include everything involved in getting a product backlog item all the way to done. If a team’s definition of done includes creating automated tests to validate the story (and that would be a good idea), the effort to create those tests should be included in the story point estimate.

Story points can be a hard concept to grasp. But the effort to fully understand that points represent effort as impacted by the amount of work, the complexity of the work and any risk or uncertainty in the work will be worth it.

Improve this answer
32
  • 1
    Yes, but he gives no rules how to convert each personal forecast of the amount of work, of the complexity, of risks into a one number estimate. So, as I said everybody understands Stoty Points differently. What is the use of them then...
    – Chris Brettini
    Commented Jan 26, 2020 at 9:12
  • 1
    You can use hours if you want. But the amount of time you take to achieve something doesn't express the effort. Same logic applies if you're paid by the amount of hours, then the way you do activities / tasks is different than when you're paid by activity / task.
    – Tiago Peres
    Commented Jan 26, 2020 at 9:52
  • 2
    Dan Radigan - «Hours don't account for the non-project related work that inevitably creeps into our days (emails, etc). Hours have an emotional attachment. Each team will estimate work on a slightly different scale, which means their velocity (measured in points) will naturally be different. This, in turn, makes it impossible to play politics using velocity as a weapon. Story points reward team members for solving problems based on difficulty, not time spent (This keeps team members focused on shipping value, not spending time).»
    – Tiago Peres
    Commented Jan 26, 2020 at 10:54
  • 3
    @ChrisBrettini Story points are more accurate because humans are notoriously bad at estimating time. It's hard to estimate a number of hours to allow for "there is still some uncertainty about how this will be implemented". It's easy to compare a task to something you did before and decide if it's bigger, smaller, or similar. Points deliberately aren't the same across teams, which means you can't say to Team A "here, Team B says this is four days of work, so it'll take you four days, right?" Team A has to estimate it for themselves, which mitigates the "mythical man-month" problem.
    – anaximander
    Commented Jan 27, 2020 at 11:16
  • 1
    Also, "The customer isn't intrested whether it is hard or not - they want to know the amount of time" this is true, but the customer should not see story points. Story points don't mean anything to anyone outside of the team who estimated the stories. The points can be used to decide how much work fits into a sprint, and at that point, whether it's hard or not is very important. Story points can be used to estimate how many sprints worth of work is in your backlog, which can be used to give your customer the estimated delivery date they want. (Note that this will shift as time goes on).
    – anaximander
    Commented Jan 27, 2020 at 11:20
1

Without external measuring devices, I can compare two cups of water and guess which one is fuller than the other.

I can't tell you how much exact liquid I can fit in the cup, nor can I tell you whether putting the liquid from one cup into the other will result in overflow without trying. If both are really full I may have some ability to do so; but it depends on the relative sizes of the cups and how much water appears to be in each.

My point is: while I can make inferences and deductions trying to compare the two cups to each other; I can't tell you much else, because it's unknowable without more precise measuring and a scientific process.

Software Development is anything but a scientific process -- it's about as far from science as you can get. I guess that's why we call it "Software Development" and not "Software Science"

Story points are used to measure work against work done in the same sprint; and their values are relative to the work being done. Much like the water in the cup, they have no measurement or relevance to work done in the past or work yet to be done -- that requires measurements that we don't have because we're not really able to measure the changes in environment that cause software to be built or not be built.

For instance, any of the following can affect velocity:

  • New Team member
  • Bug contains a dependency we didn't know about
  • Team member has an issue with another team member
  • a software development environment upgrade causes unforseen side-effects
  • NPM goes down
  • After starting development, a developer notices the problem is deeper than we knew
  • A developer gets confused by another developer's 'clever' code
  • Any one of the items listed here.

My point is, any estimation technique that attempts to do anything other than size the work immediately in front of you with work that is also immediately in front of you is subject to extreme disappointment.

There are two ways around this:

  1. Break down work so small it's easily estimable reliably.
  2. Work on one thing at a time, with the whole team working on it, to ensure there are no blind spots or tracks that can collide (Mob Programming).

Most teams I've seen that have run into problems with Story Points have tried to use them as some sort of estimation of how much work can be done in a sprint reliably in a dynamic environment; or comparing velocity over time, or thought of them as a reliable measurement of absolute estimation.

Improve this answer
1
  • 1
    The two-cups-of-water analogy explains quite neatly where the value of a point comes from. Given two different cups, you could eyeball that their sizes have a roughly 3:5 ratio (for example) but that doesn't mean that the smaller cup holds 3 litres, or three pints, or three of any particular unit, it's just about three-fifths of the larger one. It also explains the use of Fibonacci numbers - given a very large cup, you likely can't eyeball the difference between 25:3 and 26:3 very accurately, but you could probably decide whether it was closer to 21 or 34.
    – anaximander
    Commented Jan 28, 2020 at 9:15

Not the answer you're looking for? Browse other questions tagged or ask your own question.