22

Pivotal Tracker "strongly discourages" estimating velocity points for bugs and chores – you have to change a setting and accept a warning to be able to do it.

They explain why here, but I just don't understand it. Here is an excerpt:

By measuring velocity in terms of features only, Tracker can estimate how much real, business-valued work can be completed in future iteration, allowing you to predict when project milestones might be achieved

Chores and bugs take time. How can ignoring this help you to predict when a milestone will be reached?

Say you need to complete three more features to get to the next milestone, and you also need to complete a chore (because one of the features can't be started until that chore is done)... How could discounting the time needed for the chore help you to estimate when the milestone will be reached?

  • I think it's similar to Joel Spolsky's comment on estimating productivity: joelonsoftware.com/items/2007/10/26.html The idea is that you can measure your productivity successfully even when you don't account for bugs / chores (or interruptions / phone calls) because you can expect those to occur at about the same rate over a large time. – wrhall Oct 22 '13 at 17:05
14

I know this thread is a little old now, but as a developer at Pivotal, I don't completely agree with any of the existing answers.

The philosophy behind not estimating bugs isn't that bux fixing doesn't deliver business value, it's that introducing a defect into the app and then fixing it does not represent net forward momentum.

For example, let's imagine that two teams are implementing the same feature set on iOS and Android. The iOS team has bug/chore estimation turned off, whereas the Android team has it turned on.

The iOS and Android teams both estimate the same story at 2 points. They both finish the story in the same amount of time, but in the next iteration, it turns out the two teams have introduced bugs with their implementation.

The iOS team has only introduced one bug. They fix it in an hour.

The Android team has introduced three bugs, and assigns them one point each. It takes them a day and a half to fix them.

The iOS team is moving faster than the Android team, but Android's velocity is now higher. This throws off the planning of future iterations, making it appear as if the Android team is moving more quickly towards a viable release, when in fact they may be introducing bugs into their implementation at a faster rate than the iOS team, and therefore accomplishing their goals more slowly.

Sometimes, however, there are defects which were not introduced by your team. Maybe it's a legacy codebase, and the defect is as old as the hills. In this case, it doesn't make sense for this story to drag down your velocity, and you should probably log it in Tracker as a new Feature, rather than a Bug.

Of course, it's ultimately up to you how you want to do story estimation. I've worked with a few teams who had bug/chore estimation turned on, although I personally prefer to have it turned off. Tracker won't judge you!

  • If the Android team fixed those three bugs as well as all other tasks scheduled for the sprint, isn't the boost in velocity accurate? They were, after all, able to complete the equivalent of a higher estimate of effort/complexity, and that higher number should be applicable when deciding how much work to take on in future sprints. – Wandering NPC Jun 1 '15 at 3:08
  • On the other hand, if they take on the bugs in the same sprint and don't complete all of the other tasks, estimating the bugs serves as establishing "scope creep" during the active sprint, and the stories that weren't completed are accurately 'failed' in that sprint and pushed off to the next one, and the number of sprints remaining to empty the backlog goes up. All of this is accurate, and keeps the truth of things visible. – Wandering NPC Jun 1 '15 at 3:10
  • 5
    The fundamental mistake here is that story points (and thus velocity) cannot be accurately measured across teams. Some frameworks like SAFe attempt to standardize the definition of a story point across teams, but there are plenty of practitioners who think this is a stunningly bad idea. It creates a false equivalence, and misuses a capacity metric as a productivity metric. Velocity is not, and was never intended to be, a measure of productivity. – Todd A. Jacobs Oct 20 '16 at 23:04
  • I actually totally agree with all of that, CodeGnome! The only reason I used two teams working parallel in my somewhat idealized thought experiment was to compare the relative effects of bug/chore estimation on each team. In practice, comparing velocity across teams as a measure of "productivity" is a really bad idea and that's probably more important than whether you do or do not estimate bugs and chores. – Ryan McLeod Nov 18 '16 at 19:00
  • Hey @RyanMcLeod, does the same logic apply to chores? That is, should a chore be made into a feature when it is based on new work that came from outside of the project scope? – Adam Bossy-Mendoza Nov 27 '17 at 20:42
16

"Feature Velocity" vs. Capacity-Based Scheduling

Pivotal Labs is making a marketing decision with this design choice. By addressing themselves to "business value" as the key metric, they are implicitly discounting the importance of tracking team capacity or project scheduling in a broader sense.

Another way to look at this is that they are trying to sell a dashboard for management targets rather than iteration planning. The Pivotal Tracker FAQ says (emphasis mine):

By measuring velocity in terms of features only, Tracker can estimate how much real, business-valued work can be completed in future iteration, allowing you to predict when project milestones might be achieved[.]

So, their idea is that bugs and chores aren't "real" work and don't add value. I disagree, but that's the point that they choose to make with their target audience. They also seem to discount the idea that bugs and chores consume team capacity, and assume that the cost of these non-valuable tasks can be amortized over time, leaving management free to focus on setting targets for milestones.

They also say:

In contrast to features, bugs and chores tend to emerge over time, and while they are a necessary part of your project, they can be thought of as a constant drag on business-valued output - an ongoing cost of doing business.

In other words, rather than make the cost of bugs and chores explicitly visible to the project, they discount them in order to provide a management dashboard that is more focused on feature-delivery than capacity-based scheduling. In my opinion, this prevents proper control of the project's budget and team resource constraints, but there are clearly people who disagree.

Enabling Points for Bugs/Chores

In fairness to Pivotal Labs, they do allow you to disable this misfeature. You can allocate points to bugs and chores, with the caveat (dire warning?) that it's an irreversible decision for the project.

While discouraged, it is possible to enable estimation for bugs and chores, in project settings. However, it is not possible to revert that setting once your project has any estimated bugs or chores.

Personally, I generally recommend that all tasks chargeable against a project's budget or that consume team capacity be made highly visible and properly accounted for. With Pivotal Tracker, the choice is yours.

  • 2
    As a Product Manager, I do think they have a point. My team estimates bugs and it goes towards the team velocity. When we're in the home stretch towards a release, and the critical bugs have been estimated too, we can more accurately ascertain how much work is left. On the flip side, it looks like the team is still "producing" at the same rate, when really they are rolling out less features and more bug fixes. This is inevitable, but I can see the argument for the velocity showing this decreased output. It could make the effects of buggy code even clearer to the team and the management. – Holly Jul 11 '13 at 21:45
  • @Holly That's interesting... But PT says points represent effort, not value. In which case it would be wrong to read the velocity number as "how much business value the team is currently producing", because a low-effort feature might provide a lot of value, and vice versa. – callum Oct 23 '13 at 13:18
  • @callum I was really pontificating on the value of bug fixes only, the same wouldn't apply to features. Points could not be "how much business value", i'm just saying that in if you estimated a story to be 5 points and then after closing it found 2 bugs, each of which is estimated at a point to fix, but if you'd found those during the sprint you would have just fixed them, there could be some value on that side to not counting those 2 points of late bug fix work as part of velocity. BTW I don't know that I would advocate for this myself, I'm just thinking about it. – Holly Oct 29 '13 at 22:23
7

You should estimate bugs and non-functional requirements

I am not familiar with Pivotal Tracker. However, looks like your question is more general:

  • Should I not be estimating bugs?
  • Should I not be estimating technical debt or non-functional requirements? I think this is what you are referring to as "chore".

In my opinion, you should be estimating these. Pivotal Tracker seems to be taking the view that only "features" deliver value to the customers. I would argue that fixing bugs in production improves user experience and delivers value. See here for more. The exception to this is if bugs are found in completed stories in the functional/regression testing or even the post-release stages, they should be fixed by the developers who introduced them, without getting any point credits for the same.

Similarly, if you improve performance, stability or security, that also improves user experience and delivers value. See here for more.

By not estimating these, you sweep these under the carpet, and you run the risk of piling up bugs and technical debt.

  • +1 Bugs and chores require resources such as time, money, or labor, and should therefore be a visible cost to the project. – Todd A. Jacobs Feb 28 '13 at 20:12
0

Another interesting point is that in order to estimate a bug, you have to understand why it's not working, and that's usually the longest of the work.

So it can take a lot of time and speculation for the estimation.

What we did without Pivotal Tracker was calculating and "average value" (based on the time it took to fix the bug, transformed into points), so we would always give this value to bugs (and sometimes have big differences with the reality, but they compensate with time, as we re-adjust the bug value each sprint.

  • 1
    I think this answer is off-base. Estimating bugs is a function of your technical debt: if you have good unit tests and CI, or even just well-written code, you should be able to estimate the level of effort required to at least look into the source of a bug with some accuracy. In other words, it's just an initial time-box. All estimates can be wrong--that can be adjusted for--but if the team lacks sufficient visibility or knowledge to grok the relative size of an issue...well, that's a pretty big bug-reporting or development-team process problem IMHO. – Todd A. Jacobs Jul 11 '13 at 1:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.