0

I wanted to get an idea of categorizing bugs based on how easy is it to solve and how much benefit it will give me. for e.g., if there is a bug which will take say an hour (double file close etc.) to solve vs another which takes a day (segmentation fault). But if solving the first bug is not very important, then I'll probably work on the second one.

How can I categorize bugs based on cost-benefit or similar metric and how can one prove that this is effective?

[EDIT] Let's say it is possible to categorize bugs based on bug characteristics e.g. security vulnerability, memory error, logic error etc. On the other dimension there could be parameters like difficulty (easy, medium, hard). Are there other dimensions I should be looking for. To simplify things, I can assume two things:

  1. Every programmer in the team is equally capable of solving any bug
  2. There is no deadline
5
  • 2
    Hi Aditya, welcome to PMSE! I've often wondered the same. I edited your post though to focus on the problem but also mention that the answers should ideally be backed up with references or even just experiences. We aren't really about just posting only links, as we also want to be a learning resource that is built on top of the expert knowledge of our community. Hope this helps! :)
    – jmort253
    Aug 3, 2013 at 1:06
  • Hi @jmort253, Thanks for the edit. I think many programmers would like to have an idea about which bugs to solve first to get maximum benefit (in terms of time/other resources).
    – A. K.
    Aug 3, 2013 at 6:28
  • Please don't cross-post. programmers.stackexchange.com/questions/207030/… Aug 3, 2013 at 17:43
  • I did because, in the beginning I was not sure which place is the best to ask this question. I could have deleted from one place but if you see I have got very different answers from both places and very meaningful.
    – A. K.
    Aug 3, 2013 at 18:03
  • If you're not sure about where to post, just ask. But cross posting is generally frowned upon unless you tailor the question to each site individually. (The only tailoring was done by me.) Still, I'll link the two together as it may be useful for future visitors to see both. In this case, I'm not 100% convinced harm was done. The question for our community is this: Is this a project management question, a programmers question, or both? If both, let's just leave it as is.
    – jmort253
    Aug 3, 2013 at 23:17

3 Answers 3

3

This is a multi-criteria decision problem or multi-objective optimisation problem, depending on how you want to look at it.

There are hundreds of methods for assessing these problems, from simple intuitive methods through diagrams like decision trees to linear programming to more exotic stuff like nature-inspired computing.

The simplest is to estimate expected value and expected cost, and to derive expected outcome. This is the intuitive model that most people follow. A slight elaboration is the Weighted Sum Method, where you arrange alternatives and criteria, then weight them and sum the result.

The key problem: multiple solutions

The key thing to understand is that you might not find a single best option, given the method and the inputs. Optimisation problems can and often do throw up a "Pareto front" of alternatives that cannot be strictly ranked from "best" to "worst". You can use tools to make the decision easier, but you will still need to exercise judgement.

An example that was used in a course I took was based on work commissioned by an iron ore company. They wanted to optimise iron ore crushing plants for profitability. This means deciding how much money to spend on an ore-crushing plant. Different configurations of crushers produce different fineness of ore, which fetches different market prices.

So this particular group of academics applied a genetic programming system to evolve model plants, with profitability as the selection function. At the end of the runs there would be a long string of alternative crushing plant designs generated that were indistinguishable for profitability, trading off between iron ore grade and construction and operations cost.

Ultimately the problem that the commissioning mining company ended with was the same. There was no one best solution. There were many equally best solutions.

4
  • Genetic algorithms don't generally solve for "best," but rather for "good enough." Otherwise, I very much agree with your answer. I upvoted your answer, but think the last sentence should be changed to "equally valid" or some such.
    – Todd A. Jacobs
    Aug 3, 2013 at 15:29
  • They can solve for lots of things, it depends on the design of the evolver (genetic programming is different from genetic algorithms, by the way, a confusing little world that). As for "equally best", it captures that often you can't distinguish two alternatives. Either dominates all the other solutions, but they cannot dominate each other. By themselves either is clearly best. Together they are best ... equally. Aug 3, 2013 at 15:48
  • 1
    In English, best is the superlative of good and means "excelling all others." You therefore can't have multiple best options in the same category.
    – Todd A. Jacobs
    Aug 3, 2013 at 16:11
  • Which is why my form of words works. It breaks the ordinary meaning of best, because the ordinary meaning of best misleads people into an incorrect line of thought: that a best solution exists. It often doesn't. Aug 4, 2013 at 0:42
2

TL;DR

There's no silver bullet for cost/benefit analysis of any kind. Value is determined by your stakeholders, and varies from organization to organization. If you search for dissertations on the subject, you may find various mathematical models for bug triage, but real-world agile practice is typically to hold a discussion to estimate effort and assign resources and priorities, while legacy organizations often rely on organizational policies and (sometimes) expert opinions to classify bugs.

Perspective Matters

The following are certainly gross over-generalizations, but they are truisms for most projects I have personally worked on. YMMV.

The Project Management Perspective

From a project management perspective, bugs typically fall into three broad categories:

  1. Features that don't work as expected.
  2. Features (or mis-features) that have negative side-effects.
  3. Change requests disguised as bug reports.

The Product Owner Perspective

Bugs can be categorized as one or more of:

  1. Resource drains that reduce project capacity to deliver new features.
  2. Value-decreasing issues that make the product less marketable.
  3. Value-decreasing issues that make the product less shippable.

The Development Team Perspective

Bugs may be:

  1. Change requests in disguise.
  2. A correct solution to an incorrect assumption or specification.
  3. A defect in the documentation or user training, or some other failure to communicate effectively with the end-users.
  4. An actual defect in the implementation.

Prioritization vs. Estimation

The Scrum perspective is that it isn't the developers' job to determine what bugs or features should be handles first. The responsibility is actually divided between the Development Team and the Product Owner as follows.

  1. The Development Team gives a rough estimate of the work involved for each bug during Backlog Grooming.
  2. The Product Owner prioritizes the bugs the same way he prioritizes all other user stories on the Product Backlog.
  3. The Development Team and the Product Owner do some horse-trading during Sprint Planning when finer-grained estimates and relative project value are available.

In the end, though, the developers are responsible for level-of-effort estimates, while the Product Owner is responsible for determining the cost/benefit of a bug or feature within the project. When this division of labor doesn't result in a self-evident relative value to the project, the Scrum Team as a whole can bring their multiple perspectives together to hammer out at satisfactory solution.

1

I agree with all the above answers. I would like to add from the business perspective too. You can discuss with all the business stake holders and your own team too and get their priority. Fixing the bugs based on business priority is very important as their input tells us which functionality is needed to run the business. Some of the bugs does not require fixing as it is not important to business and can be deferred or it is nice to have not must to have feature.

In my view I fix the bugs what business need after all business only provided this opportunity.

If business gets the benefit that is the greatest benefit for us and cost wise too.

3
  • It is true that business drives things. But the thing is you don't want your 'customers' to find bugs. If they find bugs in your program then they have already lost trust.
    – A. K.
    Aug 3, 2013 at 17:11
  • Users, and business people, don't always understand what goes on under the hood. The project managers and developers just plain have to use good judgement, take business needs into consideration, and make sure they're also fixing things that address the goal of making a profit. This is why it's good for developers to have business experience and project managers to have development experience. On the other hand, I've seen developers waste their time on some pretty stupid things, so I think this can go both ways.
    – jmort253
    Aug 3, 2013 at 23:12
  • I agree with both of you. May be the context was missing. My assumption was at User Acceptance Testing (UAT). Ofcourse having many bugs at UAT is indicative of not doing good development and prior phases like SIT or so. Some times the bugs will be more in UAT because users use or test what the functionality is required for business and developers and testers may not. Aug 4, 2013 at 5:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.