There are two distinct projects where we are trying to define a bug prediction model.

We used bug at file level and defined co-factors:

  • Number of changes per file,
  • Developers who left per file.

Project 1 presented for both positive co-factors.

Project 2 presented changes per file positive and developers leavers negative.

I can interpret in project 1, more changes in file more bugs. More developers leave (lose know-how) then more bugs.

How come project 2 we can have the less developers leaves the project, the more bugs?

The only thing come in the top of my mind is that people are not skilled, or that the project is too old and people must be very careful to touch the project and to not break anything.

  • 1
    Perhaps you don't have the right indicators? Developers leave for many reasons, and many of those reasons are not related to code quality.
    – MCW
    Sep 24 '14 at 14:49
  • @MarkC.Wallace Thanks for your comment. Actually, we counted the number of leavers developers who worked in a particular file. This implies that the leaver person had knowledge about that file and know-how is lost. We are not worried about the reason a developer left the project.
    – Dora Maris
    Sep 24 '14 at 15:27
  • 1
    You cannot conclude anything from this. Results are random and a range. You cannot uncover the natural range of bugs from studying only two projects. This is like trying to understand car accidents in your country using two trips with two drivers along a ten mile strip on route 27. Sep 24 '14 at 19:46
  • Are there rules that define when the developers should leave or not? Can they leave the file when there are open bugs?or, when they are leaving file, are they removing the bygs? This can help to investigate further the reasons
    – user5529
    Sep 26 '14 at 9:46
  • You may be interested in reading Stephen Kan's Metrics and Models in Software Quality Engineering. There are various models for software quality and reliability, along with techniques to manage and improve them that are discussed in several chapters of the book. However, I'm not sure that your ideas relating people leaving to quality is appropriate. Also, 2 projects is not likely to yield sufficient data points for any kind of meaningful analysis.
    – Thomas Owens
    Sep 26 '14 at 13:33

Don't Perform Bad Science

The fields of computer science and project management currently have no models that can accurately predict software errors. There are various theories and models, but none of them will provide you with guaranteed results. See cyclomatic complexity as just one example where there is disagreement about the measure's correlation to defects.

In your case, you are attempting to perform statistical analysis without sufficient thought given to either test construction, controlled variables, or the null hypothesis. You are also confusing causation and correlation. Even if you have a validly-constructed test with adequate controls, and even if you are able to reject the null hypothesis, you would be extremely hard-pressed to use either of these projects as direct proof of cause and effect.

Introspect Your Projects

A better use of your time would be to look at Project 2 carefully to see why you have such a high defect rate. Inspect your processes, form a hypothesis, and then attempt to adapt your processes to reduce the defects.

A good project manager won't spend time attempting to form a "universal theory of everything" based on extrapolations from two projects. Instead, an experienced project manager would spend that time fixing Project 2 to improve its chances of success, or communicating the likelihood of failure to the portfolio manager so that a strategic decision can be made about whether or not the project should be terminated as a cost- or quality-control measure.

  • 3
    +1 on bad science. You cannot interpret and draw any conclusion from two projects looking at two variables. Try 50 projects and 50 variables with several experiments, and you may find something worth studying a tad bit more. Sep 24 '14 at 19:43
  • I really like all these theories CG knows!
    – Tiago Cardoso
    Sep 27 '14 at 9:25

It may or may not be valid to assume the developers on Project 2 are not skilled. You would need to gauge their performance across multiple projects before you could come to a reasonable conclusion as to their skill level.

Perhaps with regards to Project 2, it does not matter who is working on it. Perhaps the project is so complex that, as you say, anyone touching it is likely to break something. I would focus more on that issue than anything else.


Wrong approach!

You can't predict software errors, the same way you can't predict car accidents.

As programmers gain experience they learn from their previous mistakes, but make more sophisticated errors.

Number of changes per file

Not a good indicator of anything. The changes could be from additions or bugs, typos or logic errors.

What is a file? Some files have multiple functions and lots of lines of code. others may have a single function, with even more lines of code, or maybe almost no code. Some others may be header files that don't do anything, but are needed for compiling.

(If the above means nothing to you, then you need to get up to learn about it, as I wrote here.)

Imagine deciding the value of a car based on how often it was washed.

Developers who left per file

I have no idea what you are trying to infer from this metric. Sounds like trying to predict the popularity of a fast-food joint based on how many quarters they gave as change.

The rules about bugs is to test from early stages of development, and to keep a 1:1 or 2:1 ratio of programmers to testers. Then you can safely assume the testing-debugging stage will take as long as the time originally estimated to write the code.

The later you start testing and the fewer testers you have, then more bugs will live and start growing in the software and the debugging stage will take longer. Then you can double or triple the testing/debugging time.


  • Because fixing code that was written recently (today) is easier than fixing code written a while ago (days, weeks or months). Code that's not fresh in a programmer's head needs to be re-read and understood.

  • Newer code is often based on older code, so if there's a mistake earlier on, it may waterfall all the way down and cause trouble everywhere.

Lastly: Number of bugs is not a good indication of anything, unless you factor in their severity, importance and impact. Both a typo and intermittently crashing the system are important bugs. One will take a few minutes to fix, the other may take a real long time, as intermittent crashes are almost impossible to recreate and find.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.