21

Quality is one of the four key project constraints, which need to be planned and controlled by a project manager during the entire project lifecycle. In order to plan and control it the PM, first of all, has to understand how to measure it.

What are possible and most effective methods of measurement of quality in a software development project?

3
  • 1
    Possible methods are googlable. What are you hoping to accomplish with this question?
    – Marcie
    Apr 6, 2011 at 20:57
  • The "hooks" for measuring are very dependent on the specific methodology. Are you talking about one of the lean/agile approaches, a "traditional" approach, or something else? Apr 7, 2011 at 4:27
  • I'm surprised that no one really emphasized 'requirements' in most answers bellow? The requirements will establish your 'limits' or 'range' for what's expected and acceptable. Your measured quality will be practically entirely based on those requirements. As most have pointed out, it is subjective. In one case one person may see an action as 'good enough' and another as 'not good', but if your requirements quantified it properly, then and only then could a measure of quality be established by comparing actual results to the stated requirements.
    – Jeach
    Feb 15, 2013 at 21:24

6 Answers 6

7

Depending from circumstances, you can use different techniques to evaluate the quality of a software product:

  • Completness: What part of the needed features are actually implemented
  • Asking users: What is the feeling of typical users about the software?
  • Metrics: Some metrics can give you a good idea about the quality of the code
  • Process: The use (or not) of certain processes is a good hint about the quality of a development process. Bugtracking, automated tests, versioning tools...
  • Bug detection: As BillThor explained, bug detection rate is a good indicator
5

If you are talking about quality in terms of correctness and stability after release, then bug defect rate is the easiest way to measure it. The problem with this metric is that it only works after you've released it. It's already too late.

Even though it's too late, I'm going to suggest this is still the best measure. You can at least change your development practices to improve it for the next release.

That's what happens with project development. You do what you think is necessary in regards to quality, ship it, take your lumps, and improve. This is why it's important to have experienced developers and testers. They've already taken their lumps.

Some practices we rely on to address quality:

  • Automated integration tests.
  • Unit Tests.
  • Acceptance tests for every story (we use user stories for requirements)
  • Feature Level Tests (testing groups of stories)
  • Code reviews (all our code is reviewed using a tool)
  • Risk tracking
  • Manual testing for each story, immediately after it is implemented.
  • Automated performance tests.

Many of these practices have come about as corrective measures after we learned we could do better. It's an iterative process. We change every release.

2
  • Unit Testing will also determine if the bug is in the implementation or spec. In other words, if code coverage is approaching 100% and all tests are passing, then the problem is obviously in the understanding of the solution. Apr 7, 2011 at 18:33
  • 2
    @John: to an extent. Even if code coverage touches every line (100%), you will never have 100% path coverage. And unit tests, by design, ignore integration issues, where many bugs can lie. Apr 7, 2011 at 18:42
4

As it was described above quality is very wide and subjective matter. What comes to my mind is for example:

a) meeting customer needs - if you have comfort to collaborate customer you can measure it by demonstrating your product to him, and have excellent feedback about this aspect of quality.

Lot of aspect is significant here starting from product robustness through implementation of most needed features finishing on usability of product and comfort of work for customer. All aspects are more or less included in feedback.

Hard to think about any tools that can measure this aspect of quality :). This is where Agile approach with short iteration and frequent user feedback might improve quality (satisfaction) a lot.

b) low level of bugs - as single measure that shows you how well requirements were turned into code. This value can be measured and there are lot of tools that can help you. (JIRA, Bugzilla)

c) code quality - not related directly to level of bugs - but rather to difficulties in later modifications of code and product maintenance. This is more 'fragile' aspect of quality and you definitely needs a tool to measure it:

  • Cobertura (http://cobertura.sourceforge.net/) - unit test coverage tool - generally the more automated tests the easier to do anything with code without breaking it
  • PMD (http://pmd.sourceforge.net/) - static code analysis tool - the more readable is code the easier is to modify it

This aspect might be also described as Technical Debt (http://en.wikipedia.org/wiki/Technical_debt)(kind of compromise that were done on account of code quality, to be able keep deadlines for example)

d) another aspects might be scalability, performance, high availability there are also some general and also very specialized tools to help measure such aspect of products. Let me mention only :

  • Grinder (http://grinder.sourceforge.net)
  • Jmeter (http://jakarta.apache.org/jmeter/)
4

Depending on your definition quality is highly subjective, and therefore difficult to measure. As the project is being developed, it will be difficult to measure "How well the product meets the consumers needs".

One of the traditional measures of quality is the bug detection rate. This is relatively easy to measure, and somewhat less subjective. It is believed that a relatively fixed percentage of bugs will go undetected. Projects with a high rate of detected bugs will likely have a high rate of undetected bugs. The rate of bug detection will vary depending on the techniques used and skill of your testers.

There are a variety of bug tracking tools which can be used. Most will allow you to track bug counts over time. Aiming for zero bugs at all times may improve overall quality. One method to do this is "Fix bugs first". This may have multiple benefits, including lowering overall bug rate.

You may want to categorize bugs by source: Specification, Code, and Test. Expect relatively equal counts for each category. Tracking injection and detection phases may help estimate how many errors will remain. Tracking the testing methodology used may also be useful.

Usability may be considered part of product quality. Unfortunately, testing contaminates the testers, so you may need a source of new testers.

1
  • Of course, if "quality" is how well a product serves its consumer, the absence of bugs is not a measure of quality -- it could be bug-free and also value-free. Of course, it is hard to be bug-laden and valuable at the same time (certain microcomputer software products notwithstanding). May 31, 2012 at 23:11
1

Quality is a measure of how well a product serves its consumers.

If the consumer is a machine, then quality is likely very quantifiable and measurable. Issues like duty life, certified correct answers to anticipated questions, lack of defect, timing, etc are all relatively straightforward to measure for many cases, if hard to guarantee generally (but that's engineering).

If the consumer is a human or a human business, then quality starts well before product development, and will be measured in the pirate metrics (AAARR), and can be measured by a series of experiments on the working system, such as registering uses of features, customer ratings, monitoring social networks, usability studies, alpha test cycles, A/B testing of features, etc.

The development process can likewise be measured to see how well it serves its participants, such as with timely delivery, sustainable pace, non-turbulent flow of product, innovation, simplicity, component defect testing, product feature testing, and trending of productivity over time. These are all hard enough to do, but there is a lot of research.

Are we talking about software projects? There are volumes written, and many products for testing the quality of software (complexity, duplication, memory use, defect density, churn, and on and on).

0

What are possible and most effective methods of measurement of quality in a software development project?

The most effective method of measuring software quality is user feedback.

We're a small software development house known for our quality (it's worth pointing out that we assertively avoid estimating deadlines in favour of doing the job right - no matter how long it takes).

We use the following step-by-step process for each project:

  1. Evaluate the potential users / people working for your customer, and pick someone who understands the system needs and communicates well. We like to call this person the "product champion".
  2. Release early and release often, albeit make it clear that your software is still under development.
  3. Have your champion test your latest releases and send you regular feedback. If they do a good job, tell their boss (recognition goes a long way). If they do a poor job, find another champion.
  4. As the system nears completion / reaches a functionality milestone, get more users / all the users on it, so that they can test it too. Make sure all their feedback goes through the champion.

Bear in mind that we develop custom software for various clients. If, on the other hand, you're developing an off-the-shelf package, then you need to keep the software in-house until it's ready, but I'm sure you can adopt a similar strategy to the one listed above with a solid team of beta testers.

Then if you want to use metrics to analyse your ability to fulfil the above philosophy, conduct the following user survey:

  1. Are these users actually using the system? (sounds silly but you'd be surprised)
  2. Are they happy with it?

On a side note, we don't bother with any metrics / analysis tools that measure code quality, as this does not add any value to our operations. But that's easy to say when you're as small as us, so I guess it depends on the size of your company.

Good luck.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.