Can EVM be Used to Track and Manage Quality?

Question

Currently, I think I've got pretty good tools when it comes to measuring Budget (enough and more Excel docs that are regularly updated), Scope (against a mutually agreed upon statement of work), Time (detailed time records). I like that these can be all integrated in an EVM (Earned Value Management) system. However, I would also like to measure and track Quality.

Does it make sense to use EVM in relation to Quality? Quality, in this case, is briefly defined as completion of most important features, reported bugs per feature, and customer feedback (based on a scale of 1-5 on completed items).

Here's a quick description of the project itself: Custom Web Application, including mobile app development (price comparison software) Budget: $200K, Length: 6 months

Answer 1

Earned Value Management Isn't a Quality Control Framework

In the "Limitations" section for EVM, Wikipedia clearly states:

EVM has no provision to measure project quality, so it is possible for EVM to indicate a project is under budget, ahead of schedule and scope fully executed, but still have unhappy clients and ultimately unsuccessful results.

While one could argue that a well-controlled project is less likely to have quality issues, the fact is that any effect on quality will be (at best) a tangential side-effect. It may be possible to shoehorn quality controls into EVM by by pegging various earning rules to ancillary quality controls—in much the same way that agile methods bake quality controls into the "definition of done"—but explicitly controlling for quality is simply outside the scope of EVM's design goals.

Comparison with Six Sigma

Contrast Earned Value Management with another framework like Six Sigma, which is explicitly designed for quality control. For example, the Wikipedia entry says (emphasis mine):

Six Sigma seeks to improve the quality of process outputs by identifying and removing the causes of defects (errors) and minimizing variability in manufacturing and business processes. It uses a set of quality management methods, including statistical methods[.]

What It All Means

No one says you can't perform quality control while using Earned Value Management. It just means that EVM, by itself, does not control for quality.

Answer 2

Regular reporting of status within the team is a tool to track performance, and is probably the most commonly used. Unfortunately it isn't a mathy approach, which can cause some stakeholders undue angst.

CodeGenome provided some links regarding Earned Schedule in his answer to a question of mine. Willl also provides links regarding control charts in his answer to the same question.

If you are looking at Agile approaches there is the concept of burn-down charts, which is conceptually similar to EVM.

When looking for a tool to use remember that tracking project performance is a means to an end, so whatever tool you use needs to do well only those things you need it to do well. So think about the key aspects of your project that need to be monitored (scope? schedule? budget? benefits? quality?) and figure out what your ideal tool/process/approach needs to offer before you select a tool. Maybe you need to use several tools, maybe just one. It all depends on your project's needs.

Answer 3

No, at least not directly. Earned value is strictly a cost control tool and, to a slightly lesser degree, a schedule control tool. However, what EV provides are early clues of an accruing variance and, with analysis, markers that allow you to predict with a good degree of confidence where you are headed. The result of those markers is the action to find out why you are accruing variances (either favorable or unfavorable), which may enable you to uncover issues with development as it relates to quality. But you have other tools, e.g., control charts, inspections and audits, to directly uncover quality issues and threats.

Answer 4

I think this is an important issue; when project managers actively measure time and cost, but not quality, it is easy to guess which one suffers. The fact the project managers themselves are measured on time and cost more closely the quality does not help either, particularly where organisational culture strongly focuses on short-term efficiency at the expense of effectiveness, (which in many cases winds up increasing time and cost due to scope changes and rework). I have been experimenting with a solution. What I have done is attach several key effectiveness criteria to each work package. (As a 'comment' on the cell containing the work package name in Excel). This is not integrated enough to provide data or metrics at this stage, but at least it opens a dialogue with stakeholders (especially senior stakeholders) on what is required to be effective and provides a benchmark to assess effecfiveness against.