What is a good way to set control limits on a schedule?

Question

I want to try to track project progress using Earned Value Management approaches, mainly for my own education rather than out of a pressing need. The metric used to define planned value (PV) and actual work completed (Earned Value (EV)) is "% Complete" as generated by MS Project rather than a dollar value, which is OK because schedule is the main thing that I can track due to enterprise constraints I have to work within.

What I am interested in is establishing robust upper and lower control limits on the schedule, but I find that these limits get wider later in the schedule compared to earlier on if I use something simple like setting limits at +- 10% of PV. This reflects the real-life uncertainty of precisely when things will happen if they are far into the future, which I am OK with. But if I'm setting control limits I probably want those limits "tighter" the further into the future I look because the impacts of slippage are also greater.

I have considered setting arbitrary control limits (e.g. +- one reporting period) but would be happier if there was a more objective/"mathy" approach. Does anyone out there have any suggestions?

Answer 1

By control limits, do you mean an acceptable tolerance for cost and schedule variances? If so, 10% is the typical benchmark, at least commonly accepted with those systems that are ANSI compliant, like in the DoD. In other words, your CAC is acceptable if within + or - 10% of your BAC. You can certainly tighten it up based on your project needs.

To keep terms straight, PV is planned value, EV is earned value (which you have as AC), and AC is actual costs. And everything is measured in dollars.

EV is truly a cost control tool. Schedule control using EV has problems. Earned schedule is an approach that resolves those problems. Look into that, as well. My experience there is only book read; never done it for real, yet.

Answer 2

This may not quite answer your question but one approach to your problem might be to look at the use of control charts, which concentrate less on whether a particular process is within its bounds (+/- 10% or whatever upper and lower bounds you select) and more on identifying patterns of change between those bounds.

It's typical to use the seven run rule, where the presence of seven sequential measurements that are below or above the mean, or are consistently increasing or decreasing, indicates that a process is 'out of control' and requires action.

This might be useful in your case because it would allow you to slacken and tighten your upper and lower bounds but still identify problems with the process that you are attempting to control. In my current work environment I find that the identification of patterns in cost or schedule accuracy is more useful than a measurement at a given point in time. The random 'outliers' in the schedule (maybe someone was sick or a vital piece of hardware failed) are still important but I'm generally more concerned with patterns over time. If I see, for example, that our velocity (we're using Scrum) is consistently decreasing I know that I've got a bigger, possibly systemic, problem.

I think your +/- 10% rule is reasonable but if month-on-month your accuracy was -1%, -2%, -3%, -4%, -5%, -6% and then -7% I'd be worried. If it were 6%, -2%, 4%, 7%, -1%, 9% and then 3% I'd still expect to have to do plenty of explaining but I'd be much more comfortable that the causes of these inaccuracies were random rather than as a result of a wider problem within the project.

Answer 3

I like the answers so far, so I'll approach it from a different angle -

Your question for me as two parts - one about control limits, and another about a "mathy" way to figure it.

But for these two questions there's really a single answer. EV is at its core simply a more objective, "mathy" way to measure and monitor progress. So you don't need any more math, just use the formulas that EVE uses already.

As for the control limits, again that's what EV is for. It provides an objective assessment, which tells you how you're doing. But 'how you're doing' is a function of both your planning and your risk tolerance/expectations. These limits are or your own design, based on the level and confidence of your planning, execution, assumptions, risk tolerance, contingencies, experience, etc.

If it's work you've never done before, then maybe a variance of 20% is acceptable, A) because you don't know enough to be more accurate (and you know that about your planning), and B) because you added some contingency to cover a possible variance of that size.

On the other side, a 5% variance could indicate trouble because it's routine work with hard data supporting the planning.

The point is, control limits are related to the project not EV. And EV "is" the 'mathy' process.

Hope that helps.

Answer 4

Acceptable Tolerances Vary

I understand the core of your question to be:

I am interested in is establishing robust upper and lower control limits on the schedule[.]

Acceptable schedule tolerance varies based on project requirements and specific business objectives, and I'm unaware of an ex cathedra formula for divining acceptable tolerance. In this case, I think your best bet is to evaluate what level of schedule risk a given project can tolerate before its planned value drops unacceptably, and then periodically track your current earned value against that metric.

Put another way, you can track your Schedule Performance Index, calculated as SPI = EV / PV or SPI = BCWP / BCWS, with an ideal goal of SPI >= 1.0.

Problems with SPI

Using the SPI doesn't always get you where you want to go. Wikipedia describes the limitations of SPI calculations thus:

The use of SPI in EVM is rather limited in forecasting schedule performance problems because it is dependent on the completion of earned value on the Critical Time Path (CTP).

Limitations of EVM

Wikipedia also describes some limitations of earned-value management as follows:

EVM has no provision to measure project quality, so it is possible for EVM to indicate a project is under budget, ahead of schedule and scope fully executed, but still have unhappy clients and ultimately unsuccessful results.

Potential Alternatives to "Pure" EVM

One potential alternative to traditional earned-value management may be Earned Schedule (ES) management. The topic is also very lightly covered by Wikipedia.

I have no direct experience with ES, so your mileage will probably vary. However, I personally believe that something other than pure EVM is required to properly control scheduling because Earned Value Management simply has other design goals.

There's no reason you can't have multiple controls on a project. In fact, I firmly believe that one should. With that in mind, I'd make EVM one project control methodology among many, rather than putting all your eggs into one basket.