3

Earned Value Management (EMV) sounds like a great and objective method for reporting.

Precondition for applying EVM is that tasks or results are well defined and well distributed during the project.

I'm wondering if it is possible to apply EVM to projects with a high probability of change, e.g. R&T projects.

Within R&T projects unforeseen iteration may appear or a former closed task has to be reopened due to new findings.

Those aspects are usually not understood as risk, due to the fact that everybody expects them to appear.

Regarding R&T vs. R&D: For sure, EVM is successfully applied on lots of development projects. From my point of view, EVM is a tool to visualise results or effects caused by the usual uncertainty of projects managed by waterfall model (e.g. delays, changes in estimate at completion).

Regarding the TRL, R&D is situated somewhere between 4 and 8 (my impression). In compare to that, R&T is situated somewhere between 1 and 6 (again my impression).

In addition, I state that the level of uncertainty decreases with higher TRL: Ones estimates get more robust regarding project planning aspects (e.g. time, schedule, tasks) as higher the TRL is.

Those are quite general statement and for sure not always correct or applicable. Nevertheless they reflect (at least my) general observations.

Having this, to which level of uncertainty is EMV applicable or: Can EVM deliver a benefit to projects being subject of regular change?

Improve this question
7
  • 1
    EVM is used extensively on programs that have a large R&D component such as building future aircraft, space craft or robots. For example, it us used extensively on Defense projects and NASA programs.
    – Mark Phillips
    May 18, 2015 at 1:20
  • @MarkPhillips so the answer is just do it as ususal?
    – Tob
    May 23, 2015 at 13:27
  • Not sure what you're referring to as 'usual' but classic EVM is used every day on very large programs with large R&D components. These often include planning packages which account for work that is not, as yet, as sharply defined as other work. Is there a particular aspect of R&D and EVM that's driving the question?
    – Mark Phillips
    May 27, 2015 at 0:56
  • @MarkPhillips There is no additional Aspekt than the ones I raised in my question: Is EVM applicable regarding R&T projects. I think in compare to R&D the level of uncertainty is higher on R&T. I would accept an answer stating that it is applicable. Nevertheless I would be happy about some reasoning or lessons learned.
    – Tob
    May 29, 2015 at 10:53
  • 1
    That's helpful. Sounds like you're talking about basic research/science. Yes, EVM is applicable. The uncertainty you're talking about influences how you define work packages. Where uncertainty is only about execution the wp is, for example, complete a 3 meter wall. Where uncertainty is all tech/science risk the wp is something like complete x test. Pass or fail is irrelevant. The goal is to complete the test.
    – Mark Phillips
    Jun 1, 2015 at 3:21

3 Answers 3

Reset to default
1

A research project is by definition innovative and, for this reason, uncertain. Especially at the beginning, researchers might have only a rough idea of the hypothesis that they want to validate.

Novelty, complexity and uncertainty make the use of an Agile approach a good choice in the Research & Technology context.

There is the possibility to use an Agile implementation of the Earned Value Management principle, known as AgileEVM (that adopts values defined in Scrum). I have never tested the validity of using EVM on my Scrum projects, but it can be worth reading this paper to know more about it. Here it is the Abstract:

Scrum is an Agile Project Management Framework. This framework specifically focuses on maximizing Return on Investment (ROI). Scrum, however, does not define how to manage and track costs to evaluate actual ROI against the vision. A reasonable cost measurement that integrates with Scrum would help provide an additional feedback loop.

We adapted Earned Value Management (EVM), using values defined in Scrum. The result is called AgileEVM (Agile Earned Value Management) and is a simplified set of earned value calculations. From the values in Scrum, we derived a release date estimate using mean velocity and from this equation, generated an equivalent equation using traditional EVM techniques, thus establishing the validity of using EVM with the Scrum framework. Finally, we used this technique on two projects to further test our hypothesis. This investigation also helped us determine the utility of AgileEVM.

An extract from the Conclusions paragraph:

For the ScrumMaster, it is clear that metrics that are familiar go a long way to ease the discomfort that new, unfamiliar methodologies can induce. The analysis that AgileEVM provides, along with the burndown method, helps to substantiate intuition and provides executives with quantitative data in a consistent manner. The cost analysis, with its forecast Estimate at Complete and Estimate to Complete are valuable to Agile stakeholders calculating estimated ROI. Agile stakeholders who are responsible for making budget decisions find this information extremely valuable.

Our recommendation is that AgileEVM be used in conjunction with the Burndown chart and team velocity as supporting data. One important caveat is that change is expected on Agile projects and so the AgileEVM metrics are derived from what is true at each Sprint boundary.

Providing the team and Agile stakeholders with useful and understandable data is vital to the "rudder" with which the Scrum team steers toward better processes and continuous improvement. By providing the burndown and AgileEVM metrics together, the team is better equipped to succeed.

Improve this answer
1
  • Quote a good paper. The presented approach tries to gain the EVM benefits on Scrum managed projects. So it's not classical EVM but the resulting estimations are similar. Thanks for the answer. I didn't "accept" your answer until now because I am still trying to figure out if the conventional EVM will work for R&T projects.
    – Tob
    May 23, 2015 at 16:25
1

EVM is designed to signal schedule and budget variances and treats both early and late finishes as undesirable. That makes EVM a poor fit for research projects since tasks cannot be finished or started early without signaling a problem.

In a project of known scope and with activities of well understood value, earned value management can be an effective source of signals for a management team. For example, a manufactured good can accrue value as it is refined from raw material to a finished good along a predefined manufacturing process.

Scrum is a better fit for research, since it seeks to maximize value given available resources. Early finishes are desirable in Scrum since they allow a team to add additional work scope. By monitoring a research team's velocity it is possible to forecast what backlog items can be completed under the available budget and schedule. If necessary, the backlog can be reprioritized to increase the chances of the research team completing the highest value research objectives.

Improve this answer
5
  • 1
    EVM does not do anything to minimize variances. It only calculates variances. Undesirable is a judgment call from the person interpreting the variances. A very early finish variance is not undesirable on its face but rather may indicate an undesirable cause, such as extremely weak quality in delivery.
    – David Espina
    May 17, 2015 at 10:36
  • @DavidEspina would you agree that in implementations of EVM with a linkage to the organization's accounting system that an early finish variance can be a disincentive? If budget is allocated specifically to work packages, then an early finish can have the consequence of a team losing budget back to a project or program reserve. Unless teams have the authority to access and reallocate savings from under-runs, the incentive for managers will be to finish on time in order to preserve budget for their local team.
    – Michael Hogan
    May 17, 2015 at 15:18
  • 1
    FV is time. Cost variance and variance at completion are budget. If a work package finishes with a favorable cost variance, i.e., cost at completion is less than budget at completion, those dollars should go back to the project. If the work is finished, why would anyone associated with that work package care if those dollars are returned? After all, the work is finished and they don't need it anymore.
    – David Espina
    May 17, 2015 at 15:26
  • @DavidEspina Losing the dollars becomes an issue in multilevel organizations where those work packages represent the funding allocated to teams. If work packages close and funding is pulled from a team back to the program or project reserve, then it is possible for team members to be left without work.
    – Michael Hogan
    May 17, 2015 at 22:23
  • 1
    That doesn't sound like a project to me, but I'll take your word for it. :)
    – David Espina
    May 17, 2015 at 23:55
0

For me this comment seems to answer my question best:

EVM in Research and Technology

I'll keep this answer as accepted until the comment is converted to an answer...

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.