4

Is there a theory and/or study backed way to measure/track and forecast research activities?

As it seems clear that R&D must be treated differently than production work for example, I could not find a reasonable metrics to track and forecast research activity.

I do find work on the necessity of tracking and forecasting research (e.g.Industrial Research Performance Management), but I do not find any description on how to do it.

The reason for my question is that research seems to be to unpredictable and non-linear in order to be measured in the same way as e.g. the production of a car.

So let's set up a case. Say a company has 30,000,000$ of research budget per year and the budget-owner wants to see where attention (and possibly improvement of the research process) is needed. The complete budget is divided in smaller projects. Which tools/metrics/processes are available to aid the budget-owner to see where projects are performing in a way that improvement (return on invest, speed or resource loading) is possible?

  • 1
    what industry are you in? – ashga Sep 21 '15 at 9:52
  • I an working in the aerospace industry. Mainly focusing on Materials and Aerodynamics. – rul30 Sep 21 '15 at 10:15
  • are you using any particular methodology in your projects? – ashga Sep 21 '15 at 11:32
  • 1
    The agile software development methodology Scrum was inspired by techniques to organise unpredictable research into Polymers. Software development, as a design process, has many similarities to research. Check out the techniques used to track velocity in agile software development. – Nathan Cooper Sep 21 '15 at 15:16
  • 1
    Closely related to the original question before OP edits: In Scrum, how to estimate research stories? – Todd A. Jacobs Sep 22 '15 at 7:25
3

The reason for my question is that research seems to be to unpredictable and non-linear in order to be measured in the same way as e.g. the production of a car.

That is the whole basis of R&D. Ultimately your ability to forecast progress on a project is inversely proportional to how "researchy" it is. In other words, if you are early in your discovery effort there are too many unknowns to adequately predict progress. In contrast, once you get to the point where progress can be predicted you aren't really doing research per se.

So what can you do in terms of metrics/methodology? I think basically you have to define what success and failure look like up-front and track appropriate metrics to assess whether a project should be killed. In pharma it is simple as success/failure is typically defined as showing statistical significance in each of a series of studies/trials.

For your industry it may be more complex. Your best bet is to look at your organization's past projects. Categorize them as successes or as failures, and figure out what metrics made each a success/failure. You should then be able to plot an ongoing project somewhere in the spectrum of success/failure based on those metrics.

  • Would you happen to have some examples of those investigations at a pharma company? – rul30 Sep 21 '15 at 23:29
  • Go to clinicaltrials.gov, pick your favorite disease and look through the (maybe thousands) of human trials under the primary and secondary endpoints to see what they measured in terms of success metrics. Success is almost always a statistically significant difference from placebo/control, with statistical significance conventionally defined as P < 0.05. – Doug B Sep 22 '15 at 13:51
  • so basically you are saying that there is no umbrella metrics that exist in every field/industry? – Ooker Dec 31 '16 at 10:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.