How do you reduce card size and limit scope creep in R&D tasks?

Question

Several of my team members do Research&Development work, which by nature involves a lot of exploration and uncertainty, and requirements are not always clear since we don't know what is out there.

For those reasons our R&D cards typically stay on the board for 2+ weeks, whereas our average cycle time from Priority to Done tends to be around 7 days. Those cards are also particularly vulnerable to scope creep.

How can we address this and make those cards flow faster? I have read Managing scope creep in Agile and How to minimize scope creep?, but I felt the answers to those questions didn't address contexts where uncertainty is much higher than in regular feature development.

Answer 1

TL;DR

If you're doing commercial R&D, then you should focus on directed development and manage the project around the research methodology chosen. Less directed research should simply be time boxed (financially or otherwise) so that the necessary reporting deliverables are handled appropriately.

If your research is directed, it should then be decomposable into manageable chunks. Whether those chunks are on Kanban cards or not, a structured methodology inherently combats story size and scope creep.

Analysis

Research is exploratory. However, good research is directed exploration. Consider some examples:

1. Take ice core samples from the Arctic to test for carbon levels.
2. Mapping a portion of the sea floor.
3. Vaccine development for a new disease.

All of these things are experimental or exploratory in nature, but they can still be decomposed into steps, phases, tasks, and time boxes. While pure research is generally less constrained to deliver rapid results, research should still be designed to eventually yield results, even if the result is an affirmation of the null hypothesis.

While you can use Scrum or Kanban to manage R&D, I think it's likely to be the wrong approach because the business goal of R&D is neither to improve cycle times nor throughput, but to identify the success or failure of a hypothesis as rapidly as practicable.

Even in areas like pure mathematics where some problems aren't well-aligned to typical project management approaches, the researchers should have a defined methodology or plan of attack for the problem. If not, then you simply can't measure progress or deviation from a plan, which is pretty much the point of project management as a discipline.

There's Always an Artifact

Even the most esoteric, non-results driven research is expected to produce some sort of artifact within a time box. That may be a research paper, a monograph, a scholarly article, or a symposium talk. Whatever that deliverable is, that's what you manage towards.

Imagine an endowment where an academic is given a $100,000 endowment to spend a year thinking about a math problem like P versus NP. Assuming no formal methodology, the patron probably still expects regular updates on the process, or on how the money is being spent (unless it's simply a stipend). They are also likely expecting a report on the results or lack thereof at the end of the funding period.

If you truly can't manage a given set of research goals as a project, then at least manage the research as a set of artifacts and time boxes to be effectively managed. Paid research always has deliverables; you might just have to work a bit harder to identify what those deliverables are, and how best to track them.

Answer 2

I think the very nature of R&D projects means there is no such thing as scope creep. You are trying to control that which does not exist. At least, not the way we define scope in other types of projects. In R&D, scope is controlled by the simple mechanism of running out of time and / or money or the patience of whoever is cutting the checks.

So if you are trying to better your seven-day average, then you have an artificial stop of work that starts to climb over that seven days or maybe 10 days, then redirect your staff to the next R&D task. Then @ThomasOwens' question remains: should you try to speed things up?