How to estimate without relying on SMEs to estimate based on similar projects?

Question

I create MS Project schedules for the government and recently moved to a new position on a staff where I am being asked to develop and control schedules for tasks that are rather vague, i.e. create a schedule that will tell us how long it will take to develop a regulation and how to resource load it. My previous schedule development experience was in creation or fielding of tangible items: heavy equipment, weapons, software/hardware systems, etc.

How should I proceed, other than relying on the SMEs to estimate previous durations or work hours for similar documents?

Answer 1

TL;DR

1. Be extremely aware of the underlying assumptions implicit in your estimation process.
2. Only the people who will actually perform the work are qualified to estimate how long it will take them to perform a given task within the context of your project..
3. You should be developing your estimates with the active cooperation of the people who will actually be producing the work increment.

Beware of False Equivalencies

Any suggestions on how to proceed other than relying on the SMEs to estimate previous durations/workhours for similar documents?

One of the central tenets of effective estimation is that only the people who will actually perform the work are qualified to estimate how long it will take them to perform a given task. Rather than looking at other projects, or asking subject-matter experts who may not be directly involved in the project, you should be developing your estimates with the active cooperation of the people who will actually be producing the work increment. By trying to bypass task-performers for the estimation, you run the risk of applying invalid estimates to your schedule.

By applying estimates of tasks performed by other people, you are making several implicit assumptions:

1. You cannot trust the estimates of the task performers on your project.
2. The work done by someone else is equivalent to the work to be done within your project.
3. The skills and resources within your project team are equivalent to the "prior art" project you are using as your baseline.
4. The context in which the work will be done is functionally equivalent between projects and organizations.

Think of it this way: if Alice could perform a task in one day, but Bob will take three days, it really doesn't matter that it takes Alice 66% less time to perform the task because she isn't on your project team—Bob is, and accurate scheduling is about estimating based on Bob's capabilities to get the work done.

Basing work-effort on something other than the skills and resources actually at hand is really setting a management target rather than developing a valid estimate. It may simply not be a valid estimate for your team, and assuming that it is will likely lead to a post-mortem blame-game rather than a finished set of deliverables.

Your mileage may vary. That's the point. :)

Answer 2

Do you have access to other similar deliverables? In some cases the names of contributors are listed. From that and a bit of digging you may be able to roughly determine the size and skillsets required for the team.

Even with SME input developing something like regulations is going to be difficult at the best of times because it generally involves getting a committee to agree on something. Timeboxing discussion is going to be very important.

Worst case scenario is you could make up some numbers and milestones for your first prototype/draft/chapter. A small piece of work that is still a deliverable. You won't be anywhere close unless you are really lucky, but it will give you some experience to know for the next go-round though.

A bit of advice I could provide is that if the estimates are too big, the work/creep will fill the extra, see Parkinson's law.

Answer 3

I manage projects for the government; (and am also Mark in Virginia, oddly enough), but I don't find this to be a problem.

How long does it take to develop a regulation. Depends entirely on the scope of the regulation - law will take longer than policy which will take longer than guidance.

But fundamentally the steps are the same as developing code (which tech wonk was it who quipped that code is law for the internet?) You have to define the scope/objectives, identify the constraints, identify the stakeholders, refine the scope & constraints based on stakeholder input. That's the plan. Then you design the policy, develop text based on the design and then coordinate and test. You should have SME's for each of those stages, just like when writing code.

I've had success in devising the plan and then informing the stakeholders that of the plan and then driving consensus within the plan. We left a lot of potential topics on the floor because we didn't believe we could reach consensus within the time limits we'd set.

Answer 4

You have a "soft" product that could take anywhere from a few days to years to create, you are new to this work, and you do not want to rely on SMEs. This is very simple to schedule because neither accuracy or precision of either the estimate or planning value is possible. All you need is one line--create regulation--and load everyone's name in that package and then create the duration to equal whatever arbitrary deadline you were given. There's no way to track against it so don't bother trying.

Seriously, you need SMEs who will be part of the regulation developing process. And even then, your estimate range will be huge so the accuracy of your planning value will be rather low anyway. If you are used to tangible products that are built over and over again on multiple projects, where your forecasts have a high degree of reliability and validity, you will have culture shock with a more soft product such as this.