Firstly, there is no canonical answer to this, but there are some pragmatic solutions. Secondly, you can't estimate what you can't measure. You therefore need to devise a proxy metric to create your estimate or to graph your cone of uncertainty. Thirdly, the project's assumptions and risks (particularly including schedule risks) should be defined and documented.
Bound Infinite Ignorance
By definition, the scale of what you don't know extends to infinity. In other words, you can quantify what you do know, but you can't really quantify what you don't know.
Within any given problem domain, though, the question is "When will I probably know enough to get the job done?" That's more answerable; pragmatically, you estimate your team's level of ignorance on a subject by subtracting what they do know from what they think they need to learn, then estimate the level of effort involved in learning a sufficiency of that subject.
Spikes and Proxy Metrics
Use story spikes or proxy metrics for a first pass at estimating learning effort. An example of a proxy metric might be how long it will take the team to read 3,000 pages of documentation. That's not measuring learning directly, but it's an approximation of a measurable learning-related task.
Story spikes are another useful tool. For example, you might pick a minimalist, throw-away feature to develop for the purposes of estimating learning effort. You might spend four weeks trying to build the feature the same way you might take an open-book test, building, reading, and learning as you go.
Whether the feature is ultimately finished or not doesn't matter; nor does it matter if the feature is inherently useful. The goal is to get a feel for the time and effort that will be needed to master the knowledge domain sufficiently to deliver some set of defined features. Granted that it's only an estimate, it's still based on empirical results and will be more useful than an arbitrary guess.
Let's say that each team member can usefully absorb 100 pages per day of available documentation, including the time it takes to read it, try out some examples, and try to make sense of it within a cognitive framework. For each person to read the entire body of documentation without skimming would therefore take 300 business days, which is significantly in excess of the roughly 252 business days available each year.
Instead, let's say that you decide to build a toy project where the team will build a simplistic application that counts the microseconds between button presses. Each team member will focus on an area of specialization within the documentation, and try to read/learn/build the feature as a group within a four-week time box.
At the end of the time box, the team should have a sense of how easy or hard it will be to build the feature set defined for the project while learning as they go. Most importantly for your purposes, you can use this estimate to determine whether a year is a reasonable time frame or not for the team to deliver the desired feature set.
For example, the team could:
- Do a test project with the new technology.
- Use the results to estimate the Product Backlog.
- Use the team's historical velocity to estimate the schedule at an 80% confidence interval.
- Compare that schedule estimate against management targets to see if the schedule is reasonable and the pace of development seems sustainable over the life of the project.
"Cone of Uncertainty" Adjustments
Whether you use a proxy metric or a story spike, you also need to account for the team's uncertainty by presenting the estimate as a range. In software development, the cone of uncertainty is typically assigned as a "fudge factor" of 25-400% of the first estimate. (Boehm, B. Software Engineering Economics, Prentice-Hall, 1981.)
Management targets are often aggressive, and may ignore the team's scheduling estimates or the cone of uncertainty. That's why:
- The project manager must make it clear that an estimate is an educated guess and not a guarantee.
- The project's assumptions are clearly documented.
- The uncontrolled risk associated with the uncertainly must be documented as an accepted risk that's owned by the management team.
- The project has controls to detect when the projected schedule is out of tolerance.
The key here is to ensure that the uncertainty and lack of domain knowledge is factored into the project plan. As long as you don't hand-wave it, estimating your team's uncertainty can serve a useful purpose in helping the organization to define a realistic project schedule.