How can we manage schedule risks brought into a project by 3rd party software?

Question

Working in web-development, I face a common situation when clients want to use 3rd party software as a starting point of the project. That could be a CMS, an e-commerce site, or something else. Motivation is usually budget-driven; it seems to the client that having a web site built atop of some off-the-shelf product will cut costs. It is not always true. For example:

• The client (often a non-technical person) was told that it's super-easy to build a 3-page website using a famous CMS. However, nobody knows what it takes to build a 100-page website, with a blog feature and 3 levels of paid membership.
• The team doesn't know the CMS before the project starts, so adding requested features become a hard task, and the existing CMS code often stands in the way of effective development.
• It also may turn out that there's no decent documentation. The team would create similar features faster without that CMS, but the client still insists on using it.
• Adding custom design to the CMS-based site may cause trouble for HTML developers, requiring development of non-trivial code (for example, an obscure menu system).
• It may turn out that the CMS is not compatible with the hosting that was usually used for deployment, the CMS has internal flaws preventing scalability in production, or that the solution requires enormous hardware setup to operate properly under load.

That leads to questions I'd like to ask:

• How can we convince a client that his solution should not require a custom CMS, or that it may take longer to develop a custom solution using pre-existing code? Clients usually has arguments like 'the biggest sites on the Internet are using this CMS,' but we don't really know how much effort was put but those biggest sites to run the CMS properly.
• How to include 3rd party (or pre-existing) code into our initial estimation and risk-management calculations? Any numbers and techniques are welcome.

Answer 1

TL;DR

You aren't really asking a risk-management question. At heart, the question is really about how to estimate projects where you have no baseline values. A common solution is to use a pilot project to generate planning values as inputs to a full-scale project.

Ability to Estimate

Your project's ability to estimate accurately (or at all) depends a great deal on the team's experience with a specific knowledge domain. If you have zero experience with a given technology or project, then your project schedule is a black box; there is no way to estimate it other than guesswork.

Even assuming some knowledge, all projects have a cone of uncertainty. Generally, you will plot a preliminary schedule based on some planning values, often adjusted by a fudge factor representing the amount of uncertainty the team has about the project. For example, inexperienced teams might multiply their initial productivity values (e.g. velocity) by 0.6, or pad their schedule by multiplying time estimates by 1.4.

The actual value of the fudge factor is less important than the clear knowledge that it is an adjustment to the planning values. Remember: an estimate is an informed opinion, not an iron-clad guarantee.

Pilot Projects

If your project involves skills outside the team's current area of expertise, or if there's a need to perform some tasks in order to calibrate the schedule for a larger project, then you need a pilot project.

For example, rather than trying to estimate a 100-page website with a tool-chain no one has experience with, you might plan a pilot project for a 5-page web site. The important deliverables from a pilot project like this aren't actually the web pages; the real deliverables are experience with the proposed solution and better input values for the larger project.

For instance, you might do your 5-page study in two weeks, but discover that the solution won't scale the way you need it to. Or, you might find that initial setup takes a week, and developers can average a page a day thereafter.

Whatever the results, you use the data from the pilot project along with a new set of more-informed fudge factors to generate the schedule or estimate for the larger project. This may or may not be linear: a 100-page web project isn't necessarily 20 times the complexity of a 5-page project. However, your cone of uncertainty should be smaller after a pilot study, and your estimates will be much more likely to fall within the boundaries of acceptable variance.

Answer 2

Question #1: this is a business case. To convince a customer of anything, you need to build an argument. Develop a case that exhibits several alternatives from which the customer can choose, the benefits/costs/risks of each alternative, and your recommendation. Your recommendation needs to be supported and substantiated with industry standards and facts that you find in your research, not just your opinion likely biased by your pursuit of profit. Your entire presentation needs to be supported by the relationship you have developed with your customer. You would not be just a vendor pitching your wares but a trusted adviser.

Question #2: This is a risk management question. And you would apply normal risk management techniques. There are few projects out there that are not dependent upon something external that could adversely or favorably impact your schedule and budget. So this is normal; welcome to project management 101.

Research what you can about the external dependency, e.g., reputation in the market place, history of being late or on time, references, etc. What you can't learn, you need to draft an appropriate assumption (a source of risk). You load your schedule with the external milestones based on that assumption so you can communicate clearly how your performance would be impacted if those assumptions do not bear fruit. You can load schedule reserves to accommodate something coming in late and, most of all, you communicate clearly to your customer where you have dependencies, the risks you and your customer must accept, and any updates to what you are observing.

You can only control what you can control. Most of the very random variables that affect our performance, both favorably and unfavorably, are way out of our control, which is very counter intuitive to all the business type A's managing projects, but such is life. All you can do is watch for them, plan for them (risk management), communicate what you are observing in a timely way, and accept the fact this is all very normal.

Answer 3

To answer your second question - you would apply risk management techniques. There is a lot of material on the internet regarding the entire process. There is a good worked-through example of risk management on a software project in the Risk Register User Guide.