There is a relationship between the level of knowledge uncertainty at the beginning of a product development project and the number of technical setbacks or failures that can be expected during its development cycle. Recovery from these in-scope failures increases the cost of the project, and that is the cost that must be protected by Management Reserve (MR). The correlation between uncertainty, failures and management reserve is notionally shown in the figure below. Low uncertainty can be considered as product development within the “Known-Known” knowledge domain. As more uncertainty is accepted, the product development effort moves into the “Known-Unknown” knowledge domain, where more failures should be expected, demanding greater MR. More cutting edge product development ventures lead to high uncertainty (the “Unknown-Unknown” knowledge domain), more expected failures expected and the need for even more MR.

Just to be clear, I’m defining MR as the budget set aside to cover the additional costs incurred as a result of the uncertainties associated with the basic requirements and scope of the project. I’m not talking about budget reserve to cover the increased cost of worse than expected performance or the sins of bad estimation. That’s the subject of another blog.

My thinking on this subject has been primarily inspired and informed by the work of Dr. Glenn Havskjold, PhD of Pratt & Whitney Rocketdyne (PWR). Dr. Havskjold has developed the idea of the “Prodecol” (Product Development Control Lever) methodology, for use in estimating the cost of the inevitable test-fail-fix (TFF) cycles encountered during development of complex, technically innovative products. This methodology is described in his American Institute of Aeronautics and Astronautics papers, AIAA-2009-5436, Parts 1, 2, & 3, “Developing Innovative Products on Budget and on Schedule”, published in August 2009. He defines a metric called the Technical Uncertainty Factor (TUF) to quantify the level of uncertainty or risk at the beginning of a product development cycle. For his business, the TUF is a composite of the uncertainty related to the maturity of technology and design, the level of design complexity, and the overall knowledge of, and experience in the operating environment of the design.

By conducting expert opinion assessments of the technology maturity, design complexity, and operating environment uncertainty, a composite TUF is established for the product at the beginning of its development. The product TUF is then mapped to the number of significant TFF cycles encountered during product development. Dr. Havskjold found that for his business, there was a distinct correlation between the TUF and the resultant number of failures during the development cycle. Further, by reviewing the business financial data he was able to determine the average cost of fixing these failures. He combined all of this research into what he calls the Prodecol Chart. This chart, used in conjunction with TUF criteria and the product development history and technical expertise of a business, offers product development teams a way to predict a major portion of the development costs.

I believe that the same approach can be used to address the broader uncertainties of project management, including those associated with the market environment; human, physical and financial resource availability and stability; and customer relationship management to facilitate the estimation of the complete project MR. Specific details of the uncertainties, failures and related cost of fixing those failures will be different for every business. However, I believe that an approach like this can be useful for any business involved in high complexity, technically innovative, product development work.

The following is a set of suggested steps in building an MR estimating tool for a business:
• Every business must examine it’s own uncertainty vulnerabilities to define a suite of characteristics that their by subject matter experts can use to build a composite uncertainty level. The table below offers some thoughts on the kinds of uncertainty drivers that might be considered and how to characterize their level of uncertainty.

• Panels of technical and programmatic subject matter experts (SME’s) must be formed to go back and assess to the best of their ability, the uncertainty levels that existed prior to the development of past projects.
• Quality and cost records associated with those same past projects must be gathered and reviewed by SME’s to identify the number of significant technical and programmatic failures, and determine the average cost of resolving them.
• Mapping the Uncertainty Level and No. of Failures should reveal the correlation curve for product development in that business, and applying the average cost to fix each failure completes the tool.
• The tool can then be used by SME’s to evaluate the Uncertainty level of new proposed projects to determine the number of expected failures and the MR required to resolve them.
• In addition, the tool can be used in reverse to help plan the program. If the MR budget is fixed, this same tool can then define the allowable number of failures and the allowable uncertainty at the beginning of development. Any gap between the current uncertainty and the “affordable” uncertainty, then can be used to define both the nature and broad scope of pre-development technical and/or programmatic uncertainty mitigation effort that could be performed to help ensure that the formal development program will meet it’s cost objectives.

Clearly, this isn’t an off-the-shelf tool for a business. They must put their own SME talent, and cost and quality records to build their own tool. I firmly believe though, that once this effort has been completed, the business and its project managers will have a valuable tool that will make them more successful in the future.

As a final comment I’ll add that any historically based estimating tool like this, must be kept current with the latest product development history. This is a critical knowledge management task for the project