Session 4C: Beyond the Matrix: The Quantitative Cost and Schedule Risk Management Imperative

Dr. Christian Smart is a Senior Programmatic Risk Analyst with NASA’s Jet Propulsion Laboratory. He has experience supporting both NASA and the Department of Defense in the theory and application of risk, cost, and schedule analytics for cutting-edge programs, including nuclear propulsion and hypersonic weapon systems. For several years he served as the Cost Director for the Missile Defense Agency. An internationally recognized expert on risk analysis, he is the author of Solving for Project Risk Management: Understanding the Critical Role of Uncertainty in Project Management (McGraw-Hill, 2020).

Dr. Smart received the 2021 Frank Freiman lifetime achievement award from the International Cost Estimating and Analysis Association. In 2010, he received an Exceptional Public Service Medal from NASA for the application of risk analysis. Dr. Smart was the 2009 recipient of the Parametrician of the Year award from the International Society of Parametrics Analysts. Dr. Smart has BS degrees in Mathematics and Economics from Jacksonville State University, an MS in Mathematics from the University of Alabama in Huntsville (UAH), and a PhD in Applied Mathematics from UAH.

In the modern world, we underappreciate the role of uncertainty and tend to be blind to risk. As the Nobel Prize–winning economist Kenneth Arrow once wrote, “Most individuals underestimate the uncertainty of the world . . . our knowledge of the way things work, in society or in nature, comes trailing clouds of vagueness.” The resistance to recognition of risk and uncertainty includes project management. As a colleague of mine aptly put it, “Project management types, especially, have a tendency to treat plans as reality.” However, projects of all types – weapon systems, robotic and human space efforts, dams, tunnels, bridges, the Olympics, etc. – experience regular and often extreme cost growth and schedule delays. Cost overruns occur in 80% or more of project development efforts and schedule delays happen in 70% or more. For many types of projects, average cost growth is 50% or more and these costs double in more than one in every six. These widespread and enduring increases reflect the high degree of risk inherent in projects. If there were no recurring history of cost and schedule growth, there would be no need for resource risk analysis and management. The planning of such projects would be as easy as planning a trip to a local dry cleaner. Instead, the tremendous risk inherent in such projects necessitates the consideration of risk and uncertainty throughout a program’s life cycle.
In practice the underestimation of project risk manifests itself in one of four ways. First, projects often completely ignore variation and rely exclusively on point estimates. As we will demonstrate, overlooking risk in the planning stages guarantees cost growth and schedule delays. Second, even when variation is considered, there is often an exclusive reliance on averages. We will show that there is much more to risk than simple averages. Third, even when the potential consequences are considered, risk matrices are often used in place of a rigorous quantitative analysis overreliance on risk matrices. We will provide proof that qualitative risk matrices underestimate the true degree of uncertainty. Fourth, an often-overlooked weakness in quantitative applications is the human element. There is an innate bias early in a project’s lifecycle to perceive less risk than is present which leads to a significant underestimation in uncertainty until late in a project’s development, at which point many of the risks a project faces have either been avoided, mitigated, or confronted. Even the application of best practices in risk analysis often lead to uncertainty ranges that are significantly tighter than indicated by history. Reasons for this phenomenon are discussed and the calibration of risk analysis outputs to historical cost growth and schedule delays is presented as a remedy.