Abstract
Several decades after the mechanics of quantitative uncertainty analysis (QUA) for risk assessment and regulatory cost analysis were developed and refined, QUA still rarely reaches the minds of decision-makers. The most common justification for this situation is that “decision-makers want a number, not a set of statistical distributions.” This may be an accurate assessment of their druthers, but one obvious though perhaps impractical retort is to say that if decision-makers insist on misleading point estimates, then we need new and better decision-makers. This article offers a way out of this dilemma. Decision-makers do not have to understand (or even receive) all the information contained in a complete QUA, but they do have to drive the QUA. They need to instruct analysts how to approach the phenomena they analyze (parameter uncertainty, model uncertainty, interindividual variability, offsetting and second-order effects, and the monetary value of future uncertainty reductions), they need to insist that uncertainties in cost be treated a priori as exactly as important as uncertainties in risk, and—even more importantly—they need to instruct analysts which estimator(s) to seek, report, and explain. Here we offer 10 detailed principles to guide decision-makers into a new relationship with risk and cost analysts—10 observations about how “eyes wide open” point estimates can vastly outperform point estimates handed to the decision-maker without context, justification, or honesty about the value judgments they impose upon the decision. A decision-maker who explains “I chose Option A because its benefits of 2.345 exceed its costs of 1.234” can be replaced by a dollar-store calculator. We need decision-makers who can say “I chose Option A because the spectrum of benefits it likely offers, to these citizens, considering the range of costs it likely imposes, makes it a superior choice to any other.” QUA, performed carefully and following clear policy instructions, can empower decision-makers to earn their influential roles.
Similar content being viewed by others
References
Bogen KT (1995a) Methods to approximate joint uncertainty and variability in risk. Risk Anal 15(3):411–419
Bogen KT (1995b) A note on compounded conservatism. Risk Anal 14:379–381
Burgman MA, Keith DA, Walshe TV (1999) Uncertainty in comparative risk analysis for threatened Australian plant species. Risk Anal 19(4):585–598
Cooke RM (ed) (2009) Uncertainty modeling in dose response: bench testing environmental toxicity. Wiley, Hoboken
Cox LAT (2012) Confronting deep uncertainties in risk analysis. Risk Anal 32:1607
Cullen AC, Frey HC (1999). Probabilistic techniques in exposure assessment: a handbook for dealing with variability and uncertainty in models and inputs. Plenum Press, New York
Cummings CL, Kuzma J (2017) Societal risk evaluation scheme (SRES): scenario-based multi-criteria evaluation of synthetic biology applications. PLoS ONE 12(1):e0168564
Dudley SE, Gray GM (2012) Improving the use of science to inform environmental regulation. In: Johnston J (ed) Institutions and incentives in regulatory science. Lexington Books, New York
Finkel AM (1990a) Confronting uncertainty in risk management: a guide for decision-makers. Resources for the Future, Center for Risk Management monograph, Washington, DC
Finkel AM (1990b) A simple formula for calculating the “mass density” of a lognormally-distributed characteristic: applications to risk analysis. Risk Anal 10(2):291–301
Finkel AM (1991) Edifying presentation of risk estimates: not as easy as it seems. J Policy Anal Manag 10(2):296–303
Finkel AM (1995) Towards less misleading comparisons of uncertain risks: the example of aflatoxin and alar. Environ Health Perspect 103(4):376–385
Finkel AM (2014) The cost of nothing trumps the value of everything: the failure of regulatory economics to keep pace with improvements in quantitative risk analysis. Mich J Environ Adm Law 4(1):91–156
Finkel AM (2018) Demystifying evidence-based policy analysis by revealing hidden value-laden constraints. In: Governance of emerging technologies: aligning policy analysis with the public’s values. Hastings Center Special Report 48(S1):S21–S49
Finkel AM, Evans JS (1987) Evaluating the benefits of uncertainty reduction in environmental health risk management. J Air Pollut Control Assoc 37(10):1164–1171
Ginsberg G, Toal BF (2009) Quantitative approach for incorporating methylmercury risks and omega-3 fatty acid benefits in developing species-specific fish consumption advice. Environ Health Perspect 117(2):267–275
Gray GM, Cohen JT (2012) Rethink chemical risk assessment. Nature 489:27–28
Gray GM, Graham JD (1991) Risk assessment and clean air policy. J Policy Anal Manag 10(2):286–295
Groth E III (2017) Scientific foundations of fish-consumption advice for pregnant women: epidemiological evidence, benefit-risk modeling, and an integrated approach. Environ Res 152:386–406
Hammitt JK, Shlyakhter AI (1999) The expected value of information and the probability of surprise. Risk Anal 19:135–152
Hammond JS, Keeney RL, Raiffa H (1998) Smart choices: a practical guide to making better decisions. Harvard Business Review Press, Cambridge
Harrington W, Morgenstern RD, Nelson P (2000) On the accuracy of regulatory cost estimates. J Policy Anal Manag 19:297
Hattis DB, Anderson EL (1999) What should be the implications of uncertainty, variability, and inherent “biases”/“conservatism” for risk management decision-making? Risk Anal 19(1):95–107
Krupnick A et al (2006) Not a sure thing: making regulatory choices under uncertainty. Resources for the Future, Washington DC
Kruser JM et al (2017) “Best case/worst case”: training surgeons to use a novel communication tool for high-risk acute surgical problems. J Pain Symptom Manag 53(4):711–719
Laxminarayan R, Macauley MK (eds) (2012) The value of information: methodological frontiers and new applications in environment and health. Springer, Dordrecht
Limpert E, Stahel W (undated) Log-normal distribution: its widespread use and some basic hints for its application. Compendium of weblinks, available at https://stat.ethz.ch/~stahel/lognormal/
Linkov I, Trump BD, Wender BA, Seager TP, Kennedy AJ, Keisler JM (2017) Integrate life-cycle assessment and risk analysis results, not methods. Nat Nanotechnol 12(8):740–743
Malloy T, Trump BD, Linkov I (2016) Risk-based and prevention-based governance for emerging materials. Environ Sci Technol 50(13):6822–6824
Morgan MG, Henrion M (1992) Uncertainty: a guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge University Press, Cambridge
National Academy of Sciences (1994) Science and judgment in risk assessment. National Academy Press, Washington
National Academy of Sciences (2008) Science and decisions: advancing risk assessment. National Academy Press, Washington
National Academy of Sciences (2011) Review of the Environmental Protection Agency’s draft IRIS assessment of formaldehyde. National Academy Press, Washington
Nichols AL, Zeckhauser RJ (1986) The perils of prudence: how conservative risk assessments distort regulation. Regulation 10:13–24
Presidential/Congressional Commission on Risk Assessment and Management (1997) Risk assessment and risk management in regulatory decision-making. Available at https://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=55006
Robinson LA, Hammitt JK, Zeckhauser RJ (2016) Attention to distribution in US regulatory analyses. Rev Environ Econ Policy 10(2):308–328
Shapiro S, Fisher E, Wagner W (2012) The enlightenment of administrative law: looking inside the agency for legitimacy. Wake Forest Law Rev 47:463–502
Sherman R (2016) Roger Goodell warns fans about risks of watching the Super Bowl. Article in SB Nation, Feb. 5, available at https://www.sbnation.com/2016/2/5/10924418/roger-goodell-concussions-super-bowl-nfl-couches-are-dangerous
Shlyakhter AI (1994) An improved framework for uncertainty analysis: accounting for unsuspected errors. Risk Anal 14:441–447
Society for Risk Analysis (2016) Technical symposium on “Transparency and uncertainty analysis: benefits and pitfalls.” Abstracts available at http://birenheide.com/sra/2016AM/program/singlesession.php3?sessid=W1-E&order=3#3
Stahl CH, Cimorelli AJ (2005) How much uncertainty is too much and how do we know? A case example of the assessment of ozone monitor network options. Risk Anal 25:1109
Stern A (2015) “Comments on Uncertainty Analysis,” (presentation to the U.S. EPA) available at https://www.epa.gov/sites/production/files/2015-01/documents/session3b_stern_talking_points.pdf
Sunstein CR (2002) Risk and reason: safety, law, and the environment. Cambridge University Press, Cambridge
Thompson KM, Graham JD (1996) Going beyond the single number: using probabilistic risk assessment to improve risk management. Hum Ecol Risk Assess 2:1008–1034
Trump B, Cummings C, Kuzma J, Linkov I (2017) A decision analytic model to guide early-stage government regulatory action: applications for synthetic biology. Regul Gov. https://doi.org/10.1111/rego.12142
Weitzman ML (2011) Fat-tailed uncertainty in the economics of catastrophic climate change. Rev Environ Econ Policy 5(2):275–292
Wilson R, Crouch EAC, Zeise L (1985) Uncertainty in risk assessment. In: Hoel DG et al, (eds) Risk quantitation and regulatory policy. Cold Spring Harbor
Yokota F, Thompson KM (2004) The value of information in environmental health risk management decisions: past, present, future. Risk Anal 24:635–650
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Finkel, A.M., Gray, G. Taking the reins: how regulatory decision-makers can stop being hijacked by uncertainty. Environ Syst Decis 38, 230–238 (2018). https://doi.org/10.1007/s10669-018-9681-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10669-018-9681-x