Advertisement

Recommended practices in global sensitivity analysis

  • Andrea Saltelli
  • Daniele Vidoni
  • Massimiliano Mascherini
Part of the NATO Science for Peace and Security Series C: Environmental Security book series (NAPSC)

Practices for global sensitivity analysis of model output are described in a recent textbook (Saltelli et al., 2007). These include (i) variance based techniques for general use in modelling, (ii) the elementary effect method for factor screening for factors-rich models and (iii) Monte Carlo filtering. In the present work we try to put the practices into the context of their usage. We start by describing the present debate on the use of scientific models, and how uncertainty and sensitivity analysis can assist is testing model quality. We discuss Type I, II and III errors in the context of sensitivity analysis and what are the requirements for a good analysis. We also present sensitivity analysis in relation to post normal science (PNS) and model pedigrees.

Keywords

uncertainty analysis sensitivity analysis impact assessment Monte Carlo post normal science 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Baudrillard, J., 1999, Revenge of the Crystal, Pluto Press, London, UK, p. 92.Google Scholar
  2. Beck, M. B., Ravetz, J. R., Mulkey, L. A., and Barnwell, T. O., 1997, Stochastic Hydrology and Hydraulics, 11, 229–254.CrossRefGoogle Scholar
  3. Beven, K. J., 2001, Rainfall-Runoff Modelling: The Primer. John Wiley and Sons, Ltd, Chichester. See also (31), pp. 151–192, 2004.Google Scholar
  4. Chatfield, C., 1993, Model uncertainty, data mining and statistical inference, Journal of the Royal Statistical Society A, 158(3), 419–466.CrossRefGoogle Scholar
  5. Crichton, M., 2004, State of Fear, Harper Collins, New York, New York.Google Scholar
  6. Dunn, W. N., 1997, Cognitive Impairment and Social Problem Solving: Some Tests for Type III Errors in Policy Analysis, Graduate School of Public and International Affairs, University of Pittsburgh, Pittsburgh.Google Scholar
  7. European Commission SEC, 2005, 791 IMPACT ASSESSMENT GUIDELINES, 15 June 2005, http://ec.europa.eu/governance/docs/index en.htm
  8. Environmental Protection Agency, Models Guidance Draft — November 2003 Draft Guidance on the Development, Evaluation, and Application of Regulatory Environmental Models Prepared by: The Council for Regulatory Environmental Modeling, http://cfpub.epa.gov/crem/cremlib.cfm.
  9. Funtowicz, S. O. and Ravetz, J. R., 1990, Uncertainty and Quality in Science for Policy, Kluwer Academic, Dordrecht, NL.Google Scholar
  10. Funtowicz, S. O. and Ravetz, J. R., 1993, Science for the post-normal age. Futures, 25, 735 755.CrossRefGoogle Scholar
  11. Funtowicz, S., O'Connor, M., Faucheux, S., Froger, G., and Munda, G., 1996, Emergent complexity and procedural rationality: post-normal science for sustainability, In: Costanza R, Segura O, Martinez-Alier J, editors. Getting down to earth: practical applications of ecological economics. Washington (D.C.): Island Press, 1996:223–48.Google Scholar
  12. Hoeting, J. A., Madigan, D., Raftery A. E., and Volinsky C. T., 1999, Bayesian model averaging: a tutorial, Statistical Science, 14(4), 382417.Google Scholar
  13. Kennedy, P., 2007, A Guide to Econometrics, Fifth edition, Blackwell, Oxford, UK.Google Scholar
  14. Konikov, L. F. and Bredehoeft, J. D., 1992, Groundwater models cannot be validated, Advances in Water Resources, 15(1), 75–83.CrossRefGoogle Scholar
  15. Leamer, E. E., 1990, “Let's take the con out of econometrics” and “sensitivity analysis would help”, in Modelling Economic Series (ed. Granger, C. W. J.), Clarendon, Oxford.Google Scholar
  16. Macilwain, C., 2006, Safe and sound? Nature, 442, 242–243 (20 July 2006).CrossRefGoogle Scholar
  17. Michaels, D., 2005, Doubt is their product, Scientiffc American, 292(6), 96–101.CrossRefGoogle Scholar
  18. Office of Management and Budget (OMB), 2002, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies; Notice; Republication, http://www.whitehouse.gov/omb/inforeg/, February 22, 2002.
  19. Office of Management and Budget (OMB), 2006, Proposed Risk Assessment Bulletin (January 9, 2006), http://www.whitehouse.gov/omb/inforeg/
  20. Oreskes, N., Shrader-Frechette, K., and Belitz, K., 1994, Verification, validation, and confirmation of numerical models in the Earth sciences, Science, 263, 641–646.CrossRefGoogle Scholar
  21. Pilkey, O. and Pilkey-Jarvis, L., 2007, Useless Arithmetic: Why Environmental Scientists Can't Predict the Future, Columbia University Press, Irvington, New York.Google Scholar
  22. Rosen, R., 1991, Life Itself – A Comprehensive Inquiry into Nature, Origin, and Fabrication of Life. Columbia University Press, Irvington, New York.Google Scholar
  23. Saltelli, A., Andres, T., Campolongo, F., Cariboni J., Gatelli D., Ratto, M., Saisana, M., and Tarantola, S., 2007, Sensitivity Analysis of Scientific Models, John Wiley and Sons, to appear in winter 2007. (forthcoming)Google Scholar
  24. Saltelli, K., Chan, A., and Scott, M., Editors, 2000, Sensitivity Analysis, Probability and Statistics series, John Wiley and Sons. (forthcoming)Google Scholar
  25. Saltelli, A., Tarantola, S., Campolongo, F., and Ratto, M., 2004, Sensitivity Analysis in Practice. A Guide to Assessing Scientific Models, John Wiley and Sons, Chichester, New York.Google Scholar
  26. Taleb, N., 2007, The Black Swan, Penguin, London.Google Scholar
  27. Van der Sluijs, J. P., 2002, A way out of the credibility crisis of models used in integrated environmental assessment, Futures, 34, 133–146.CrossRefGoogle Scholar
  28. Van der Sluijs, J. P., Craye, M., Funtowicz, S., Kloprogge, P., Ravetz, J., and Risbey, J., 2005, Combining quantitative and qualitative measures of uncertainty in model-based environmental assessment: the NUSAP system, Risk Analysis, 25(2), 481–492.Google Scholar

Copyright information

© Springer Science + Business Media B.V 2009

Authors and Affiliations

  • Andrea Saltelli
    • 1
  • Daniele Vidoni
    • 1
  • Massimiliano Mascherini
    • 1
  1. 1.The European Commission, Joint Research CentreInstitute for the Protection and Security of the Citizen (IPSC), TP 361IspraItaly

Personalised recommendations