Abstract
Sensitivity analysis (SA) is an important tool for assessing and reducing uncertainties in computer-based models. This chapter presents a comprehensive review of some commonly used SA methods, including gradient-based, variance-based, and regression-based methods. Features and applicability of those methods are described and illustrated with some examples. Merits and limitations of different methods are explained, and the criteria of choosing appropriate SA methods for different applications are suggested.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
E.A. Anderson, National Weather Service River Forecast System – Snow Accumulation and Ablation Model (NOAA, Silver Spring, 1973)
J.G. Arnold, R. Srinivasan, R.S. Muttiah, J.R. Williams, Large area hydrologic modeling and assessment. Part I: model development. J. Am. Water Resour. Assoc. 34, 73–89 (1998). https://doi.org/10.1111/j.1752-1688.1998.tb05961.x
D.L. Beres, D.M. Hawkins, Plackett – Burman technique for sensitivity analysis of many-parametered models. Ecol. Model. 141, 171–183 (2001). https://doi.org/10.1016/S0304-3800(01)00271-X
K. Beven, TOPMODEL: a critique. Hydrol. Process. 11, 1069–1085 (1997). https://doi.org/10.1002/(SICI)1099-1085(199707)11:9<1069::AID-HYP545>3.0.CO;2-O
E. Borgonovo, W. Castaings, S. Tarantola, Model emulation and moment-independent sensitivity analysis: an application to environmental modelling. Environ. Model. Softw. 34, 105–115 (2012). https://doi.org/10.1016/j.envsoft.2011.06.006
G.E.P. Box, J.S. Hunter, The 2k-p fractional factorial designs: part I. Technometrics 3, 311–351 (1961a). https://doi.org/10.1080/00401706.1961.10489951
G.E.P. Box, J.S. Hunter, The 2k-p fractional factorial designs: part II. Technometrics 3, 449–458 (1961b). https://doi.org/10.1080/00401706.1961.10489967
R.J.C. Burnash, R.L. Ferral, R.A. McGuire, A Generalized Streamflow Simulation System: Conceptual Modeling for Digital Computers (US Department of Commerce, National Weather Service, Sacramento, 1973)
F. Campolongo, A. Saltelli, Sensitivity analysis of an environmental model: an application of different analysis methods. Reliab. Eng. Syst. Saf. 57, 49–69 (1997). https://doi.org/10.1016/S0951-8320(97)00021-5
F. Campolongo, J. Cariboni, A. Saltelli, An effective screening design for sensitivity analysis of large models. Environ. Model. Softw. 22, 1509–1518 (2007). https://doi.org/10.1016/j.envsoft.2006.10.004
F. Campolongo, A. Saltelli, J. Cariboni, From screening to quantitative sensitivity analysis. A unified approach. Comput. Phys. Commun. 182, 978–988 (2011). https://doi.org/10.1016/j.cpc.2010.12.039
J. Cariboni, D. Gatelli, R. Liska, A. Saltelli, The role of sensitivity analysis in ecological modelling. Ecol. Model. 203, 167–182 (2007). https://doi.org/10.1016/j.ecolmodel.2005.10.045
H.A. Chipman, E.I. George, R.E. McCulloch, BART: Bayesian additive regression trees. Ann. Appl. Stat. 4, 266–298 (2010). https://doi.org/10.1214/09-AOAS285
D.C. Collins, R. Avissar, An evaluation with the Fourier amplitude sensitivity test (FAST) of which land-surface parameters are of greatest importance in atmospheric modeling. J. Clim. 7, 681–703 (1994). https://doi.org/10.1175/1520-0442(1994)007<0681:AEWTFA>2.0.CO;2
R. Confalonieri, G. Bellocchi, S. Bregaglio, M. Donatelli, M. Acutis, Comparison of sensitivity analysis techniques: a case study with the rice model WARM. Ecol. Model. 221, 1897–1906 (2010). https://doi.org/10.1016/j.ecolmodel.2010.04.021
S.A. Cryer, P.L. Havens, Regional sensitivity analysis using a fractional factorial method for the USDA model GLEAMS. Environ. Model. Softw. 14, 613–624 (1999). https://doi.org/10.1016/S1364-8152(99)00003-1
R.I. Cukier, C.M. Fortuin, K.E. Shuler, A.G. Petschek, J.H. Schaibly, Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I theory. J. Chem. Phys. 59, 3873–3878 (1973). https://doi.org/10.1063/1.1680571
C. Daniel, On varying one factor at a time. Biometrics 14, 430–431 (1958)
R.E. Dickinson, A. Henderson-Sellers, P.J. Kennedy, M.F. Wilson, Biosphere-Atmosphere Transfer Scheme (BATS) for the NCAR Community Climate Model (NCAR, Boulder, 1986). https://doi.org/10.5065/D6668B58
E. Dion, L. VanSchalkwyk, E.F. Lambin, The landscape epidemiology of foot-and-mouth disease in South Africa: a spatially explicit multi-agent simulation. Ecol. Model. 222, 2059–2072 (2011). https://doi.org/10.1016/j.ecolmodel.2011.03.026
Q. Duan, J. Schaake, V. Andreassian, S. Franks, G. Goteti, H.V. Gupta, Y.M. Gusev, F. Habets, A. Hall, L. Hay, T. Hogue, M. Huang, G. Leavesley, X. Liang, O.N. Nasonova, J. Noilhan, L. Oudin, S. Sorooshian, T. Wagener, E.F. Wood, Model parameter estimation experiment (MOPEX): an overview of science strategy and major results from the second and third workshops. J. Hydrol. 320, 3–17 (2006). https://doi.org/10.1016/j.jhydrol.2005.07.031
E. Eirola, E. Liitiäinen, A. Lendasse, F. Corona, M. Verleysen, Using the Delta test for variable selection, in ESANN 2008 Proceedings, European Symposium on Artificial Neural Networks, Bruges, Belgium (2008)
L. Foglia, M.C. Hill, S.W. Mehl, P. Burlando, Sensitivity analysis, calibration, and testing of a distributed hydrological model using error-based weighting and one objective function. Water Resour. Res. 45, W6427 (2009). https://doi.org/10.1029/2008WR007255
A. Francos, F.J. Elorza, F. Bouraoui, G. Bidoglio, L. Galbiati, Sensitivity analysis of distributed environmental simulation models: understanding the model behaviour in hydrological studies at the catchment scale. Reliab. Eng. Syst. Saf. 79, 205–218 (2003). https://doi.org/10.1016/S0951-8320(02)00231-4
H.C. Frey, S.R. Patil, Identification and review of sensitivity analysis methods. Risk Anal. 22, 553–578 (2002). https://doi.org/10.1111/0272-4332.00039
H.C. Frey, A. Mokhtari, T. Danish, Evaluation of selected sensitivity analysis methods based upon applications to two food safety process risk models. Department of Civil, Construction, and Environmental Engineering, North Carolina State University, Raleigh, NC (2003)
J.H. Friedman, Multivariate adaptive regression splines. Ann. Stat. 19, 1–67 (1991). https://doi.org/10.1214/aos/1176347963
F. Galton, Regression towards mediocrity in hereditary stature. J. Anthropol. Inst. G. B. Irel. 15, 246–263 (1886)
Y. Gan, Q. Duan, W. Gong, C. Tong, Y. Sun, W. Chu, A. Ye, C. Miao, Z. Di, A comprehensive evaluation of various sensitivity analysis methods: a case study with a hydrological model. Environ. Model. Softw. 51, 269–285 (2014). https://doi.org/10.1016/j.envsoft.2013.09.031
Y. Gan, X.-Z. Liang, Q. Duan, H.I. Choi, Y. Dai, H. Wu, Stepwise sensitivity analysis from qualitative to quantitative: application to the terrestrial hydrological modeling of a conjunctive surface-subsurface process (CSSP) land surface model. J. Adv. Model. Earth Syst. 7, 648–669 (2015). https://doi.org/10.1002/2014MS000406
M. Gibbs, D.J.C. MacKay, Efficient implementation of Gaussian processes. Unpublished manuscript (1997)
J. Grant, K.J. Curran, T.L. Guyondet, G. Tita, C. Bacher, V. Koutitonsky, M. Dowd, A box model of carrying capacity for suspended mussel aquaculture in Lagune de la Grande-Entrée, Iles-de-la-Madeleine, Québec. Ecol. Model. 200, 193–206 (2007). https://doi.org/10.1016/j.ecolmodel.2006.07.026
A. van Griensven, T. Meixner, S. Grunwald, T. Bishop, M. Diluzio, R. Srinivasan, A global sensitivity analysis tool for the parameters of multi-variable catchment models. J. Hydrol. 324, 10–23 (2006). https://doi.org/10.1016/j.jhydrol.2005.09.008
D.M. Hamby, A review of techniques for parameter sensitivity analysis of environmental models. Environ. Monit. Assess. 32, 135–154 (1994). https://doi.org/10.1007/BF00547132
J.C. Helton, Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal. Reliab. Eng. Syst. Saf. 42, 327–367 (1993). https://doi.org/10.1016/0951-8320(93)90097-I
A. Henderson-Sellers, A factorial assessment of the sensitivity of the BATS land-surface parameterization scheme. J. Clim. 6, 227–247 (1993). https://doi.org/10.1175/1520-0442(1993)006<0227:AFAOTS>2.0.CO;2
B. Henderson-Sellers, A. Henderson-Sellers, Sensitivity evaluation of environmental models using fractional factorial experimentation. Ecol. Model. 86, 291–295 (1996). https://doi.org/10.1016/0304-3800(95)00066-6
T. Homma, A. Saltelli, Importance measures in global sensitivity analysis of nonlinear models. Reliab. Eng. Syst. Saf. 52, 1–17 (1996). https://doi.org/10.1016/0951-8320(96)00002-6
R.L. Iman, W.J. Conover, The use of the rank transform in regression. Technometrics 21, 499–509 (1979). https://doi.org/10.1080/00401706.1979.10489820
A.J. Jakeman, R.A. Letcher, J.P. Norton, Ten iterative steps in development and evaluation of environmental models. Environ. Model. Softw. 21, 602–614 (2006). https://doi.org/10.1016/j.envsoft.2006.01.004
J.P.C. Kleijnen, Sensitivity analysis and related analyses: a review of some statistical techniques. J. Stat. Comput. Simul. 57, 111–142 (1997). https://doi.org/10.1080/00949659708811805
T. Lenhart, K. Eckhardt, N. Fohrer, H.G. Frede, Comparison of two different approaches of sensitivity analysis. Phys. Chem. Earth 27, 645–654 (2002). https://doi.org/10.1016/S1474-7065(02)00049-9
D.J.C. MacKay, Introduction to Gaussian processes, in Neural Networks and Machine Learning, ed. by C.M. Bishop, (Springer, Berlin, 1998). pp 133–165
S. Marino, I.B. Hogue, C.J. Ray, D.E. Kirschner, A methodology for performing global uncertainty and sensitivity analysis in systems biology. J. Theor. Biol. 254, 178–196 (2008). https://doi.org/10.1016/j.jtbi.2008.04.011
L.S. Matott, J.E. Babendreier, S.T. Purucker, Evaluating uncertainty in integrated environmental models: a review of concepts and tools. Water Resour. Res. 45, W6421 (2009). https://doi.org/10.1029/2008WR007301
M.D. McKay, Evaluating Prediction Uncertainty (Los Alamos National Laboratory (LANL), Los Alamos, 1995)
M.D. McKay, R.J. Beckman, W.J. Conover, A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21, 239–245 (1979). https://doi.org/10.1080/00401706.2000.10485979
A. Mokhtari, H.C. Frey, Sensitivity analysis of a two-dimensional probabilistic risk assessment model using analysis of variance. Risk Anal. 25, 1511–1529 (2005). https://doi.org/10.1111/j.1539-6924.2005.00679.x
M.D. Morris, Factorial sampling plans for preliminary computational experiments. Technometrics 33, 161–174 (1991). https://doi.org/10.1080/00401706.1991.10484804
J. Nossent, P. Elsen, W. Bauwens, Sobol’ sensitivity analysis of a complex environmental model. Environ. Model. Softw. 26, 1515–1525 (2011). https://doi.org/10.1016/j.envsoft.2011.08.010
A.B.. Owen, Orthogonal arrays for computer experiments, integration and visualization. Stat. Sin. 2, 439–452 (1992)
H. Pi, C. Peterson, Finding the embedding dimension and variable dependencies in time series. Neural Comput. 6, 509–520 (1994). https://doi.org/10.1162/neco.1994.6.3.509
R.L. Plackett, J.P. Burman, The design of optimum multifactorial experiments. Biometrika 33, 305–325 (1946). https://doi.org/10.1093/biomet/33.4.305
O. Rakovec, M.C. Hill, M.P. Clark, A.H. Weerts, A.J. Teuling, R. Uijlenhoet, Distributed evaluation of local sensitivity analysis (DELSA), with application to hydrologic models. Water Resour. Res. 50, 409–426 (2014). https://doi.org/10.1002/2013WR014063
S. Razavi, B.A. Tolson, D.H. Burn, Review of surrogate modeling in water resources. Water Resour. Res. 48, W7401 (2012). https://doi.org/10.1029/2011WR011527
B. Renard, D. Kavetski, G. Kuczera, M. Thyer, S.W. Franks, Understanding predictive uncertainty in hydrologic modeling: the challenge of identifying input and structural errors. Water Resour. Res. 46, W5521 (2010). https://doi.org/10.1029/2009WR008328
D.E. Reusser, W. Buytaert, E. Zehe, Temporal dynamics of model parameter sensitivity for computationally expensive models with the Fourier amplitude sensitivity test. Water Resour. Res. 47, W7551 (2011). https://doi.org/10.1029/2010WR009947
E. Rodríguez-Camino, R. Avissar, Comparison of three land-surface schemes with the Fourier amplitude sensitivity test (FAST). Tellus A 50, 313–332 (1998). https://doi.org/10.3402/tellusa.v50i3.14529
A. Saltelli, Sensitivity analysis: could better methods be used? J. Geophys. Res. 104, 3789–3793 (1999). https://doi.org/10.1029/1998JD100042
A. Saltelli, R. Bolado, An alternative way to compute Fourier amplitude sensitivity test (FAST). Comput. Stat. Data Anal. 26, 445–460 (1998). https://doi.org/10.1016/S0167-9473(97)00043-1
A. Saltelli, S. Tarantola, K.P.S. Chan, A quantitative model-independent method for global sensitivity analysis of model output. Technometrics 41, 39–56 (1999). https://doi.org/10.2307/1270993
A. Saltelli, S. Tarantola, F. Campolongo, Sensitivity analysis as an ingredient of modeling. Stat. Sci. 15, 377–395 (2000). https://doi.org/10.1214/ss/1009213004
A. Saltelli, M. Ratto, S. Tarantola, F. Campolongo, Sensitivity analysis for chemical models. Chem. Rev. 105, 2811–2827 (2005). https://doi.org/10.1021/cr040659d
A. Saltelli, M. Ratto, S. Tarantola, F. Campolongo, Sensitivity analysis practices: strategies for model-based inference. Reliab. Eng. Syst. Saf. 91, 1109–1125 (2006). https://doi.org/10.1016/j.ress.2005.11.014
A. Saltelli, M. Ratto, T. Andres, F. Campolongo, Global Sensitivity Analysis: The Primer (Wiley, Chichester, 2008)
J.H. Schaibly, K.E. Shuler, Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. II applications. J. Chem. Phys. 59, 3879–3888 (1973). https://doi.org/10.1063/1.1680572
D. Shahsavani, A. Grimvall, Variance-based sensitivity analysis of model outputs using surrogate models. Environ. Model. Softw. 26, 723–730 (2011). https://doi.org/10.1016/j.envsoft.2011.01.002
A. Sieber, S. Uhlenbrook, Sensitivity analyses of a distributed catchment model to verify the model structure. J. Hydrol. 310, 216–235 (2005). https://doi.org/10.1016/j.jhydrol.2005.01.004
I.M. Sobol’, Quasi-Monte Carlo methods. Prog. Nucl. Energy 24, 55–61 (1990). https://doi.org/10.1016/0149-1970(90)90022-W
I.M. Sobol’, Sensitivity analysis for nonlinear mathematical models. Math. Model. Comput. Exp. 1, 407–414 (1993)
I.M. Sobol’, Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math. Comput. Simul. 55, 271–280 (2001). https://doi.org/10.1016/S0378-4754(00)00270-6
D. Steinberg, P.L. Colla, K. Martin, MARS User Guide (Salford Systems, San Diego, 1999)
C.B. Storlie, L.P. Swiler, J.C. Helton, C.J. Sallaberry, Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models. Reliab. Eng. Syst. Saf. 94, 1735–1763 (2009). https://doi.org/10.1016/j.ress.2009.05.007
X.Y. Sun, L. Newham, B. Croke, J.P. Norton, Three complementary methods for sensitivity analysis of a water quality model. Environ. Model. Softw. 37, 19–29 (2012). https://doi.org/10.1016/j.envsoft.2012.04.010
B. Tang, Orthogonal array-based Latin hypercubes. J. Am. Stat. Assoc. 88, 1392–1397 (1993). https://doi.org/10.2307/2291282
Y. Tang, P. Reed, T. Wagener, K. van Werkhoven, Comparing sensitivity analysis methods to advance lumped watershed model identification and evaluation. Hydrol. Earth Syst. Sci. 11, 793–817 (2007). https://doi.org/10.5194/hess-11-793-2007
C. Tong, PSUADE User’s Manual (Lawrence Livermore National Laboratory (LLNL), Livermore, 2005)
T. Turányi, Sensitivity analysis of complex kinetic systems. Tools and applications. J. Math. Chem. 5, 203–248 (1990). https://doi.org/10.1007/BF01166355
L. Uusitalo, A. Lehikoinen, I. Helle, K. Myrberg, An overview of methods to evaluate uncertainty of deterministic models in decision support. Environ. Model. Softw. 63, 24–31 (2015). https://doi.org/10.1016/j.envsoft.2014.09.017
T. Wagener, K. van Werkhoven, P. Reed, Y. Tang, Multiobjective sensitivity analysis to understand the information content in streamflow observations for distributed watershed modeling. Water Resour. Res. 45, W2501 (2009). https://doi.org/10.1029/2008WR007347
H.M. Wainwright, S. Finsterle, Y. Jung, Q. Zhou, J.T. Birkholzer, Making sense of global sensitivity analyses. Comput. Geosci. UK 65, 84–94 (2014). https://doi.org/10.1016/j.cageo.2013.06.006
W.E. Walker, P. Harremoës, J. Rotmans, J.P. van der Sluijs, M.B. van Asselt, P. Janssen, M.P. Krayer Von Krauss, Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support. Integr. Assess. 4, 5–17 (2003). https://doi.org/10.1076/iaij.4.1.5.16466
G.G. Wang, S. Shan, Review of metamodeling techniques in support of engineering design optimization. J. Mech. Des. 129, 370–380 (2007). https://doi.org/10.1115/1.2429697
J. Wang, X. Li, L. Lu, F. Fang, Parameter sensitivity analysis of crop growth models based on the extended Fourier amplitude sensitivity test method. Environ. Model. Softw. 48, 171–182 (2013). https://doi.org/10.1016/j.envsoft.2013.06.007
C. Wang, Q. Duan, C. Tong, W. Gong, A GUI platform for uncertainty quantification of complex dynamical models. Environ. Model. Softw. 76, 1–12 (2016). https://doi.org/10.1016/j.envsoft.2015.11.004
J.E. Ward, R.E. Wendell, Approaches to sensitivity analysis in linear programming. Ann. Oper. Res. 27, 3–38 (1990). https://doi.org/10.1007/BF02055188
H. Weyl, Mean motion. Am. J. Math. 60, 889–896 (1938)
Acknowledgment
This study was supported by the National Natural Science Foundation of China (41505092) and National Key Research and Development Program of China (2017YFC1404000).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2019 Springer-Verlag GmbH Germany, part of Springer Nature
About this entry
Cite this entry
Gan, Y., Duan, Q. (2019). Sensitivity Analysis Methods. In: Duan, Q., Pappenberger, F., Wood, A., Cloke, H., Schaake, J. (eds) Handbook of Hydrometeorological Ensemble Forecasting. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39925-1_65
Download citation
DOI: https://doi.org/10.1007/978-3-642-39925-1_65
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39924-4
Online ISBN: 978-3-642-39925-1
eBook Packages: Earth and Environmental ScienceReference Module Physical and Materials ScienceReference Module Earth and Environmental Sciences