Population Ecology

, Volume 56, Issue 1, pp 7–19 | Cite as

Meta-models as a straightforward approach to the sensitivity analysis of complex models

Special feature: Original article Mathematical Models for Effective Environmental Management

Abstract

Complex simulation models are important tools in applied ecological and conservation research. However sensitivity analysis of this important class of models can be difficult to conduct. High level interactions and non-linear responses are common in complex simulations, and this necessitates a global sensitivity analysis, where each parameter is tested at a range of values, and in combination with changes in many other parameters. We reviewed the literature, searching for population viability analyses that used simulation models. We found only 9 out of the 122 simulation population viability analysis used global sensitivity analysis. This result is typical of other simulation models in applied ecology, where global sensitivity analysis is rare. We then demonstrate how to conduct a meta-modeling sensitivity analysis, where a simpler statistically fit function (the meta-model, also known as the surrogate model or emulator) is used to approximate the behavior of the complicated simulation. This simpler meta-model is interrogated to inform on the behavior of simulation model. We fit two example meta-models, a generalized linear model and a boosted regression tree, to exemplify the approach. Our hope is that by going through these techniques thoroughly they will become more widely adopted.

Keywords

Boosted regression trees Generalized linear models Meta-models Population viability analyses Simulation model 

Supplementary material

10144_2013_422_MOESM1_ESM.pdf (296 kb)
Supplementary material 1 (PDF 296 kb)
10144_2013_422_MOESM2_ESM.r (31 kb)
Supplementary material 2 (R 31 kb)

References

  1. Bode M, Brennan KEC (2011) Using population viability analysis to guide research and conservation actions for Australia’s threatened malleefowl Leipoa ocellata. Oryx 45:513–521CrossRefGoogle Scholar
  2. Bonesi L, Rushton SP, Macdonald DW (2007) Trapping for mink control and water vole survival: identifying key criteria using a spatially explicit individual based model. Biol Conserv 136:636–650CrossRefGoogle Scholar
  3. Borgonovo E, Castaings W, Tarantola S (2012) Model emulation and moment-independent sensitivity analysis: an application to environmental modelling. Environ Modell Softw 34:105–115CrossRefGoogle Scholar
  4. Buckley YM, Briese DT, Rees M (2003) Demography and management of the invasive plant species Hypericum perforatum. II. Construction and use of an individual-based model to predict population dynamics and the effects of management strategies. J Appl Ecol 40:494–507CrossRefGoogle Scholar
  5. Coutts SR, van Klinken RD, Yokomizo H, Buckley YM (2011) What are the key drivers of spread in invasive plants: dispersal, demography or landscape: and how can we use this knowledge to aid management? Biol Invasions 13:1649–1661CrossRefGoogle Scholar
  6. Cross PC, Beissinger SR (2001) Using logistic regression to analyze the sensitivity of PVA models: a comparison of methods based on African wild dog models. Conserv Biol 15:1335–1346CrossRefGoogle Scholar
  7. Curtis JMR, Naujokaitis-Lewis I (2008) Sensitivity of population viability to spatial and nonspatial parameters using grip. Ecol Appl 18:1002–1013PubMedCrossRefGoogle Scholar
  8. Dalerum F, Shults B, Kunkel K (2008) Estimating sustainable harvest in wolverine populations using logistic regression. J Wildlife Manage 72:1125–1132CrossRefGoogle Scholar
  9. Decesare NJ, Whittington J, Hebblewhite M, Robinson H, Bradley M, Neufeld L, Musiani M (2011) The role of translocation in recovery of woodland caribou populations. Conserv Biol 25:365–373PubMedGoogle Scholar
  10. Elith J, Leathwick JR, Hastie T (2008) A working guide to boosted regression trees. J Anim Ecol 77:802–813PubMedCrossRefGoogle Scholar
  11. Ellner SP, Fieberg J (2003) Using PVA for management despite uncertainty: effects of habitat, hatcheries, and harvest on salmon. Ecology 84:1359–1369CrossRefGoogle Scholar
  12. Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29:1189–1232CrossRefGoogle Scholar
  13. Girvetz EH (2010) Removing erosion control projects increases bank swallow (Riparia riparia) population viability modeled along the Sacramento River, California, USA. Biol Conserv 143:828–838CrossRefGoogle Scholar
  14. Gramacy RB, Taddy M (2010) Categorical inputs, sensitivity analysis, optimization and importance tempering with tgp version 2, an R package for treed Gaussian process models. J Stat Softw 33:1–48Google Scholar
  15. Haines AM, Tewes ME, Laack LL, Grant WE, Young J (2005) Evaluating recovery strategies for an ocelot (Leopardus pardalis) population in the United States. Biol Conserv 126:512–522CrossRefGoogle Scholar
  16. Hunter CM, Moller H, Fletcher D (2000) Parameter uncertainty and elasticity analyses of a population model: setting research priorities for shearwaters. Ecol Model 134:299–323CrossRefGoogle Scholar
  17. Johnson LR (2010) Implications of dispersal and life history strategies for the persistence of Linyphiid spider populations. Ecol Model 221:1138–1147CrossRefGoogle Scholar
  18. Kleijnen JPC, Noordegraaf AV, Nielen M (2001) Sensitivity analysis of censored output through polynomial, logistic, and tobit regression meta-models: Theory and case study. Proc 2001 Winter Simul Conf 1:486–491Google Scholar
  19. Lacy RC, Lindenmayer DB (1995) A simulation study of the impacts of population subdivision on the mountain brushtail possum Trichosurus caninus Ogilby (Phalangeridae: Marsupialia), in south-eastern Australia. II. Loss of genetic variation within and between subpopulations. Biol Conserv 73:131–142Google Scholar
  20. Lookingbill TR, Gardner RH, Ferrari JR, Keller CE (2010) Combining a dispersal model with network theory to assess habitat connectivity. Ecol Appl 20:427–441PubMedCrossRefGoogle Scholar
  21. McCarthy MA, Burgman MA, Ferson S (1995) Sensitivity analysis for models of population viability. Biol Conserv 73:93–100Google Scholar
  22. McCarthy MA, Andelman SJ, Possingham HP (2003) Reliability of relative predictions in population viability analysis. Conserv Biol 17:982–989CrossRefGoogle Scholar
  23. Meyer KM, Gimpel K, Brandl R (2007) Viability analysis of endangered crayfish populations. J Zool 273:364–371CrossRefGoogle Scholar
  24. Ratto M, Castelletti A, Pagano A (2012) Emulation techniques for the reduction and sensitivity analysis of complex environmental models. Environ Modell Softw 34:1–4CrossRefGoogle Scholar
  25. Ricker WE (1954) Stock and recruitment. J Fish Res Board Canada 11:559–623CrossRefGoogle Scholar
  26. Ridgeway G (2006) gbm: Generalized Boosted Regression Models. R package version 1.5–7. http://CRAN.R-project.org/package=gbm
  27. Rohmer J, Foerster E (2011) Global sensitivity analysis of large-scale numerical landslide models based on Gaussian-Process meta-modeling. Comput Geosci 37:917–927CrossRefGoogle Scholar
  28. Saltelli A, Scott M (1997) Guest editorial: the role of sensitivity analysis in the corroboration of models and its link to model structural and parametric uncertainty. Reliab Eng Syst Safe 57:1–4CrossRefGoogle Scholar
  29. Saltelli A, Tarantola S, Chan KPS (1999) A quantitative model-independent method for global sensitivity analysis of model output. Technometrics 41:39–56CrossRefGoogle Scholar
  30. Sawchik J, Dufrene M, Lebrun P, Schtickzelle N, Baguette M (2002) Metapopulation dynamics of the bog fritillary butterfly: modelling the effect of habitat fragmentation. Acta Oecol 23:287–296CrossRefGoogle Scholar
  31. Schtickzelle N, Wallis De Vries MF, Baguette M (2005) Using surrogate data in population viability analysis: the case of the critically endangered cranberry fritillary butterfly. Oikos 109:89–100CrossRefGoogle Scholar
  32. Storlie CB, Swiler LP, Helton JC, Sallaberry CJ (2009) Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models. Reliab Eng Syst Safe 94:1735–1763CrossRefGoogle Scholar
  33. Sun XY, Newham LTH, Croke BFW, Norton JP (2012) Three complementary methods for sensitivity analysis of a water quality model. Environ Modell Softw 37:19–29CrossRefGoogle Scholar
  34. Urban MC, Tewksbury JJ, Sheldon KS (2012) On a collision course: competition and dispersal differences create no-analogue communities and cause extinctions during climate change. Proc R Soc B Biol Sci 279:2072–2080CrossRefGoogle Scholar
  35. Vincenzi S, Crivelli AJ, Jesensek D, Rubin JF, Poizat G, De Leo GA (2008) Potential factors controlling the population viability of newly introduced endangered marble trout populations. Biol Conserv 141:198–210CrossRefGoogle Scholar
  36. Zabel RW, Scheuerell MD, McClure MM, Williams JG (2006) The interplay between climate variability and density dependence in the population viability of Chinook salmon. Conserv Biol 20:190–200PubMedCrossRefGoogle Scholar

Copyright information

© The Society of Population Ecology and Springer Japan 2013

Authors and Affiliations

  1. 1.Environmental Decisions Group, The School of Biological SciencesThe University of QueenslandBrisbaneAustralia
  2. 2.Center for Environmental Risk ResearchNational Institute for Environmental StudiesTsukubaJapan

Personalised recommendations