Multi-level emulation of complex climate model responses to boundary forcing data

  • Giang T. Tran
  • Kevin I. C. Oliver
  • Philip B. Holden
  • Neil R. Edwards
  • András Sóbester
  • Peter Challenor


Climate model components involve both high-dimensional input and output fields. It is desirable to efficiently generate spatio-temporal outputs of these models for applications in integrated assessment modelling or to assess the statistical relationship between such sets of inputs and outputs, for example, uncertainty analysis. However, the need for efficiency often compromises the fidelity of output through the use of low complexity models. Here, we develop a technique which combines statistical emulation with a dimensionality reduction technique to emulate a wide range of outputs from an atmospheric general circulation model, PLASIM, as functions of the boundary forcing prescribed by the ocean component of a lower complexity climate model, GENIE-1. Although accurate and detailed spatial information on atmospheric variables such as precipitation and wind speed is well beyond the capability of GENIE-1’s energy-moisture balance model of the atmosphere, this study demonstrates that the output of this model is useful in predicting PLASIM’s spatio-temporal fields through multi-level emulation. Meaningful information from the fast model, GENIE-1 was extracted by utilising the correlation between variables of the same type in the two models and between variables of different types in PLASIM. We present here the construction and validation of several PLASIM variable emulators and discuss their potential use in developing a hybrid model with statistical components.


Probabilistic prediction Multi-level emulators Model hierarchy Spatio-temporal data Intermediate complexity model 

Supplementary material

382_2018_4205_MOESM1_ESM.pdf (30.1 mb)
Supplementary material 1 (pdf 30863 KB)


  1. Barnett TP, Latif M, Graham N, Flugel M, Pazan S, White W (1993) ENSO and ENSO-related predictability. Part I: Prediction of equatorial Pacific Sea surface temperature with a hybrid coupled ocean–atmosphere model. J Clim 6(8):1545–1566CrossRefGoogle Scholar
  2. Barnett TP, Preisendorfer R (1987) Origins and levels of monthly and seasonal forecast skill for United states furface air temperature determined by canonical correlation analysis. Mon Weather Rev 115(9):1825–1850CrossRefGoogle Scholar
  3. Bastos LS, O’Hagan A (2009) Diagnostics for Gaussian process emulators. Technometrics 51(4):425–438CrossRefGoogle Scholar
  4. Bayarri MJ, Berger JO, Cafeo J, Garcia-Donato G, Liu F, Palomo J, Parthasarathy RJ, Paulo R, Sacks J, Walsh D (2007) Computer model validation with functional output. Ann Stat 35(5):1874–1906. CrossRefGoogle Scholar
  5. Boukouvalas A, Cornford D (2009) Dimension reduction for multivariate emulation. Tech. Rep. November 2009, Aston University, BirminghamGoogle Scholar
  6. Bounceur N, Crucifix M, Wilkinson RD (2015) Global sensitivity analysis of the climate-vegetation system to astronomical forcing: an emulator-based approach. Earth Syst Dyn 6(1):205–224CrossRefGoogle Scholar
  7. Challenor PG, McNeall D, Gattiker J (2010) Assessing the probability of rare climate events. In: O’Hagan A, West M (eds) The Oxford handbook of applied bayesian analysis. Oxford University Press, New York, pp 403–430 chap 16Google Scholar
  8. Cimatoribus AA, Drijfhout SS, Dijkstra HA (2012) A global hybrid coupled model based on atmosphere-SST feedbacks. Clim Dyn 38(3–4):745–760CrossRefGoogle Scholar
  9. Conti S, O’Hagan A (2010) Bayesian emulation of complex multi-output and dynamic computer models. J Stat Plann Inference 140(3):640–651CrossRefGoogle Scholar
  10. Cook RD, Nachtsheim CJ (1980) A comparison of algorithms for constructing exact D-optimal designs. Technometrics 22:315–324CrossRefGoogle Scholar
  11. Edwards NR, Marsh R (2005) Uncertainties due to transport-parameter sensitivity in an efficient 3-D ocean-climate model. Clim Dyn 24(4):415–433CrossRefGoogle Scholar
  12. Edwards NR, Cameron D, Rougier J (2011) Precalibrating an intermediate complexity climate model. Clim Dyn 37(7–8):1469–1482CrossRefGoogle Scholar
  13. Foley AM, Holden PB, Edwards NR, Mercure JF, Salas P, Pollitt H, Chewpreecha U (2016) Climate model emulation in an integrated assessment framework: a case study for mitigation policies in the electricity sector. Earth Syst Dyn 7(1):119–132CrossRefGoogle Scholar
  14. Forrester AI, Sóbester A, Keane AJ (2007) Multi-fidelity optimization via surrogate modelling. Proc R Soc A Math Phys Eng Sci 463(2088):3251–3269CrossRefGoogle Scholar
  15. Fraedrich K (2012) A suite of user-friendly global climate models: hysteresis experiments. Eur Phys J Plus 127(5):53CrossRefGoogle Scholar
  16. Fraedrich K, Jansen H, Kirk E, Luksch U, Lunkeit F (2005) The planet simulator: towards a user friendly model. Meteorol Z 14(3):299–304CrossRefGoogle Scholar
  17. Geil KL, Zeng X (2015) Quantitative characterization of spurious numerical oscillations in 48 CMIP5 models. Geophys Res Lett 42(12):5066–5073. CrossRefGoogle Scholar
  18. Haberkorn K, Sielmann F, Lunkeit F, Kirk E, Schneidereit A, Fraedrich K (2009) Planet simulator climate. Tech. rep., Meteorologisches Institut, Universität HamburgGoogle Scholar
  19. Hardoon DR, Szedmak S, Shawe-Taylor J (2004) Canonical correlation analysis; an overview with application to learning methods. Neural Comput 16(12):2639–2664CrossRefGoogle Scholar
  20. Higdon D, Gattiker J, Williams B, Rightley M (2008) Computer model calibration using high-dimensional output. J Am Stat Assoc 103(482):570–583CrossRefGoogle Scholar
  21. Holden PB, Edwards NR, Garthwaite PH, Wilkinson RD (2015) Emulation and interpretation of high-dimensional climate model outputs. J Appl Stat 42(9):2038–2055CrossRefGoogle Scholar
  22. Holden PB, Edwards NR (2010) Dimensionally reduced emulation of an AOGCM for application to integrated assessment modelling. Geophys Res Lett 37(21): L21707CrossRefGoogle Scholar
  23. Holden PB, Edwards NR, Oliver KIC, Lenton TM, Wilkinson RD (2010) A probabilistic calibration of climate sensitivity and terrestrial carbon change in GENIE-1. Clim Dyn 35(5):785–806CrossRefGoogle Scholar
  24. Holden PB, Edwards NR, Müller SA, Oliver KIC, Death RM, Ridgwell A (2013) Controls on the spatial distribution of oceanic d13CDIC. Biogeosciences 10(3):1815–1833. CrossRefGoogle Scholar
  25. Holden PB, Edwards NR, Garthwaite PH, Fraedrich K, Lunkeit F, Kirk E, Labriet M, Kanudia A, Babonneau F (2014) PLASIM-ENTSem v1.0: a spatio-temporal emulator of future climate change for impacts assessment. Geosci Model Dev 7(1):433–451CrossRefGoogle Scholar
  26. Holden PB, Edwards NR, Fraedrich K, Kirk E, Lunkeit F, Zhu X (2016) PLASIM GENIE v1.0: a new intermediate complexity AOGCM. Geosci Model Dev 9:3347–3361. CrossRefGoogle Scholar
  27. Kennedy MC, O’Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87(1):1–13CrossRefGoogle Scholar
  28. Kennedy MC, O’Hagan A (2001) Bayesian calibration of computer models. J R Stat Soc Ser B (Stat Methodol) 63(3):425–464CrossRefGoogle Scholar
  29. La Lee, Carslaw KS, Pringle KJ, Mann GW (2012) Mapping the uncertainty in global CCN using emulation. Atmos Chem Phys 12(20):9739–9751CrossRefGoogle Scholar
  30. Labriet M, Joshi SR, Vielle M, Holden PB, Edwards NR, Kanudia A, Loulou R, Babonneau F (2015) Worldwide impacts of climate change on energy for heating and cooling. Mitig Adapt Strat Glob Chang 20(7):1111–1136CrossRefGoogle Scholar
  31. Lenton TM, Williamson MS, Edwards NR, Marsh R, Price aR, Ridgwell aJ, Shepherd JG, Cox SJ (2006) Millennial timescale carbon cycle and climate change in an efficient Earth system model. Clim Dyn 26(7–8):687–711CrossRefGoogle Scholar
  32. Liakka J, Nilsson J, Lofverstrom M (2012) Interactions between stationary waves and ice sheets: linear versus nonlinear atmospheric response. Clim Dyn 38(5–6):1249–1262CrossRefGoogle Scholar
  33. Loeppky JL, Sacks J, Welch WJ (2009) Choosing the sample size of a computer experiment: a practical guide. Technometrics 51(4):366–376CrossRefGoogle Scholar
  34. Lucarini V, Fraedrich K, Lunkeit F (2010) Thermodynamic analysis of snowball earth hysteresis experiment: efficiency, entropy production and irreversibility. Q J R Meteorol Soc 136(646):2–11. CrossRefGoogle Scholar
  35. Lunt DJ, Williamson MS, Valdes PJ, Lenton TM, Marsh R (2006) Comparing transient, accelerated, and equilibrium simulations of the last 30,000 years with the GENIE-1 model. Clim Past 2(2):221–235CrossRefGoogle Scholar
  36. Maniyar DM, Cornford D, Boukouvalas A (2007) Dimensionality reduction in the emulator setting. Tech. Rep. October 2007, Neural Computing research group. Aston University, BirminghamGoogle Scholar
  37. Mardia KV, Marshall RJ (1984) Maximum likelihood estimation of models for residual covariance in spatial regression. Biometrika 71(1):135–146CrossRefGoogle Scholar
  38. Marsh R, Sa Müller, Yool A, Edwards NR (2011) Incorporation of the C-GOLDSTEIN efficient climate model into the GENIE framework: “eb_go_gs” configurations of GENIE. Geosci Model Dev 4(4):957–992CrossRefGoogle Scholar
  39. Matthews HD, Caldeira K (2007) Transient climate-carbon simulations of planetary geoengineering. Proc Nat Acad Sci USA 104(24):9949–54CrossRefGoogle Scholar
  40. Mcneall DJ (2008) Dimension reduction in the Bayesian analysis of a numerical climate model. PhD Thesis, University of SouthamptonGoogle Scholar
  41. Morris MD, Mitchell TJ (1995) Exploratory designs for computational experiments. J Stat Plann Inference 43:381–402CrossRefGoogle Scholar
  42. Oa Saenko, Schmittner A, Weaver AJ (2004) The Atlantic Pacific seesaw. J Clim 17(11):2033–2038.;2 CrossRefGoogle Scholar
  43. Oakley JE, O’Hagan A (2004) Probabilistic sensitivity analysis of complex models: a Bayesian approach. J R Stat Soc Ser B (Stat Methodol) 66(3):751–769. CrossRefGoogle Scholar
  44. Osborne JW, Costello AB (2009) Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pan Pac Manag Rev 12(2):131–146Google Scholar
  45. Peltier W (2004) Global glacial isostasy and the surface of the ice-age earth: the ICE-5G (VM2) model and GRACE. Annu Rev Earth Planet Sci 32(1):111–149CrossRefGoogle Scholar
  46. Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. The MIT Press, CambridgeGoogle Scholar
  47. Richman MB (1986) Rotation of principal components. J Climatol 6(3):293–335CrossRefGoogle Scholar
  48. Romanova V, Lohmann G, Grosfeld K, Butzin M (2006) The relative role of oceanic heat transport and orography on glacial climate. Quatern Sci Rev 25(7–8):832–845CrossRefGoogle Scholar
  49. Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–423CrossRefGoogle Scholar
  50. Santner TJ, Williams BJ, Notz WI (2003) The design and analysis of computer experiments. Springer, New YorkCrossRefGoogle Scholar
  51. Schmittner A, Silva TAM, Fraedrich K, Kirk E, Lunkeit F (2011) Effects of mountains and ice sheets on global ocean circulation. J Clim 24(11):2814–2829CrossRefGoogle Scholar
  52. Stocker TF, Johnsen SJ (2003) A minimum thermodynamic model for the bipolar seesaw. Paleoceanography 18(4):1087. CrossRefGoogle Scholar
  53. Syu HH, Neelin DJ, Gutzler D (1995) Seasonal and interannual variability in a hybrid coupled GCM. J Clim 8:2121–2143CrossRefGoogle Scholar
  54. Thompson SL, Warren SG (1982) Parameterization of outgoing infrared radiation derived from detailed radiative calculations. J Atmos Sci 39:2667–2680CrossRefGoogle Scholar
  55. Tran GT (2017) Developing a multi-level gaussian process emulator of an atmospheric general circulation model for palaeoclimate modelling. PhD Thesis, University of SouthamptonGoogle Scholar
  56. Tran GT, Oliver KIC, Sobester A, Toal DJJ, Holden PB, Marsh R, Challenor PG, Edwards NR (2016) Building a traceable climate model hierarchy with multi-level emulators. Adv Stat Climatol Meteorol Oceanogr 2(1):1–21CrossRefGoogle Scholar
  57. Weaver AJ, Eby M, Wiebe EC, Bitz CM, Duffy PB, Ewen TL, Fanning AF, Holland MM, Macfadyen A, Matthews HD, Meissner KJ, Saenko O, Schmittner A, Wang H, Masakazu Y (2001) The UVic earth system climate model: model description, climatology, and applications to past, present and future climates. Atmos Ocean 39(4):361–428CrossRefGoogle Scholar
  58. Wilkinson RD (2010) Bayesian calibration of expensive multivariate computer experiments. In: Biegler L, Biros G, Ghattas O, Heinkenschloss M, Keyes D, Mallick B, Marzouk Y, Tenorio L, Waanders BB, Willcox K (eds) Computational methods for large-scale inverse problems and quantification of uncertainity. Wiley, Chichester chap 10Google Scholar
  59. Williamson M, Lenton T, Shepherd J, Edwards N (2006) An efficient numerical terrestrial scheme (ENTS) for Earth system modelling. Ecol Model 198(3–4):362–374CrossRefGoogle Scholar
  60. Williamson D, Blaker AT, Hampton C, Salter J (2015) Identifying and removing structural biases in climate models with history matching. Clim Dyn 45(5–6):1299–1324. CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Giang T. Tran
    • 1
  • Kevin I. C. Oliver
    • 1
  • Philip B. Holden
    • 2
  • Neil R. Edwards
    • 2
  • András Sóbester
    • 3
  • Peter Challenor
    • 4
  1. 1.Ocean and Earth SciencesUniversity of SouthamptonSouthamptonUK
  2. 2.Environment, Earth and EcosystemsThe Open UniversityMilton KeynesUK
  3. 3.Engineering and the Environment, University of SouthamptonSouthamptonUK
  4. 4.College of Engineering, Mathematics and Physical SciencesUniversity of ExeterExeterUK

Personalised recommendations