Performance of multi-model ensembles for the simulation of temperature variability over Ontario, Canada

  • Aly Al Samouly
  • Chanh Nien Luong
  • Zhong LiEmail author
  • Spencer Smith
  • Brian Baetz
  • Maysara Ghaith
Original Article


Climate ensembles utilize outputs from multiple climate models to estimate future climate patterns. These multi-model ensembles generally outperform individual climate models. In this paper, the performance of seven global climate model and regional climate model combinations were evaluated for Ontario, Canada. Two multi-model ensembles were developed and tested, one based on the mean of the seven combinations and the other based on the median of the same seven models. The performance of the multi-model ensembles were evaluated on 12 meteorological stations, as well as for the entire domain of Ontario, using three temperature variables (average surface temperature, maximum surface temperature, and minimum surface temperature). Climate data for developing and validating the multi-model ensembles were collected from three major sources: the North American Coordinated Regional Downscaling Experiment, the Digital Archive of Canadian Climatological Data, and the Climactic Research Unit’s TS v4.00 dataset. The results showed that the climate ensemble based on the mean generally outperformed the one based on the median, as well as each of the individual models. Future predictions under the Representative Concentration Pathway 4.5 (RCP4.5) scenario were generated using the multi-model ensemble based on the mean. This study provides credible and useful information for climate change mitigation and adaption in Ontario.


Regional climate model NA-CORDEX Multi-model ensemble Temperature variability Ontario 



This research was supported by the Natural Science and Engineering Research Council of Canada. We acknowledge the World Climate Research Programme’s Working Group on Regional Climate, and the Working Group on Coupled Modelling, former coordinating body of CORDEX and responsible panel for CMIP5. We also thank the climate modelling groups (listed in Table 2 of this paper) for producing and making available their model output. We also acknowledge the Earth System Grid Federation infrastructure an international effort led by the U.S. Department of Energy’s Program for Climate Model Diagnosis and Intercomparison, the European Network for Earth System Modelling and other partners in the Global Organisation for Earth System Science Portals (GO-ESSP). We would like to express our very great appreciation to Dr. Alessandro Selvitella for his valuable advice and guidance for the statistical techniques used in this research paper.


  1. Barfus K, Bernhofer C (2014) Assessment of GCM performances for the Arabian Peninsula, Brazil, and Ukraine and indications of regional climate change. Environ Earth Sci 72:4689–4703. CrossRefGoogle Scholar
  2. Chai T, Draxler RR (2014) Root mean square error (RMSE) or mean absolute error (MAE)?—arguments against avoiding RMSE in the literature. Geosci Model Dev 7:1247–1250. CrossRefGoogle Scholar
  3. Dasari HP, Salgado R, Perdigao J, Challa VS (2014) A regional climate simulation study using WRF-ARW model over Europe and evaluation for extreme temperature weather events International. J Atmos Sci 704079:1–22Google Scholar
  4. Demerse C (2016) Ignoring climate change will cost us too—big time. Clean Energy Canada. Accessed 22 Sep 2017
  5. Devineni N, Sankarasubramanian A, Ghosh S (2008) Multimodel ensembles of streamflow forecasts: role of predictor state in developing optimal combinations. Water Resour Res 44:W09404. CrossRefGoogle Scholar
  6. Giorgi F, Jones C, Asrar GR (2009) Addressing climate information needs at the regional level: the CORDEX framework. World Meteorol Organ (WMO) Bull 58:175Google Scholar
  7. Hagedorn R, Doblas-Reyes FJ, Palmer TN (2005) The rationale behind the success of multi-model ensembles in seasonal forecasting—I. Basic concept. Tellus A 57:219–233. Google Scholar
  8. Harris I, Jones PD, Osborn TJ, Lister DH (2014) Updated high-resolution grids of monthly climatic observations—the CRU TS3.10 dataset. Int J Climatol 34:623–642. CrossRefGoogle Scholar
  9. Herrmann F, Kunkel R, Ostermann U, Vereecken H, Wendland F (2016) Projected impact of climate change on irrigation needs and groundwater resources in the metropolitan area of Hamburg (Germany) Environ Earth Sci 75
  10. Huo AD, Li H (2013) Assessment of climate change impact on the stream-flow in a typical debris flow watershed of Jianzhuangcuan catchment in Shaanxi Province. China Environ Earth Sci 69:1931–1938. CrossRefGoogle Scholar
  11. IPCC (2013) Climate change 2013: The physical science basis. In: Stocker TF, Qin D, Plattner G-K, Tignor M, Allen SK, Boschung J, Nauels A, Xia Y, Bex V, Midgley PM (eds) Contribution of working group I to the fifth assessment report of the intergovernmental panel on climate change. Cambridge University Press, Cambridge, pp 1535Google Scholar
  12. Jarsjo J, Tornqvist R, Su Y (2017) Climate-driven change of nitrogen retention-attenuation near irrigated fields: multi-model projections for Central Asia. Environ Earth Sci 76
  13. Katz RW (1992) Role of statistics in the validation of general circulation models. Clim Res 2:35–45CrossRefGoogle Scholar
  14. Kirtman BP, Min D (2009) Multimodel ensemble ENSO prediction with CCSM and CFS. Mon Weather Rev 137:2908–2930CrossRefGoogle Scholar
  15. Krishnamurti TN et al (2000) Multimodel ensemble forecasts for weather and seasonal climate. J Clim 13:4196–4216CrossRefGoogle Scholar
  16. Lambert SJ, Boer GJ (2001) CMIP1 evaluation and intercomparison of coupled. climate models. Clim Dyn 17:83–106CrossRefGoogle Scholar
  17. Laprise R et al (2013) Climate projections over CORDEX Africa domain using the fifth-generation Canadian Regional Climate Model (CRCM5). Clim Dyn 41:3219–3246. CrossRefGoogle Scholar
  18. Lee JY, Wang B (2014) Future change of global monsoon in the CMIP5. Climate Dynamics 42:101–119. CrossRefGoogle Scholar
  19. Li Z, Huang G, Wang X, Han J, Fan Y (2016) Impacts of future climate change on river discharge based on hydrological inference: a case study of the Grand River Watershed in Ontario. CanSci Tot Environ 548:198–210. CrossRefGoogle Scholar
  20. Lucas-Picher P, Somot S, Deque M, Decharme B, Alias A (2013) Evaluation of the regional climate model ALADIN to simulate the climate over North America in the CORDEX framework. Clim Dyn 41:1117–1137. CrossRefGoogle Scholar
  21. Mezghani A et al (2017) CHASE-PL Climate Projection dataset over Poland—Bias adjustment of EURO-CORDEX simulations. Earth Syst Sci Data Discuss 2017:1–29. CrossRefGoogle Scholar
  22. MOECC (2011) Climate Ready: Ontario’s Adaptation Strategy and Action Plan 2011–2014. Ontario Ministry of the Environment and Climate Change, CanadaGoogle Scholar
  23. Nagelkerke NJD (1991) A note on a general definition of the coefficient of determination. Biometrika 78:691–692. CrossRefGoogle Scholar
  24. Palmer TN, Doblas-Reyes FJ, Hagedorn R, Weisheimer A (2005) Probabilistic prediction of climate using multi-model ensembles: from basics to applications. Philos Trans R Soc B 360:1991–1998CrossRefGoogle Scholar
  25. Perera AH, Euler D, Thompson ID (2000) Ecology of a managed terrestrial landscape: patterns and processes of forest landscapes in Ontario. UBC Press in cooperation with the Ontario Ministry of Natural Resources, VancouverGoogle Scholar
  26. Ragone F, Lucarini V, Lunkeit F (2016) A new framework for climate sensitivity and prediction: a modelling perspective. Clim Dyn 46:1459–1471. CrossRefGoogle Scholar
  27. Rotstayn LD, Jeffrey SJ, Collier MA, Dravitzki SM, Hirst AC, Syktus JI, Wong KK (2012) Aerosol- and greenhouse gas-induced changes in summer rainfall and circulation in the Australasian region: a study using single-forcing climate simulations. Atmos Chem Phys 12:6377–6404. CrossRefGoogle Scholar
  28. Rozante J, Moreira D, Godoy R, Fernandes A (2014) Multi-model ensemble: technique and validation. Geosci Model Dev Discuss 7:2933–2959CrossRefGoogle Scholar
  29. Suklitsch M, Gobiet A, Truhetz H, Awan NK, Göttel H, Jacob D (2011) Error characteristics of high resolution regional climate models over the Alpine area. Clim Dyn 37:377–390CrossRefGoogle Scholar
  30. Taylor KE (2001) Summarizing multiple aspects of model performance in a single diagram. J Geophys Res Atmos 106:7183–7192. CrossRefGoogle Scholar
  31. Tebaldi C, Knutti R (2007) The use of the multi-model ensemble in probabilistic climate projections. Philos Trans R Soc A 365:2053–2075. CrossRefGoogle Scholar
  32. Thomson AM et al (2011) RCP4. 5: a pathway for stabilization of radiative forcing by 2100. Clim Change 109:77CrossRefGoogle Scholar
  33. Wagner T, Themessl M, Schuppel A, Gobiet A, Stigler H, Birk S (2017) Impacts of climate change on stream flow and hydro power generation in the Alpine region Environ Earth Sci. Google Scholar
  34. Wallach D, Mearns L, Ruane A, Rotter R, Asseng S (2016) Lessons from climate modeling on the design and use of ensembles for crop modeling. Clim Change 139:551–564. CrossRefGoogle Scholar
  35. Wang X et al (2013) A stepwise cluster analysis approach for downscaled climate projection—a Canadian case study. Environ Model Softw 49:141–151CrossRefGoogle Scholar
  36. Wang XQ, Huang GH, Lin QG, Nie XH, Liu JL (2015) High-resolution temperature and precipitation projections over Ontario, Canada: a coupled dynamical-statistical approach. Quart J R Meteorol Soc 141:1137–1146CrossRefGoogle Scholar
  37. Weigel AP, Knutti R, Liniger MA, Appenzeller C (2010) Risks of model weighting in multimodel climate projections. J Clim 23:4175–4191. CrossRefGoogle Scholar
  38. Wotton B, Martell D, Logan K (2003) Climate change and people-caused forest fire occurrence in Ontario. Clim Change 60:275–295CrossRefGoogle Scholar
  39. Xue PF, Pal JS, Ye XY, Lenters JD, Huang CF, Chu PY (2017) Improving the simulation of large lakes in regional climate modeling: two-way lake–atmosphere coupling with a 3D hydrodynamic model of the great lakes. J Clim 30:1605–1627. CrossRefGoogle Scholar
  40. Yan RH, Gao JF, Li LL (2016) Streamflow response to future climate and land use changes in Xinjiang basin, China. Environ Earth Sci 75
  41. Zhai Y, Huang G, Wang X, Zhou X, Lu C, Li Z (2018) Future projections of temperature changes in Ottawa, Canada through stepwise clustered downscaling of multiple GCMs under RCPs. Clim Dyn. Google Scholar
  42. Zhang Q, Dool H, Saha S, Mendez M, Becker E, Peng P, Huang J (2011) Preliminary evaluation of multi-model ensemble system for monthly and seasonal prediction. In: 36th NOAA annual climate diagnostics and prediction workshop, Fort Worth, USA, 3–6 October 2011. Science and Technology Infusion Climate Bulletin, pp 124–131Google Scholar
  43. Zhao N, Chen CF, Zhou X, Yue TX (2015) A comparison of two downscaling methods for precipitation in China. Environ Earth Sci 74:6563–6569. CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Civil EngineeringMcMaster UniversityHamiltonCanada
  2. 2.Department of Computing and Software EngineeringMcMaster UniversityHamiltonCanada

Personalised recommendations