Statistical Significance of Small Ensembles of Simulations and Detection of the Internal Climate Variability: An Excitable Ocean System Case Study


The correct mathematical approach to climate change requires the knowledge of the time-dependent system’s pullback/snapshot attractor (PBA). Once the governing equations and external forcing are known, the PBA can be estimated by performing an ensemble simulation (ES) of many forward time integrations differing only by their respective initialization; the resulting ensemble mean and spread are usually considered as representative of the forced and internal variability (FV and IV), respectively. In this paper the PBA of an excitable conceptual ocean model subjected to an idealized decadal time-scale aperiodic forcing is determined and is then used to show that the system’s relaxation oscillations contribute substantially to the ensemble mean, despite their intrinsic nature: as a consequence, a clear separation between the FV and IV is impossible in this case study. This provides an example of dynamical behaviour which may be typical of climate ESs under fluctuating aperiodic forcing. The impact of the number of ensemble members N on the statistical significance of the ES is then investigated. The complexity of realistic climate modelling currently imposes N = O(100): how significant is the statistical information derived from such small ESs? To answer this question for the present case study, the knowledge of the PBA is exploited to carry out a systematic comparison between the latter and small ESs with N = 50, also by using novel quantifiers specifically conceived for this purpose. The results reveal a remarkable significance of such ESs beyond the predictability time and may provide useful information for the design of future realistic ESs.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17


  1. 1.

    Ghil, M., Chekroun, M.D., Simonnet, E.: Climate dynamics and fluid mechanics: natural variability and related uncertainties. Physica D 237, 2111–2126 (2008)

    ADS  MathSciNet  MATH  Google Scholar 

  2. 2.

    Pierrehumbert, R.T.: Principles of Planetary Climate. Cambridge University Press, Cambridge (2010)

    Google Scholar 

  3. 3.

    Chekroun, M.D., Simonnet, E., Ghil, M.: Stochastic climate dynamics: Random attractors and time-dependent invariant measures. Physica D 240, 1685–1700 (2011)

    ADS  MathSciNet  MATH  Google Scholar 

  4. 4.

    Bódai, T., Károlyi, G., Tél, T.: A chaotically driven model climate: extreme events and snapshot attractors. Nonlinear. Process. Geophys. 18, 573–580 (2011)

    ADS  Google Scholar 

  5. 5.

    Bódai, T., Károlyi, G., Tél, T.: Fractal snapshot components in chaos induced by strong noise. Phys. Rev. 83E, 046201 (2011)

    ADS  Google Scholar 

  6. 6.

    Bódai, T., Tél, T.: Annual variability in a conceptual climate model: snapshot attractors, hysteresis in extreme events, and climate sensitivity. Chaos 22, 023110 (2012)

    ADS  MathSciNet  Google Scholar 

  7. 7.

    Ghil, M.: The complex physics of climate change and climate sensitivity: a grand unification (Alfred Wegener Medal Lecture). European Geosciences Union General Assembly 2012, Vienna, Austria, European Geosciences Union, EGU2012-14438-1 (2012)

  8. 8.

    Ghil, M.: A mathematical theory of climate sensitivity or, how to deal with both anthropogenic forcing and natural variability? In: Chang, C.-P., Ghil, M., Latif, M., Wallace, J. (eds.) Climate Change: Multidecadal and Beyond, vol. 6, pp. 31–52. World Scientific Publishing Co., Singapore (2015)

    Google Scholar 

  9. 9.

    Daron, J.D., Stainforth, D.A.: On quantifying the climate of the nonautonomous Lorenz-63 model. Chaos 25, 043103 (2015)

    ADS  MathSciNet  Google Scholar 

  10. 10.

    Drótos, G., Bódai, T., Tél, T.: Probabilistic concepts in a changing climate: a snapshot attractor picture. J. Clim. 28, 3275–3288 (2015)

    ADS  Google Scholar 

  11. 11.

    Drótos, G., Bódai, T., Tél, T.: On the importance of the convergence to climate attractors. Eur. Phys. J. Spec. Top. 226, 2031–2038 (2017)

    Google Scholar 

  12. 12.

    Ragone, F., Lucarini, V., Lunkeit, F.: A new framework for climate sensitivity and prediction: a modelling perspective. Clim. Dyn. 46, 1459–1471 (2016)

    Google Scholar 

  13. 13.

    Herein, M., Márfy, J., Drótos, G., Tél, T.: Probabilistic concepts in intermediate-complexity climate models: a snapshot attractor picture. J. Clim. 29, 259–272 (2016)

    ADS  Google Scholar 

  14. 14.

    Pierini, S., Ghil, M., Chekroun, M.D.: Exploring the pullback attractors of a low-order quasigeostrophic ocean model: the deterministic case. J. Clim. 29, 4185–4202 (2016)

    ADS  Google Scholar 

  15. 15.

    Herein, M., Drótos, G., Haszpra, T., Márfy, J., Tél, T.: The theory of parallel climate realizations as a new framework for teleconnection analysis. Sci. Rep. 7, 44529 (2017)

    ADS  Google Scholar 

  16. 16.

    Ghil, M.: A century of nonlinearity in the geosciences. Earth Space Sci. 6, 1007–1042 (2019)

    ADS  Google Scholar 

  17. 17.

    Stocker, T.F., et al. (eds.): Climate Change 2013 (IPCC): The Physical Science Basis. Cambridge University Press, Cambridge (2013)

    Google Scholar 

  18. 18.

    Drótos, G., Bódai, T., Tél, T.: Quantifying nonergodicity in nonautonomous dissipative dynamical systems: an application to climate change. Phys. Rev. E. 94, 022214 (2016)

    ADS  MathSciNet  Google Scholar 

  19. 19.

    Eckmann, J.-P., Ruelle, D.: Ergodic theory of chaos and strange attractors. Rev. Mod. Phys. 57, 617–656 (1985)

    ADS  MathSciNet  MATH  Google Scholar 

  20. 20.

    Nicolis, G.: Introduction to Nonlinear Science. Cambridge University Press, Cambridge (1995)

    Google Scholar 

  21. 21.

    Ott, E.: Chaos in Dynamical Systems. Cambridge University Press, Cambridge (2002)

    Google Scholar 

  22. 22.

    Tél, T., Gruiz, M.: Chaotic Dynamics. Cambridge University Press, Cambridge (2006)

    Google Scholar 

  23. 23.

    Strogatz, S.H.: Nonlinear Dynamics and Chaos: with Applications to Physics, Biology, Chemistry, and Engineering. CRC Press, Boca Raton (2015)

    Google Scholar 

  24. 24.

    Dijkstra, H.A., Ghil, M.: Low-frequency variability of the large-scale ocean circulation: a dynamical systems approach. Rev. Geophys. 43, 3002 (2005)

    ADS  Google Scholar 

  25. 25.

    Ghil, M.: The wind-driven ocean circulation: applying dynamical systems theory to a climate problem. Discret. Contin. Dyn. Syst. A 37, 189–228 (2017)

    MathSciNet  MATH  Google Scholar 

  26. 26.

    Pierini, S.: Low-frequency variability, coherence resonance and phase selection in a low-order model of the wind-driven ocean circulation. J. Phys. Oceanogr. 41, 1585–1604 (2011)

    ADS  Google Scholar 

  27. 27.

    Pierini, S., Chekroun, M.D., Ghil, M.: The onset of chaos in nonautonomous dissipative dynamical systems: a low-order ocean–model case study. Nonlinear. Process. Geophys. 25, 671–692 (2018)

    ADS  Google Scholar 

  28. 28.

    Pikovsky, A.S., Kurths, J.: Coherence resonance in noise-driven excitable systems. Phys. Rev. Lett. 78, 775–778 (1997)

    ADS  MathSciNet  MATH  Google Scholar 

  29. 29.

    Pierini, S.: Coherence resonance in a double-gyre model of the Kuroshio Extension. J. Phys. Oceanogr. 40, 238–248 (2010)

    ADS  Google Scholar 

  30. 30.

    Crucifix, M.: Oscillators and relaxation phenomena in Pleistocene climate theory. Phil. Trans. R. Soc. A370, 1140–1165 (2012)

    ADS  Google Scholar 

  31. 31.

    Van der Pol, B.: On relaxation-oscillations. The London, Edinburgh and Dublin. Philos. Mag. J. Sci. 2, 978–992 (1926)

    Google Scholar 

  32. 32.

    Bessières, L., Leroux, S., Brankart, J.-M., Molines, J.-M., Moine, M.-P., Bouttier, P.-A., Penduff, T., Terray, L., Barnier, B., Sérazin, G.: Development of a probabilistic ocean modelling system based on NEMO 3.5: application at eddying resolution. Geosci. Model Dev. 10, 1091–1106 (2017)

    ADS  Google Scholar 

  33. 33.

    Maher, N., Milinski, S., Suarez-Gutierrez, L., Botzet, M., Dobrynin, M., Kornblueh, L., Kröger, J., Takano, Y., Ghosh, R., Hedemann, C., Li, C., Li, H., Manzini, E., Notz, D., Putrasahan, D., Boysen, L., Claussen, M., Ilyina, T., Olonscheck, D., Raddatz, T., Stevens, B., Marotzke, J.: The Max Planck Institute grand ensemble: enabling the exploration of climate system variability. J. Adv. Mod. Earth Syst. (2019).

    Article  Google Scholar 

  34. 34.

    Pierini, S.: A Kuroshio Extension system model study: decadal chaotic self-sustained oscillations. J. Phys. Oceanogr. 36, 1605–1625 (2006)

    ADS  Google Scholar 

  35. 35.

    Pierini, S.: On the crucial role of basin geometry in double-gyre models of the Kuroshio Extension. J. Phys. Oceanogr. 38, 1327–1333 (2008)

    ADS  Google Scholar 

  36. 36.

    Pierini, S., Dijkstra, H.A.: Low-frequency variability of the Kuroshio Extension. Nonlinear. Process. Geophys. 16, 665–675 (2009)

    ADS  Google Scholar 

  37. 37.

    Pierini, S.: Stochastic tipping points in climate dynamics. Phys. Rev. E 85, 027101 (2012)

    ADS  Google Scholar 

  38. 38.

    Pierini, S.: Ensemble simulations and pullback attractors of a periodically forced double-gyre system. J. Phys. Oceanogr. 44, 3245–3254 (2014)

    ADS  Google Scholar 

  39. 39.

    Vannitsem, S., De Cruz, L.: A 24-variable low-order coupled ocean–atmosphere model: OA-QG-WS v2. Geosci. Model Dev. 7, 649–662 (2014)

    ADS  Google Scholar 

  40. 40.

    De Cruz, L., Schubert, S., Demaeyer, J., Lucarini, V., Vannitsem, S.: Exploring the Lyapunov instability properties of high-dimensional atmospheric and climate models. Nonlinear. Process. Geophys. 25, 387–412 (2018)

    ADS  Google Scholar 

  41. 41.

    Vallis, G.K.: Atmospheric and Oceanic Fluid Dynamics. Cambridge University Press, Cambridge (2006)

    Google Scholar 

  42. 42.

    Arnold, L.: Random Dynamical Systems. Springer, New York (1998)

    Google Scholar 

  43. 43.

    Rasmussen, M.: Attractivity and Bifurcation for Nonautonomous Dynamical Systems. Springer, New York (2007)

    Google Scholar 

  44. 44.

    Bódai, T., Károlyi, G., Tél, T.: Driving a conceptual model climate by different processes: Snapshot attractors and extreme events. Phys. Rev. E 87, 022822 (2013)

    ADS  Google Scholar 

  45. 45.

    Sérazin, G., Jaymond, A., Leroux, S., Penduff, T., Bessières, L., Llovel, W., Barnier, B., Molines, J.-M., Terray, L.: A global probabilistic study of the ocean heat content low-frequency variability: atmospheric forcing versus oceanic chaos. Geophys. Res. Lett. 44, 5580–5589 (2017)

    ADS  Google Scholar 

  46. 46.

    Penduff, T., Barnier, B., Terray, L., Sérazin, G., Gregorio, S., Brankart, J.-M., Moine, M.-P., Molines, J.-M., Brasseur, P.: Ensembles of eddying ocean simulations for climate. CLIVAR Exchanges No. 65, Vol. 19, No. 2 (2014)

  47. 47.

    Leroux, S., Penduff, T., Bessières, L., Molines, J.-M., Brankart, J.-M., Sérazin, G., Barnier, B., Terray, L.: Intrinsic and atmospherically forced variability of the AMOC: insights from a large-ensemble ocean hindcast. J. Clim. 31, 1183–1203 (2018)

    ADS  Google Scholar 

  48. 48.

    Pikovsky, A., Rosenblum, M., Kurths, J.: Synchronization: A Universal Concept in Nonlinear Sciences. Cambridge Nonlinear Science Series 12. Cambridge University Press, Cambridge (2003)

    Google Scholar 

  49. 49.

    Shannon, C.E.: The mathematical theory of communication. Bell Syst. Technol. J. 27, 379–423 (1948)

    MathSciNet  MATH  Google Scholar 

  50. 50.

    Vissio, G., Lucarini, V.: Evaluating a stochastic parametrization for a fast–slow system using the Wasserstein distance. Nonlinear. Process. Geophys. 25, 413–427 (2018)

    ADS  Google Scholar 

Download references


This work was funded by the MOMA (PNRA16_00196) and IPSODES (PNRA18_00199-C) projects of the Italian “Programma Nazionale di Ricerche in Antartide”. Support from the University of Naples Parthenope (Contract No. DSTE315B) is also kindly acknowledged.

Author information



Corresponding author

Correspondence to Stefano Pierini.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by Valerio Lucarini.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pierini, S. Statistical Significance of Small Ensembles of Simulations and Detection of the Internal Climate Variability: An Excitable Ocean System Case Study. J Stat Phys 179, 1475–1495 (2020).

Download citation


  • Pullback attractors
  • Ensemble simulations
  • Climate change
  • Internal climate variability
  • Reduced order climate models
  • Excitable systems