Sampling Hyperspheres via Extreme Value Theory: Implications for Measuring Attractor Dimensions

Abstract

The attractor Hausdorff dimension is an important quantity bridging information theory and dynamical systems, as it is related to the number of effective degrees of freedom of the underlying dynamical system. By using the link between extreme value theory and Poincaré recurrences, it is possible to estimate this quantity from time series of high-dimensional systems without embedding the data. In general \(d \le n\), where n is the dimension of the full phase-space, as the dynamics freezes some of the available degrees of freedom. This is equivalent to constraining trajectories on a compact object in phase space, namely the attractor. Information theory shows that the equality \(d=n\) holds for random systems. However, applying extreme value theory, we show that this result cannot be recovered and that \(d<n\). We attribute this effect to the curse of dimensionality, and in particular to the phenomenon of concentration of the norm observed in high-dimensional systems. We derive a theoretical expression for d(n) for Gaussian random vectors, and we show numerically that similar curse of dimensionality effects are found for random systems characterized by non-Gaussian distributions. Finally, we show that the effect of the curse of dimensionality can be quantified using the extreme value theory, thus enabling to retrieve the degree of non-randomness of a system. We provide examples issued from real-world climate and financial datasets.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. 1.

    For a definition of the concept of curse of dimensionality, we quote Cabestany et al. [9], who define it as: “the expression of all phenomena that appear with high-dimensional data, and that have most often unfortunate consequences on the behavior and performances of learning algorithms”

References

  1. 1.

    Aggarwal, C.C., Hinneburg, A., Keim, D.A.: On the surprising behavior of distance metrics in high dimensional space. In: International Conference on Database Theory, pp. 420–434. Springer, London (2001)

  2. 2.

    Arnold, B.C., Groeneveld, R.A., et al.: Bounds on expectations of linear systematic statistics based on dependent samples. Ann. Stat. 7(1), 220–223 (1979)

    MathSciNet  Article  Google Scholar 

  3. 3.

    Balakrishnan, N., Charalambides, C., Papadatos, N.: Bounds on expectation of order statistics from a finite population. J. Stat. Plan. Inference 113(2), 569–588 (2003)

    MathSciNet  Article  Google Scholar 

  4. 4.

    Barreira, L.: Dimension and Recurrence in Hyperbolic Dynamics, vol. 272. Springer, New York (2008)

    Google Scholar 

  5. 5.

    Berchtold, S., Keim, D., Kriegel, H.: An index structure for high-dimensional data. Read. Multimedia Comput. Netw. 451, 2096 (2001)

    Google Scholar 

  6. 6.

    Böhm, C., Berchtold, S., Keim, D.A.: Searching in high-dimensional spaces: index structures for improving the performance of multimedia databases. ACM Comput. Surveys (CSUR) 33(3), 322–373 (2001)

    Article  Google Scholar 

  7. 7.

    Brin, S.: Near neighbor search in large metric spaces. In: Proceedings of the 21st VLDB Conference, Zurich, Switzerland (1995)

  8. 8.

    Buschow, S., Friederichs, P.: Local dimension and recurrent circulation patterns in long-term climate simulations. Chaos 28(8), 083124 (2018)

    ADS  MathSciNet  Article  Google Scholar 

  9. 9.

    Cabestany, J., Prieto, A., Sandoval, F.: Computational Intelligence and Bioinspired Systems: 8th International Work-Conference on Artificial Neural Networks, IWANN 2005, Vilanova i la Geltrú, Barcelona, Spain, June 8–10, 2005, Proceedings, vol. 3512. Springer, Heidelberg (2005)

    Google Scholar 

  10. 10.

    Caby, T., Faranda, D., Mantica, G., Vaienti, S., Yiou, P.: Generalized dimensions, large deviations and the distribution of rare events. arXiv preprint (2018). arXiv:1812.00036

  11. 11.

    Chebyshev, P.L.: Des valeurs moyennes. J. Math. Pures Appl. 12(2), 177–184 (1867)

    Google Scholar 

  12. 12.

    De Luca, P., Messori, G., Pons, F.M.E., Faranda, D.: Dynamical systems theory sheds new light on compound climate extremes in Europe and Eastern North America. Q. J. R. Meteorol. Soc. 146(729), 1636–1650 (2020)

    Article  Google Scholar 

  13. 13.

    Faranda, D., Lucarini, V., Turchetti, G., Vaienti, S.: Numerical convergence of the block-maxima approach to the generalized extreme value distribution. J. Stat. Phys. 145(5), 1156–1180 (2011)

    ADS  MathSciNet  Article  Google Scholar 

  14. 14.

    Demartines, P.: Analyse de données par réseaux de neurones auto-organisés. PhD thesis, Grenoble INPG (1994)

  15. 15.

    Faranda, D., Vaienti, S.: Correlation dimension and phase space contraction via extreme value theory. Chaos 28(4), 041103 (2018)

    ADS  MathSciNet  Article  Google Scholar 

  16. 16.

    Faranda, D., Freitas, J.M., Lucarini, V., Turchetti, G., Vaienti, S.: Extreme value statistics for dynamical systems with noise. Nonlinearity 26(9), 2597 (2013)

    ADS  MathSciNet  Article  Google Scholar 

  17. 17.

    Faranda, D., Messori, G., Yiou, P.: Dynamical proxies of North Atlantic predictability and extremes. Sci. Rep. 7, 41278 (2017)

    ADS  Article  Google Scholar 

  18. 18.

    Faranda, D., Ghoudi, H., Guiraud, P., Vaienti, S.: Extreme value theory for synchronization of coupled map lattices. Nonlinearity 31(7), 3326 (2018)

    MathSciNet  Article  Google Scholar 

  19. 19.

    Faranda, D., Alvarez-Castro, M.C., Messori, G., Rodrigues, D., Yiou, P.: The hammam effect or how a warm ocean enhances large scale atmospheric predictability. Nat. Commun. 10(1), 1316 (2019)

    ADS  Article  Google Scholar 

  20. 20.

    Faranda, D., Sato, Y., Messori, G., Moloney, N.R., Yiou, P.: Minimal dynamical systems model of the northern hemisphere jet stream via embedding of climate data. Earth Syst. Dyn. 10(3), 555–567 (2019)

  21. 21.

    Francois, D., Wertz, V., Verleysen, M.: The concentration of fractional distances. IEEE Trans. Knowl. Data Eng. 19(7), 873–886 (2007)

    Article  Google Scholar 

  22. 22.

    Freitas, A.C.M., Freitas, J.M., Todd, M.: Hitting time statistics and extreme value theory. Probab. Theory Relat. Fields 147(3), 675–710 (2010)

    MathSciNet  Article  Google Scholar 

  23. 23.

    Grassberger, P.: Do climatic attractors exist? Nature 323(6089), 609–612 (1986)

    ADS  Article  Google Scholar 

  24. 24.

    Grassberger, P.: Finite sample corrections to entropy and dimension estimates. Phys. Lett. A 128(6–7), 369–373 (1988)

    ADS  MathSciNet  Article  Google Scholar 

  25. 25.

    Grassberger, P., Procaccia, I.: Dimensions and entropies of strange attractors from a fluctuating dynamics approach. Physica D 13(1), 34–54 (1984)

    ADS  MathSciNet  Article  Google Scholar 

  26. 26.

    Hochman, A., Alpert, P., Harpaz, T., Saaroni, H., Messori, G.: A new dynamical systems perspective on atmospheric predictability: Eastern mediterranean weather regimes as a case study. Sci. Adv. 5(6), eaau0936 (2019)

  27. 27.

    Hochman, A., Alpert, P., Kunin, P., Rostkier-Edelstein, D., Harpaz, T., Saaroni, H., Messori, G.: The dynamics of cyclones in the twenty-first century: the eastern Mediterranean as an example. Clim. Dyn. 54, 561–574 (2019)

    Article  Google Scholar 

  28. 28.

    Indyk, P., Motwani, R.: Approximate nearest neighbors: towards removing the curse of dimensionality. In: Proceedings of the Thirtieth annual ACM Symposium on Theory of Computing, pp. 604–613. ACM, New York (1998)

  29. 29.

    Katayama, N., Satoh, S.: The sr-tree: an index structure for high-dimensional nearest neighbor queries. In: ACM Sigmod Record, vol. 26, pp. 369–380. ACM, New York (1997)

  30. 30.

    Kistler, R., Collins, W., Saha, S., White, G., Woollen, J., Kalnay, E., Chelliah, M., Ebisuzaki, W., Kanamitsu, M., Kousky, V., et al.: The ncep-ncar 50-year reanalysis: monthly means cd-rom and documentation. Bull. Am. Meteorol. Soc. 82(2), 247–267 (2001)

    ADS  Article  Google Scholar 

  31. 31.

    Lin, K.-I., Jagadish, H.V., Faloutsos, C.: The tv-tree: an index structure for high-dimensional data. VLDB J. 3(4), 517–542 (1994)

    Article  Google Scholar 

  32. 32.

    Lorenz, E.N.: Dimension of weather and climate attractors. Nature 353(6341), 241–244 (1991)

    ADS  Article  Google Scholar 

  33. 33.

    Lucarini, V., Faranda, D., de Freitas, A., de Freitas, J., Holland, M., Kuna, T., Nicol, M., Todd, M., Vaienti, S.: Extremes and Recurrence in Dynamical Systems. Monographs and Tracts. Wiley, Pure and Applied Mathematics. Wiley, New York, A Wiley Series of Texts (2016)

    Google Scholar 

  34. 34.

    Lucarini, V., Faranda, D., Wouters, J.: Universal behaviour of extreme value statistics for selected observables of dynamical systems. J. Stat. Phys. 147(1), 63–73 (2012)

    ADS  MathSciNet  Article  Google Scholar 

  35. 35.

    Mayer-Kress, G.: Dimensions and entropies in chaotic systems: quantification of complex behavior. In: Proceeding of an International Workshop at the Pecos River Ranch, New Mexico, September 11–16, 1985, vol. 32. Springer, Berlin (2012)

  36. 36.

    Messori, G., Caballero, R., Faranda, D.: A dynamical systems approach to studying midlatitude weather extremes. Geophys. Res. Lett. 44(7), 3346–3354 (2017)

    ADS  Article  Google Scholar 

  37. 37.

    Nicolis, C., Nicolis, G.: Is there a climatic attractor? Nature 311, 529–532 (1984)

    ADS  Article  Google Scholar 

  38. 38.

    Pesin, Y., Weiss, H.: A multifractal analysis of equilibrium measures for conformal expanding maps and Moran-like geometric constructions. J. Stat. Phys. 86(1–2), 233–275 (1997)

    ADS  MathSciNet  Article  Google Scholar 

  39. 39.

    Rényi, A.: On the dimension and entropy of probability distributions. Acta Math. Hung. 10(1–2), 193–215 (1959)

    MathSciNet  Article  Google Scholar 

  40. 40.

    Rényi, A. et al.: On measures of entropy and information. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1: Contributions to the Theory of Statistics. The Regents of the University of California, Oakland (1961)

  41. 41.

    Rodrigues, D., Alvarez-Castro, M.C., Messori, G., Yiou, P., Robin, Y., Faranda, D.: Dynamical properties of the north atlantic atmospheric circulation in the past 150 years in CMIP5 models and the 20CRv2c reanalysis. J. Clim. 31(15), 6097–6111 (2018)

    ADS  Article  Google Scholar 

  42. 42.

    Samuel, A.L.: Some studies in machine learning using the game of checkers. II—recent progress. In: Computer Games I, pp. 366–400. Springer, New York (1988)

  43. 43.

    Scher, S., Messori, G.: Predicting weather forecast uncertainty with machine learning. Q. J. R. Meteorol. Soc. 144(717), 2830–2841 (2018)

    ADS  Article  Google Scholar 

  44. 44.

    Schubert, S., Lucarini, V.: Covariant Lyapunov vectors of a quasi-geostrophic baroclinic model: analysis of instabilities and feedbacks. Q. J. R. Meteorol. Soc. 141(693), 3040–3055 (2015)

    ADS  Article  Google Scholar 

  45. 45.

    Shorack, G.R., Shorack, G.: Probability for Statisticians. Number 04. QA273, S4. Springer, New York (2000)

  46. 46.

    Vannitsem, S., Lucarini, V.: Statistical and dynamical properties of covariant Lapunov vectors in a coupled atmosphere-ocean model-multiscale effects, geometric degeneracy, and error dynamics. J. Phys. A Math. Theoret. 49(22), 224001 (2016)

    ADS  Article  Google Scholar 

  47. 47.

    Verleysen, M., François, D.: The curse of dimensionality in data mining and time series prediction. In: International Work-Conference on Artificial Neural Networks, pp. 758–770. Springer, Berlin (2005)

  48. 48.

    Weare, B.C., Nasstrom, J.S.: Examples of extended empirical orthogonal function analyses. Mon. Weather Rev. 110(6), 481–485 (1982)

    ADS  Article  Google Scholar 

  49. 49.

    Young, L.-S.: Dimension, entropy and Lyapunov exponents. Ergod. Theory Dyn. Syst. 2(1), 109–124 (1982)

    MathSciNet  Article  Google Scholar 

Download references

Acknowledgements

We thank P. Yiou, B. Dubrulle and S. Vaienti for useful discussion. D. F., M. C. A. C., G.M. acknowledge the support of the ERC Grant A2C2 n No. 338965. MCAC was further supported by the Swedish Research Council Vetenskapsrådet Grant No. C0629701 and GM was further supported by a grant from the Department of Meteorology of Stockholm University and by the Swedish Research Council Vetenskapsrådet Grant No. 2016-03724.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Flavio Maria Emanuele Pons.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by Valerio Lucarini.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pons, F.M.E., Messori, G., Alvarez-Castro, M.C. et al. Sampling Hyperspheres via Extreme Value Theory: Implications for Measuring Attractor Dimensions. J Stat Phys (2020). https://doi.org/10.1007/s10955-020-02573-5

Download citation

Keywords

  • Attractor dimension
  • Hausdorff dimension
  • Curse of dimensionality
  • Dinamical systems
  • Climate dynamics