Skip to main content
SpringerLink
Log in

Introduction

  • Chapter
  • First Online:
Book cover Information Geometry

  • 5458 Accesses

Abstract

We describe the setting of parametric statistics, that is, families of probability measures on a sample space, its task, that is, to identify a particular probability measure that best fits that sampling distribution, and the surprisingly rich and useful geometric structure underlying this. The latter is the topic of this book. A basic geometry quantity, the Fisher metric, a 2-tensor, measures how sensitively the distributions depend on the samples, and this leads to the Cramér–Rao inequality. It is naturally invariant under sufficient statistics, and so is a natural 3-tensor, the Amari–Chentsov tensor. In this introduction, these basic concepts and their functorial properties are described in an informal manner, and an overview of the main ideas and results of the book is given. We also provide a short historical account of the development of the theory.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In [11, p. 67] Amari uncovered a less known work of Harold Hotelling on the Fisher information metric submitted to the American Mathematical Society Meeting in 1929. We refer the reader to [11] for details.

References

  1. Amari, S.: Theory of information spaces. A geometrical foundation of statistics. POST RAAG Report 106 (1980)

    Google Scholar 

  2. Amari, S.: Differential geometry of curved exponential families curvature and information loss. Ann. Stat. 10, 357–385 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  3. Amari, S.: Differential-Geometric Methods in Statistics. Lecture Notes in Statistics, vol. 28. Springer, Heidelberg (1985)

    Book  MATH  Google Scholar 

  4. Amari, S.: Differential geometrical theory of statistics. In: Differential Geometry in Statistical Inference, Institute of Mathematical Statistics, California. Lecture Notes–Monograph Series, vol. 10 (1987)

    Google Scholar 

  5. Amari, S.: Information Geometry and Its Applications. Applied Mathematical Sciences, vol. 194. Springer, Berlin (2016)

    MATH  Google Scholar 

  6. Amari, S., Armstrong, J.: Curvature of Hessian manifolds. Differ. Geom. Appl. 33, 1–12 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  7. Amari, S., Kurata, K., Nagaoka, H.: Information geometry of Boltzmann machines. IEEE Trans. Neural Netw. 3, 260–271 (1992)

    Article  Google Scholar 

  8. Amari, S., Nagaoka, H.: Methods of Information Geometry. Translations of Mathematical Monographs, vol. 191. Am. Math. Soc./Oxford University Press, Providence/London (2000)

    MATH  Google Scholar 

  9. Ay, N., Jost, J., Lê, H.V., Schwachhöfer, L.: Information geometry and sufficient statistics. Probab. Theory Relat. Fields 162, 327–364 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  10. Ay, N., Tuschmann, W.: Dually flat manifolds and global information geometry. Open Syst. Inf. Dyn. 9, 195–200 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  11. Balian, R., Alhassid, Y., Reinhardt, H.: Dissipation in many-body systems: a geometric approach based on information theory. Phys. Rep. 131, 1–146 (1986)

    Article  MathSciNet  Google Scholar 

  12. Barndorff-Nielsen, O.: Information and Exponential Families in Statistical Theory. Wiley Series in Probability and Mathematical Statistics. John Wiley & Sons, Ltd., Chichester (2014)

    Book  MATH  Google Scholar 

  13. Barndorff-Nielsen, O.E., Jupp, P.E.: Approximating exponential models. Ann. Inst. Stat. Math. 41, 247–267 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  14. Bauer, M., Bruveris, M., Michor, P.: Uniqueness of the Fisher–Rao metric on the space of smooth densities. Bull. Lond. Math. Soc. (2016). arXiv:1411.5577. doi:10.1112/blms/bdw020

    MathSciNet  MATH  Google Scholar 

  15. Bengtsson, I., Życzkowski, K.: Geometry of Quantum States. Cambridge University Press, Cambridge (2006)

    Book  MATH  Google Scholar 

  16. Blackwell, D.: Equivalent comparisons of experiments. Ann. Math. Stat. 24, 265–272 (1953)

    Article  MathSciNet  MATH  Google Scholar 

  17. Calin, O., Udrişte, C.: Geometric Modeling in Probability and Statistics. Springer, Berlin (2014)

    Book  MATH  Google Scholar 

  18. Campbell, L.L.: An extended Chentsov characterization of a Riemannian metric. Proc. Am. Math. Soc. 98, 135–141 (1986)

    MATH  Google Scholar 

  19. Cheng, S.Y., Yau, S.T.: The real Monge–Ampère equation and affine flat structures. In: Chern, S.S., Wu, W.T. (eds.) Differential Geometry and Differential Equations. Proc. Beijing Symp., vol. 1980, pp. 339–370 (1982)

    Google Scholar 

  20. Chentsov, N.: Geometry of the “manifold” of a probability distribution. Dokl. Akad. Nauk SSSR 158, 543–546 (1964) (Russian)

    MathSciNet  Google Scholar 

  21. Chentsov, N.: Category of mathematical statistics. Dokl. Akad. Nauk SSSR 164, 511–514 (1965)

    MathSciNet  Google Scholar 

  22. Chentsov, N.: A systematic theory of exponential families of probability distributions. Teor. Veroâtn. Primen. 11, 483–494 (1966) (Russian)

    MathSciNet  MATH  Google Scholar 

  23. Chentsov, N.: A nonsymmetric distance between probability distributions, entropy and the theorem of Pythagoras. Mat. Zametki 4, 323–332 (1968) (Russian)

    MathSciNet  Google Scholar 

  24. Chentsov, N.: Statistical Decision Rules and Optimal Inference, vol. 53. Nauka, Moscow (1972) (in Russian); English translation in: Math. Monograph., vol. 53. Am. Math. Soc., Providence (1982)

    Google Scholar 

  25. Csiszár, I.: Information-type measures of difference of probability distributeions and indirect observations. Studia Sci. Math. Hung. 2, 299–318 (1967)

    MATH  Google Scholar 

  26. Csiszár, I.: On topological properties of \(f\)-divergences. Studia Sci. Math. Hung. 2, 329–339 (1967)

    MATH  Google Scholar 

  27. Csiszár, I.: Information measures: a critical survey. In: Proceedings of 7th Conference on Information Theory, Prague, Czech Republic, pp. 83–86 (1974)

    Google Scholar 

  28. Csiszár, I.: I-divergence geometry of probability distributions and minimization problems. Ann. Probab. 3, 146–158 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  29. Csiszár, I., Matúš, F.: Information projections revisited. IEEE Trans. Inf. Theory 49, 1474–1490 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  30. Diaconis, P., Sturmfels, B.: Algebraic algorithms for sampling from conditional distributions. Ann. Stat. 26(1), 363–397 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  31. Drton, M., Sturmfels, B., Sullivant, S.: Lectures on Algebraic Statistics (Oberwolfach Seminars). Birkhäuser, Basel (2009)

    Book  MATH  Google Scholar 

  32. Efron, B.: Defining the curvature of a statistical problem (with applications to second order efficiency). Ann. Stat. 3, 1189–1242 (1975), with a discussion by C.R. Rao, Don A. Pierce, D.R. Cox, D.V. Lindley, Lucien LeCam, J.K. Ghosh, J. Pfanzagl, Niels Keiding, A.P. Dawid, Jim Reeds and with a reply by the author

    Article  MathSciNet  MATH  Google Scholar 

  33. Efron, B.: The geometry of exponential families. Ann. Stat. 6, 362–376 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  34. Eguchi, S.: Second order efficiency of minimum contrast estimators in a curved exponential family. Ann. Stat. 11, 793–803 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  35. Eguchi, S.: A characterization of second order efficiency in a curved exponential family. Ann. Inst. Stat. Math. 36, 199–206 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  36. Eguchi, S.: Geometry of minimum contrast. Hiroshima Math. J. 22, 631–647 (1992)

    MathSciNet  MATH  Google Scholar 

  37. Geiger, D., Meek, C., Sturmfels, B.: On the toric algebra of graphical models. Ann. Stat. 34(5), 1463–1492 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  38. Gibilisco, P., Pistone, G.: Connections on non-parametric statistical models by Orlicz space geometry. Infin. Dimens. Anal. Quantum Probab. Relat. Top. 1(2), 325–347 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  39. Gibilisco, P., Riccomagno, E., Rogantin, M.P., Wynn, H.P. (eds.): Algebraic and Geometric Methods in Statistics. Cambridge University Press, Cambridge (2010)

    Google Scholar 

  40. Hofrichter, J., Jost, J., Tran, T.D.: Information Geometry and Population Genetics. Springer, Berlin (2017)

    Book  MATH  Google Scholar 

  41. Jeffreys, H.: An invariant form for the prior probability in estimation problems. Proc. R. Soc. Lond. Ser. A 186, 453–461 (1946)

    Article  MathSciNet  MATH  Google Scholar 

  42. Jeffreys, H.: Theory of Probability, 2nd edn. Clarendon Press, Oxford (1948)

    MATH  Google Scholar 

  43. Jost, J.: Geometry and Physics. Springer, Berlin (2009)

    Book  MATH  Google Scholar 

  44. Kass, R., Vos, P.W.: Geometrical Foundations of Asymptotic Inference. Wiley, New York (1997)

    Book  MATH  Google Scholar 

  45. Lauritzen, S.: Statistical manifolds. In: Differential Geometry in Statistical Inference, Institute of Mathematical Statistics, California. Lecture Note-Monograph Series, vol. 10 (1987)

    Google Scholar 

  46. Lauritzen, S.: Graphical Models. Oxford University Press, London (1996)

    MATH  Google Scholar 

  47. Lê, H.V.: The uniqueness of the Fisher metric as information metric. Ann. Inst. Stat. Math. 69, 879–896 (2017). doi:10.1007/s10463-016-0562-0

    Article  MathSciNet  Google Scholar 

  48. Morozova, E., Chentsov, N.: Invariant linear connections on the aggregates of probability distributions. Akad. Nauk SSSR Inst. Prikl. Mat. Preprint, no 167 (1988) (Russian)

    Google Scholar 

  49. Morozova, E., Chentsov, N.: Markov invariant geometry on manifolds of states. Itogi Nauki Tekh. Ser. Sovrem. Mat. Prilozh. Temat. Obz. 6, 69–102 (1990)

    MATH  Google Scholar 

  50. Morozova, E., Chentsov, N.: Natural geometry on families of probability laws. Itogi Nauki Tekh. Ser. Sovrem. Mat. Prilozh. Temat. Obz. 83, 133–265 (1991)

    MathSciNet  MATH  Google Scholar 

  51. Morse, N., Sacksteder, R.: Statistical isomorphism. Ann. Math. Stat. 37, 203–214 (1966)

    Article  MathSciNet  MATH  Google Scholar 

  52. Murray, M., Rice, J.: Differential Geometry and Statistics. Chapman & Hall, London (1993)

    Book  MATH  Google Scholar 

  53. Nagaoka, H., Amari, S.: Differential geometry of smooth families of probability distributions. Dept. of Math. Eng. and Instr. Phys. Univ. of Tokyo. Technical Report METR 82-7 (1982)

    Google Scholar 

  54. Pachter, L., Sturmfels, B. (eds.): Algebraic Statistics for Computational Biology. Cambridge University Press, Cambridge (2005)

    MATH  Google Scholar 

  55. Petz, D.: Quantum Information Theory and Quantum Statistics (Theoretical and Mathematical Physics). Springer, Berlin (2007)

    Google Scholar 

  56. Pistone, G., Riccomagno, E., Wynn, H.P.: Algebraic Statistics: Computational Commutative Algebra in Statistics. Chapman & Hall, London (2000)

    Book  MATH  Google Scholar 

  57. Pistone, G., Sempi, C.: An infinite-dimensional structure on the space of all the probability measures equivalent to a given one. Ann. Stat. 23(5), 1543–1561 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  58. Rao, C.R.: Information and the accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc. 37, 81–89 (1945)

    MathSciNet  MATH  Google Scholar 

  59. Rao, C.R.: Differential metrics in probability spaces. In: Differential Geometry in Statistical Inference, Institute of Mathematical Statistics, California. Lecture Notes–Monograph Series, vol. 10 (1987)

    Google Scholar 

  60. Shima, H.: The Geometry of Hessian Structures. World Scientific, Singapore (2007)

    Book  MATH  Google Scholar 

  61. Wald, A.: Statistical Decision Function. Wiley, New York (1950)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Information Theory of Cognitive Systems, MPI for Mathematics in the Sciences, Leipzig, Germany

    Nihat Ay

  2. Geometric Methods and Complex Systems, MPI for Mathematics in the Sciences, Leipzig, Germany

    Jürgen Jost

  3. Mathematical Institute of ASCR, Czech Academy of Sciences, Praha 1, Czech Republic

    Hông Vân Lê

  4. Department of Mathematics, TU Dortmund University, Dortmund, Germany

    Lorenz Schwachhöfer

  5. Santa Fe Institute, Santa Fe, NM, USA

    Nihat Ay & Jürgen Jost

Authors
  1. Nihat Ay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jürgen Jost
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hông Vân Lê
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lorenz Schwachhöfer
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Ay, N., Jost, J., Lê, H.V., Schwachhöfer, L. (2017). Introduction. In: Information Geometry. Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge / A Series of Modern Surveys in Mathematics, vol 64. Springer, Cham. https://doi.org/10.1007/978-3-319-56478-4_1

Download citation

Publish with us

Policies and ethics

Search

Navigation