Skip to main content
SpringerLink
Log in

Nonparametric Information Geometry

  • Conference paper
Geometric Science of Information (GSI 2013)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 8085))

Included in the following conference series:

Abstract

The differential-geometric structure of the set of positive densities on a given measure space has raised the interest of many mathematicians after the discovery by C.R. Rao of the geometric meaning of the Fisher information. Most of the research is focused on parametric statistical models. In series of papers by author and coworkers a particular version of the nonparametric case has been discussed. It consists of a minimalistic structure modeled according the theory of exponential families: given a reference density other densities are represented by the centered log likelihood which is an element of an Orlicz space. This mappings give a system of charts of a Banach manifold. It has been observed that, while the construction is natural, the practical applicability is limited by the technical difficulty to deal with such a class of Banach spaces. It has been suggested recently to replace the exponential function with other functions with similar behavior but polynomial growth at infinity in order to obtain more tractable Banach spaces, e.g. Hilbert spaces. We give first a review of our theory with special emphasis on the specific issues of the infinite dimensional setting. In a second part we discuss two specific topics, differential equations and the metric connection. The position of this line of research with respect to other approaches is briefly discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amari, S.I.: Differential geometry of curved exponential families—curvatures and information loss. Ann. Statist. 10(2), 357–385 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  2. Amari, S.: Differential-geometrical methods in statistics. Lecture Notes in Statistics, vol. 28. Springer, New York (1985)

    Book  MATH  Google Scholar 

  3. Amari, S.: Differential geometrical theory of statistics. In: Differential Geometry in Statistical Inference. Institute of Mathematical Statistics Lecture Notes—Monograph Series, vol. 10, pp. 19–94. Institute of Mathematical Statistics, Hayward (1987)

    Chapter  Google Scholar 

  4. Amari, S., Nagaoka, H.: Methods of information geometry. American Mathematical Society, Providence (2000); translated from the 1993 Japanese original by Daishi Harada

    MATH  Google Scholar 

  5. Bourbaki, N.: Variétés differentielles et analytiques. Fascicule de résultats / Paragraphes 1 à 7. Number XXXIII in Éléments de mathématiques. Hermann, Paris (1971)

    Google Scholar 

  6. Lang, S.: Differential and Riemannian manifolds, 3rd edn. Graduate Texts in Mathematics, vol. 160. Springer, New York (1995)

    Book  MATH  Google Scholar 

  7. Pistone, G., Sempi, C.: An infinite-dimensional geometric structure on the space of all the probability measures equivalent to a given one. Ann. Statist. 23(5), 1543–1561 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  8. Pistone, G., Rogantin, M.: The exponential statistical manifold: mean parameters, orthogonality and space transformations. Bernoulli 5(4), 721–760 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  9. Gibilisco, P., Pistone, G.: Connections on non-parametric statistical manifolds by Orlicz space geometry. IDAQP 1(2), 325–347 (1998)

    MathSciNet  MATH  Google Scholar 

  10. Cena, A.: Geometric structures on the non-parametric statistical manifold. PhD thesis, Dottorato in Matematica, Università di Milano (2002)

    Google Scholar 

  11. Cena, A., Pistone, G.: Exponential statistical manifold. Ann. Inst. Statist. Math. 59(1), 27–56 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  12. Malagò, L., Matteucci, M., Dal Seno, B.: An information geometry perspective on estimation of distribution algorithms: boundary analysis. In: GECCO 2008: Proceedings of the 2008 GECCO Conference Companion on Genetic and Evolutionary Computation, pp. 2081–2088. ACM, New York (2008)

    Chapter  Google Scholar 

  13. Imparato, D.: Exponential models and Fisher information. Geometry and applications. PhD thesis, DIMAT Politecnico di Torino (2008)

    Google Scholar 

  14. Brigo, D., Pistone, G.: Projecting the Fokker-Planck equation onto a finite dimensional exponential family. arXiv:0901.1308 (2009)

    Google Scholar 

  15. Malagò, L., Pistone, G.: A note on the border of an exponential family. arXiv:1012.0637v1 (2010)

    Google Scholar 

  16. Pistone, G.: κ-exponential models from the geometrical viewpoint. The European Physical Journal B Condensed Matter Physics 71(1), 29–37 (2009)

    Article  Google Scholar 

  17. Pistone, G.: Algebraic varieties vs. differentiable manifolds in statistical models. In: Gibilisco, P., Riccomagno, E., Rogantin, M., Wynn, H.P. (eds.) Algebraic and Geometric Methods in Statistics, pp. 339–363. Cambridge University Press (2009)

    Google Scholar 

  18. Imparato, D., Trivellato, B.: Geometry of extended exponential models. In: Algebraic and geometric methods in statistics, pp. 307–326. Cambridge Univ. Press, Cambridge (2010)

    Google Scholar 

  19. Malagò, L., Matteucci, M., Pistone, G.: Towards the geometry of estimation of distribution algorithms based on the exponential family. In: Proceedings of the 11th Workshop on Foundations of Genetic Algorithms, FOGA 2011, pp. 230–242. ACM, New York (2011)

    Google Scholar 

  20. Malagò, L., Matteucci, M., Pistone, G.: Stochastic natural gradient descent by estimation of empirical covariances. In: 2011 IEEE Congress on Evolutionary Computation (CEC), pp. 949–956 (2011)

    Google Scholar 

  21. Malagò, L.: On the geometry of optimization based on the exponential family relaxation. PhD thesis, Politecnico di Milano (2012)

    Google Scholar 

  22. Malagò, L., Matteucci, M., Pistone, G.: Natural gradient, fitness modelling and model selection: A unifying perspective. Paper #1747 IEEE Congress on Evolutionary Computation IEEE CEC 2013, Cancń México, June 20–23 (2013)

    Google Scholar 

  23. Gibilisco, P., Isola, T.: Connections on statistical manifolds of density operators by geometry of noncommutative \(L\sp p\)-spaces. Infin. Dimens. Anal. Quantum Probab. Relat. Top. 2(1), 169–178 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  24. Jenčová, A.: A construction of a nonparametric quantum information manifold. J. Funct. Anal. 239(1), 1–20 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  25. Gibilisco, P., Riccomagno, E., Rogantin, M.P., Wynn, H.P. (eds.): Algebraic and geometric methods in statistics. Cambridge University Press, Cambridge (2010)

    MATH  Google Scholar 

  26. Efron, B.: Defining the curvature of a statistical problem (with applications to second order efficiency). Ann. Statist. 3(6), 1189–1242 (1975); With a discussion by Rao, C. R., Pierce, D. A., Cox, D. R., Lindley, D. V., LeCam, L., Ghosh, J. K., Pfanzagl, J., Keiding, N., Dawid, A. P., Reeds, J., with a reply by the author

    Article  MathSciNet  MATH  Google Scholar 

  27. Barndorff-Nielsen, O.E.: Information and Exponential Families in Statistical Theory. John Wiley & Sons, New York (1978)

    MATH  Google Scholar 

  28. Brown, L.D.: Fundamentals of statistical exponential families with applications in statistical decision theory. IMS Lecture Notes. Monograph Series, vol. 9. Institute of Mathematical Statistics, Hayward (1986)

    MATH  Google Scholar 

  29. Letac, G.: Lectures on natural exponential families and their variance functions. Monografías de Matemática, Mathematical Monographs, vol. 50. Instituto de Matemática Pura e Aplicada (IMPA), Rio de Janeiro (1992)

    MATH  Google Scholar 

  30. Dawid, A.P.: Discussion of a paper by Bradley Efron. Ann. Statist. 3(6), 1231–1234 (1975)

    MathSciNet  Google Scholar 

  31. Dawid, A.P.: Further comments on: “Some comments on a paper by Bradley Efron”. Ann. Statist. 3, 1189–1242 (1975); Ann. Statist. 5(6), 1249 (1977)

    Article  MathSciNet  Google Scholar 

  32. Gzyl, H., Recht, L.: A geometry on the space of probabilities. I. The finite dimensional case. Rev. Mat. Iberoam 22(2), 545–558 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  33. Gzyl, H., Recht, L.: A geometry on the space of probabilities. II. Projective spaces and exponential families. Rev. Mat. Iberoam 22(3), 833–849 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  34. Krasnosel’skii, M.A., Rutickii, Y.B.: Convex Functions and Orlicz Spaces. Noordhoff, Groningen (1961); Russian original: Fizmatgiz, Moskva (1958)

    Google Scholar 

  35. Musielak, J.: Orlicz spaces and modular spaces. Lecture Notes in Mathematics, vol. 1034. Springer, Berlin (1983)

    MATH  Google Scholar 

  36. Rao, M.M., Ren, Z.D.: Applications of Orlicz spaces. Monographs and Textbooks in Pure and Applied Mathematics, vol. 250. Marcel Dekker Inc., New York (2002)

    MATH  Google Scholar 

  37. Adams, R.A., Fournier, J.J.F.: Sobolev spaces, 2nd edn. Pure and Applied Mathematics, vol. 140. Elsevier/Academic Press, Amsterdam (2003)

    MATH  Google Scholar 

  38. Gallavotti, G.: Statistical mechanics: A short treatise. Texts and Monographs in Physics. Springer, Berlin (1999)

    Google Scholar 

  39. Boros, E., Hammer, P.L.: Pseudo-Boolean optimization. Discrete Appl. Math. 123(1-3), 155–225 (2002); Workshop on Discrete Optimization, DO 1999 (Piscataway, NJ)

    Article  MathSciNet  MATH  Google Scholar 

  40. Grasselli, M.R.: Dual connections in nonparametric classical information geometry. Technical Report math-ph/0104031 v1, arXiv (2001)

    Google Scholar 

  41. Appell, J., Zabrejko, P.P.: Nonlinear superposition operators. Cambridge Tracts in Mathematics, vol. 95. Cambridge University Press, Cambridge (1990)

    Book  MATH  Google Scholar 

  42. Upmeier, H.: Symmetric Banach manifolds and Jordan \(C\sp \ast\)-algebras. North-Holland Mathematics Studies, vol. 104. North-Holland Publishing Co., Amsterdam (1985); Notas de Matemática (Mathematical Notes), 96

    Google Scholar 

  43. Ambrosetti, A., Prodi, G.: A primer of nonlinear analysis. Cambridge Studies in Advanced Mathematics, vol. 34. Cambridge University Press, Cambridge (1993)

    Google Scholar 

  44. Barndorff-Nielsen, O.E., Jupp, P.E.: Statistics, yokes and symplectic geometry. Ann. Fac. Sci. Toulouse Math. 6(3), 389–427 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  45. Shima, H.: The geometry of Hessian structures. World Scientific Publishing Co. Pte. Ltd., Hackensack (2007)

    Book  MATH  Google Scholar 

  46. Ambrosio, L., Gigli, N., Savaré, G.: Gradient flows in metric spaces and in the space of probability measures, 2nd edn. Lectures in Mathematics ETH Zürich. Birkhäuser, Basel (2008)

    MATH  Google Scholar 

  47. Csiszár, I., Matúš, F.: Information projections revisited. IEEE Trans. Inform. Theory 49(6), 1474–1490 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  48. Santacroce, M., Siri, P., Trivellato, B.: A dynamic approach to exponential statistical manifolds (in progress, 2013)

    Google Scholar 

  49. Arnold, L., Auger, A., Hansen, N., Ollivier, Y.: Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles. arXiv:1106.3708 (2011)

    Google Scholar 

  50. Otto, F.: The geometry of dissipative evolution equations: the porous medium equation. Comm. Partial Differential Equations 26(1-2), 101–174 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  51. Parry, M., Dawid, A.P., Lauritzen, S.: Proper local scoring rules. Ann. Statist. 40(1), 561–592 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  52. Majewski, W.A., Labuschagne, L.E.: On applications of orlicz spaces to statistical physics. arXiv:1302.3460 (2013)

    Google Scholar 

  53. Villani, C.: A review of mathematical topics in collisional kinetic theory. In: Handbook of Mathematical Fluid Dynamics, vol. I, pp. 71–305. North-Holland, Amsterdam (2002)

    Chapter  Google Scholar 

  54. Grasselli, M.R.: Dual connections in nonparametric classical information geometry. Ann. Inst. Statist. Math. 62(5), 873–896 (2010)

    Article  MathSciNet  Google Scholar 

  55. Malliavin, P.: Integration and probability. Graduate Texts in Mathematics, vol. 157. Springer, New York (1995); With the collaboration of H. Airault, L. Kay, G. Letac, Edited and translated from the French by Kay, With a foreword by Mark Pinsky

    Google Scholar 

  56. Brigo, D., Hanzon, B., Le Gland, F.: Approximate nonlinear filtering by projection on exponential manifolds of densities. Bernoulli 5(3), 495–534 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  57. Naudts, J.: Generalised Thermostatistics. Springer (2011)

    Google Scholar 

  58. Tsallis, C.: Possible generalization of Boltzmann-Gibbs statistics. J. Statist. Phys. 52(1-2), 479–487 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  59. Kaniadakis, G.: Statistical mechanics in the context of special relativity. Physical Review E 66, 056125, 1–17 (2002)

    Google Scholar 

  60. Kaniadakis, G.: Statistical mechanics in the context of special relativity. ii. Phys. Rev. E 72(3), 036108 (2005)

    Google Scholar 

  61. Newton, N.J.: An infinite-dimensional statistical manifold modelled on Hilbert space. J. Funct. Anal. 263(6), 1661–1681 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  62. Vigelis, R.F., Cavalcante, C.C.: On the φ-family of probability distributions. Journal of Theoretical Probability (2011) (online first)

    Google Scholar 

  63. Burdet, G., Combe, P., Nencka, H.: On real Hilbertian info-manifolds. In: Disordered and Complex Systems (London, 2000). AIP Conf. Proc. Amer. Inst. Phys., Melville, NY, vol. 553, pp. 153–158 (2001)

    Google Scholar 

  64. Eguchi, S.: Tubular modelling approach to statistical method for observational studies. In: 2nd International Symposium on Information Geometry and its Applications, Tokyo, December 12-16 (2005)

    Google Scholar 

  65. Zhang, J., Hästö, P.: Statistical manifold as an affine space: a functional equation approach. Journal of Mathematical Psychology 50(1), 60–65 (2006)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Collegio Carlo Alberto, Via Real Collegio 30, 10024, Moncalieri, Italy

    Giovanni Pistone

Authors
  1. Giovanni Pistone
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Sony Computer Science Laboratories Inc, Tokyo, Japan

    Frank Nielsen

  2. Thales Land Air Systems, 91470, Limours, France

    Frédéric Barbaresco

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Pistone, G. (2013). Nonparametric Information Geometry. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2013. Lecture Notes in Computer Science, vol 8085. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40020-9_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-40020-9_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-40019-3

  • Online ISBN: 978-3-642-40020-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation