Divergence Functions and Geometric Structures They Induce on a Manifold

  • Jun ZhangEmail author
Part of the Signals and Communication Technology book series (SCT)


Divergence functions play a central role in information geometry. Given a manifold \(\mathfrak {M}\), a divergence function\(\mathcal {D}\) is a smooth, nonnegative function on the product manifold \(\mathfrak {M}\times \mathfrak {M}\) that achieves its global minimum of zero (with semi-positive definite Hessian) at those points that form its diagonal submanifold \(\varDelta _{\mathfrak {M}} \subset \mathfrak {M}\times \mathfrak {M}\). In this chapter, we review how such divergence functions induce (i) a statistical structure (i.e., a Riemannian metric with a pair of conjugate affine connections) on \(\mathfrak {M}\); (ii) a symplectic structure on \(\mathfrak {M}\times \mathfrak {M}\) if they are “proper”; (iii) a Kähler structure on \(\mathfrak {M}\times \mathfrak {M}\) if they further satisfy a certain condition. It is then shown that the class of \(\mathcal {D}_\varPhi \)-divergence functions [23], as induced by a strictly convex function\(\varPhi \) on \(\mathfrak {M}\), satisfies all these requirements and hence makes \(\mathfrak {M}\times \mathfrak {M}\) a Kähler manifold (with Kähler potential given by \(\varPhi \)). This provides a larger context for the \(\alpha \)-Hessian structure induced by the \(\mathcal {D}_\varPhi \)-divergence on \(\mathfrak {M}\), which is shown to be equiaffine admitting \(\alpha \)-parallel volume forms and biorthogonal coordinates generated by \(\varPhi \) and its convex conjugate \(\varPhi ^{*}\). As the \(\alpha \)-Hessian structure is dually flat for \(\alpha = \pm 1\), the \(\mathcal {D}_\varPhi \)-divergence provides richer geometric structures (compared to Bregman divergence) to the manifold \(\mathfrak {M}\) on which it is defined.


  1. 1.
    Amari, S.: Differential Geometric Methods in Statistics. Lecture Notes in Statistics, vol. 28. Springer, New York (1985) (Reprinted in 1990)Google Scholar
  2. 2.
    Amari, S., Nagaoka, H.: Method of Information Geometry. AMS Monograph. Oxford University Press, Oxford (2000)Google Scholar
  3. 3.
    Barndorff-Nielsen, O.E., Jupp, P.E.: Yorks and symplectic structures. J. Stat. Plan. Inference 63, 133–146 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Bregman, L.M.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Phys. 7, 200–217 (1967)CrossRefGoogle Scholar
  5. 5.
    Calin, O., Matsuzoe, H., Zhang. J.: Generalizations of conjugate connections. In: Sekigawa, K., Gerdjikov, V., Dimiev, S. (eds.) Trends in Differential Geometry, Complex Analysis and Mathematical Physics: Proceedings of the 9th International Workshop on Complex Structures and Vector Fields, pp. 24–34. World Scientific Publishing, Singapore (2009)Google Scholar
  6. 6.
    Csiszár, I.: On topical properties of f-divergence. Studia Mathematicarum Hungarica 2, 329–339 (1967)zbMATHGoogle Scholar
  7. 7.
    Dombrowski, P.: On the geometry of the tangent bundle. Journal fr der reine und angewandte Mathematik 210, 73–88 (1962)zbMATHMathSciNetGoogle Scholar
  8. 8.
    Eguchi, S.: Second order efficiency of minimum contrast estimators in a curved exponential family. Ann. Stat. 11, 793–803 (1983)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Eguchi, S.: Geometry of minimum contrast. Hiroshima Math. J. 22, 631–647 (1992)zbMATHMathSciNetGoogle Scholar
  10. 10.
    Lauritzen, S.: Statistical manifolds. In: Amari, S., Barndorff-Nielsen, O., Kass, R., Lauritzen, S., Rao, C.R. (eds.) Differential Geometry in Statistical Inference. IMS Lecture Notes, vol. 10, pp. 163–216. Institute of Mathematical Statistics, Hayward (1987)CrossRefGoogle Scholar
  11. 11.
    Matsuzoe, H.: On realization of conformally-projectively flat statistical manifolds and the divergences. Hokkaido Math. J. 27, 409–421 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    Matsuzoe, H., Inoguchi, J.: Statistical structures on tangent bundles. Appl. Sci. 5, 55–65 (2003)zbMATHMathSciNetGoogle Scholar
  13. 13.
    Matsuzoe, H., Takeuchi, J., Amari, S.: Equiaffine structures on statistical manifolds and Bayesian statistics. Differ. Geom. Appl. 24, 567–578 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Matsuzoe, H.: Computational geometry from the viewpoint of affine differential geometry. In: Nielsen, F. (ed.) Emerging Trends in Visual Computing, pp. 103–123. Springer, Berlin, Heidelberg (2009)CrossRefGoogle Scholar
  15. 15.
    Matsuzoe, M.: Statistical manifolds and affine differential geometry. Adv. Stud. Pure Math. 57, 303–321 (2010)MathSciNetGoogle Scholar
  16. 16.
    Nomizu, K., Sasaki, T.: Affine Differential Geometry—Geometry of Affine Immersions. Cambridge University Press, Cambridge (1994)Google Scholar
  17. 17.
    Ohara, A., Matsuzoe, H., Amari, S.: Conformal geometry of escort probability and its applications. Mod. Phys. Lett. B 26, 1250063 (2012)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Shima, H.: Hessian Geometry. Shokabo, Tokyo (2001) (in Japanese)Google Scholar
  19. 19.
    Shima, H., Yagi, K.: Geometry of Hessian manifolds. Differ. Geom. Appl. 7, 277–290 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  20. 20.
    Simon, U.: Affine differential geometry. In: Dillen, F., Verstraelen, L. (eds.) Handbook of Differential Geometry, vol. I, pp. 905–961. Elsevier Science, Amsterdam (2000)Google Scholar
  21. 21.
    Uohashi, K.: On \(\alpha \)-conformal equivalence of statistical manifolds. J. Geom. 75, 179–184 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  22. 22.
    Yano, K., Ishihara, S.: Tangent and Cotangent Bundles: Differential Geometry, vol. 16. Dekker, New York (1973)zbMATHGoogle Scholar
  23. 23.
    Zhang, J.: Divergence function, duality, and convex analysis. Neural Comput. 16, 159–195 (2004)CrossRefzbMATHGoogle Scholar
  24. 24.
    Zhang, J.: Referential duality and representational duality on statistical manifolds. Proceedings of the 2nd International Symposium on Information Geometry and Its Applications, Tokyo, pp. 58–67 (2006)Google Scholar
  25. 25.
    Zhang, J.: A note on curvature of \(\alpha \)-connections on a statistical manifold. Ann. Inst. Stat. Math. 59, 161–170 (2007)CrossRefzbMATHGoogle Scholar
  26. 26.
    Zhang, J., Matsuzoe, H.: Dualistic differential geometry associated with a convex function. In: Gao, D.Y., Sherali, H.D. (eds.) Advances in Applied Mathematics and Global Optimization (Dedicated to Gilbert Strang on the occasion of his 70th birthday), Advances in Mechanics and Mathematics, vol. III, Chap. 13, pp. 439–466. Springer, New York (2009)Google Scholar
  27. 27.
    Zhang, J.: Nonparametric information geometry: From divergence function to referential-representational biduality on statistical manifolds. Entropy 15, 5384–5418 (2013)Google Scholar
  28. 28.
    Zhang, J., Li, F.: Symplectic and Kähler structures on statistical manifolds induced from divergence functions. In: Nielson, F., Barbaresco, F. (eds.) Proceedings of the 1st International Conference on Geometric Science of Information (GSI2013), pp. 595–603 (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Department of Psychology and Department of MathematicsUniversity of MichiganAnn ArborUSA

Personalised recommendations