# Divergence Functions and Geometric Structures They Induce on a Manifold

• Jun Zhang
Chapter
Part of the Signals and Communication Technology book series (SCT)

## Abstract

Divergence functions play a central role in information geometry. Given a manifold $$\mathfrak {M}$$, a divergence function$$\mathcal {D}$$ is a smooth, nonnegative function on the product manifold $$\mathfrak {M}\times \mathfrak {M}$$ that achieves its global minimum of zero (with semi-positive definite Hessian) at those points that form its diagonal submanifold $$\varDelta _{\mathfrak {M}} \subset \mathfrak {M}\times \mathfrak {M}$$. In this chapter, we review how such divergence functions induce (i) a statistical structure (i.e., a Riemannian metric with a pair of conjugate affine connections) on $$\mathfrak {M}$$; (ii) a symplectic structure on $$\mathfrak {M}\times \mathfrak {M}$$ if they are “proper”; (iii) a Kähler structure on $$\mathfrak {M}\times \mathfrak {M}$$ if they further satisfy a certain condition. It is then shown that the class of $$\mathcal {D}_\varPhi$$-divergence functions [23], as induced by a strictly convex function$$\varPhi$$ on $$\mathfrak {M}$$, satisfies all these requirements and hence makes $$\mathfrak {M}\times \mathfrak {M}$$ a Kähler manifold (with Kähler potential given by $$\varPhi$$). This provides a larger context for the $$\alpha$$-Hessian structure induced by the $$\mathcal {D}_\varPhi$$-divergence on $$\mathfrak {M}$$, which is shown to be equiaffine admitting $$\alpha$$-parallel volume forms and biorthogonal coordinates generated by $$\varPhi$$ and its convex conjugate $$\varPhi ^{*}$$. As the $$\alpha$$-Hessian structure is dually flat for $$\alpha = \pm 1$$, the $$\mathcal {D}_\varPhi$$-divergence provides richer geometric structures (compared to Bregman divergence) to the manifold $$\mathfrak {M}$$ on which it is defined.

## References

1. 1.
Amari, S.: Differential Geometric Methods in Statistics. Lecture Notes in Statistics, vol. 28. Springer, New York (1985) (Reprinted in 1990)Google Scholar
2. 2.
Amari, S., Nagaoka, H.: Method of Information Geometry. AMS Monograph. Oxford University Press, Oxford (2000)Google Scholar
3. 3.
Barndorff-Nielsen, O.E., Jupp, P.E.: Yorks and symplectic structures. J. Stat. Plan. Inference 63, 133–146 (1997)
4. 4.
Bregman, L.M.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Phys. 7, 200–217 (1967)
5. 5.
Calin, O., Matsuzoe, H., Zhang. J.: Generalizations of conjugate connections. In: Sekigawa, K., Gerdjikov, V., Dimiev, S. (eds.) Trends in Differential Geometry, Complex Analysis and Mathematical Physics: Proceedings of the 9th International Workshop on Complex Structures and Vector Fields, pp. 24–34. World Scientific Publishing, Singapore (2009)Google Scholar
6. 6.
Csiszár, I.: On topical properties of f-divergence. Studia Mathematicarum Hungarica 2, 329–339 (1967)
7. 7.
Dombrowski, P.: On the geometry of the tangent bundle. Journal fr der reine und angewandte Mathematik 210, 73–88 (1962)
8. 8.
Eguchi, S.: Second order efficiency of minimum contrast estimators in a curved exponential family. Ann. Stat. 11, 793–803 (1983)
9. 9.
Eguchi, S.: Geometry of minimum contrast. Hiroshima Math. J. 22, 631–647 (1992)
10. 10.
Lauritzen, S.: Statistical manifolds. In: Amari, S., Barndorff-Nielsen, O., Kass, R., Lauritzen, S., Rao, C.R. (eds.) Differential Geometry in Statistical Inference. IMS Lecture Notes, vol. 10, pp. 163–216. Institute of Mathematical Statistics, Hayward (1987)
11. 11.
Matsuzoe, H.: On realization of conformally-projectively flat statistical manifolds and the divergences. Hokkaido Math. J. 27, 409–421 (1998)
12. 12.
Matsuzoe, H., Inoguchi, J.: Statistical structures on tangent bundles. Appl. Sci. 5, 55–65 (2003)
13. 13.
Matsuzoe, H., Takeuchi, J., Amari, S.: Equiaffine structures on statistical manifolds and Bayesian statistics. Differ. Geom. Appl. 24, 567–578 (2006)
14. 14.
Matsuzoe, H.: Computational geometry from the viewpoint of affine differential geometry. In: Nielsen, F. (ed.) Emerging Trends in Visual Computing, pp. 103–123. Springer, Berlin, Heidelberg (2009)
15. 15.
Matsuzoe, M.: Statistical manifolds and affine differential geometry. Adv. Stud. Pure Math. 57, 303–321 (2010)
16. 16.
Nomizu, K., Sasaki, T.: Affine Differential Geometry—Geometry of Affine Immersions. Cambridge University Press, Cambridge (1994)Google Scholar
17. 17.
Ohara, A., Matsuzoe, H., Amari, S.: Conformal geometry of escort probability and its applications. Mod. Phys. Lett. B 26, 1250063 (2012)
18. 18.
Shima, H.: Hessian Geometry. Shokabo, Tokyo (2001) (in Japanese)Google Scholar
19. 19.
Shima, H., Yagi, K.: Geometry of Hessian manifolds. Differ. Geom. Appl. 7, 277–290 (1997)
20. 20.
Simon, U.: Affine differential geometry. In: Dillen, F., Verstraelen, L. (eds.) Handbook of Differential Geometry, vol. I, pp. 905–961. Elsevier Science, Amsterdam (2000)Google Scholar
21. 21.
Uohashi, K.: On $$\alpha$$-conformal equivalence of statistical manifolds. J. Geom. 75, 179–184 (2002)
22. 22.
Yano, K., Ishihara, S.: Tangent and Cotangent Bundles: Differential Geometry, vol. 16. Dekker, New York (1973)
23. 23.
Zhang, J.: Divergence function, duality, and convex analysis. Neural Comput. 16, 159–195 (2004)
24. 24.
Zhang, J.: Referential duality and representational duality on statistical manifolds. Proceedings of the 2nd International Symposium on Information Geometry and Its Applications, Tokyo, pp. 58–67 (2006)Google Scholar
25. 25.
Zhang, J.: A note on curvature of $$\alpha$$-connections on a statistical manifold. Ann. Inst. Stat. Math. 59, 161–170 (2007)
26. 26.
Zhang, J., Matsuzoe, H.: Dualistic differential geometry associated with a convex function. In: Gao, D.Y., Sherali, H.D. (eds.) Advances in Applied Mathematics and Global Optimization (Dedicated to Gilbert Strang on the occasion of his 70th birthday), Advances in Mechanics and Mathematics, vol. III, Chap. 13, pp. 439–466. Springer, New York (2009)Google Scholar
27. 27.
Zhang, J.: Nonparametric information geometry: From divergence function to referential-representational biduality on statistical manifolds. Entropy 15, 5384–5418 (2013)Google Scholar
28. 28.
Zhang, J., Li, F.: Symplectic and Kähler structures on statistical manifolds induced from divergence functions. In: Nielson, F., Barbaresco, F. (eds.) Proceedings of the 1st International Conference on Geometric Science of Information (GSI2013), pp. 595–603 (2013)Google Scholar