Information Theory on Lie Groups

Part of the Applied and Numerical Harmonic Analysis book series (ANHA)


Classical inequalities used in information theory such as those of de Bruijn, Fisher, Cramér, Rao, and Kullback carry over in a natural way from Euclidean space to unimodular Lie groups. The extension of core information-theoretic inequalities defined in the setting of Euclidean space to this broad class of Lie groups is potentially relevant to a number of problems relating to information-gathering in mobile robotics, satellite attitude control, tomographic image reconstruction, biomolecular structure determination, and quantum information theory. In this chapter, several definitions are extended from the Euclidean setting to that of Lie groups (including entropy and the Fisher information matrix), and inequalities analogous to those in classical information theory are derived and stated in the form of more than a dozen theorems. In all such inequalities, addition of random variables is replaced with the group product, and the appropriate generalization of convolution of probability densities is employed.


Fisher Information Fisher Information Matrix Logarithmic Sobolev Inequality Leibler Divergence Classical Information Theory 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amari, S., Nagaoka, H., Methods of Information Geometry, Translations of Mathematical Monographs Vol. 191, American Mathematical Society, Providence, RI, 2000.Google Scholar
  2. 2.
    Avez, A., “Entropy of groups of finite type,” C.R. Hebdomadaires Seances Acad. Sci. A, 275, pp. 13–63, 1972.Google Scholar
  3. 3.
    Bakry, D., Concordet, D., Ledoux, M., “Optimal heat kernel bounds under logarithmic Sobolev inequalities,” ESAIM: Probability and Statistics, 1, pp. 391–407, 1997.MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Barron, A.R., “Entropy and the central limit theorem,” Ann. Probab., 14, pp. 336–342, 1986.MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Beckner, W., “Sharp inequalities and geometric manifolds,” J. Fourier Anal. Applic. 3, pp. 825–836, 1997.MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Beckner, W., “Geometric inequalities in Fourier analysis,” in Essays on Fourier Analysis in Honor of Elias M. Stein, pp. 36–68 Princeton University Press, Princeton, NJ, 1995.Google Scholar
  7. 7.
    Blachman, N.M., “The convolution inequality for entropy powers,” IEEE Trans. Inform. Theory, 11(2), pp. 267–271, 1965.MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Brown, L.D., “A proof of the Central Limit Theorem motivated by the Cram´er–Rao inequality,” in Statistics and Probability: Essays in Honour of C.R. Rao, G. Kallianpur, P.R. Krishnaiah, and J.K. Ghosh, eds., pp. 141–148, North-Holland, New York, 1982.Google Scholar
  9. 9.
    Carlen, E.A., “Superadditivity of Fisher’s information and logarithmic Sobolev inequalities,” J. Funct. Anal., 101, pp. 194–211, 1991.MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Chirikjian, G.S., Stochastic Models, Information Theory, and Lie groups: Vol. 1, Birk¨auser, Boston, 2009.Google Scholar
  11. 11.
    Chirikjian, G.S., “Information-theoretic inequalities on unimodular Lie groups,” J. Geom. Mechan., 2(2), pp. 119–158, 2010.MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Cover, T.M., Thomas, J.A., Elements of Information Theory, 2nd ed. Wiley-Interscience, Hoboken, NJ, 2006.Google Scholar
  13. 13.
    Csisz´ar, I., “I-Divergence geometry of probability distributions and minimization problems,” Ann. Probab., 3(1), pp. 146–158, 1975.Google Scholar
  14. 14.
    Dembo, A., “Information inequalities and concentration of measure,” Ann. Prob., 25, pp. 527–539, 1997.MathSciNetGoogle Scholar
  15. 15.
    Dembo, A., Cover, T.M., Thomas, J.A., “Information theoretic inequalities,” IEEE Trans. Inform. Theory., 37(6), pp. 1501–1518, 1991.MathSciNetMATHCrossRefGoogle Scholar
  16. 16.
    Gibilisco, P., Isola, T., “Fisher information and Stam inequality of a finite group,” Bull. London Math. Soc., 40, pp. 855–862, 2008.MathSciNetMATHCrossRefGoogle Scholar
  17. 17.
    Gibilisco, P., Imparato, D., Isola, T., “Stam inequality on Zn,” Statist. Probab. Lett., 78, pp. 1851–1856, 2008.MathSciNetMATHCrossRefGoogle Scholar
  18. 18.
    Grenander, U., Probabilities on Algebraic Structures, Dover Published, New York, 2008. (originally publicated by John Wiley and Sons 1963).Google Scholar
  19. 19.
    Gross, L., “Logarithmic Sobolev inequalities,” Am. J. Math., 97, pp. 1061–1083, 1975.CrossRefGoogle Scholar
  20. 20.
    Gross, L., “Logarithmic Sobolev inequalities on Lie groups,” Illinois J. Math., 36(3), pp. 447–490, 1992.MathSciNetMATHGoogle Scholar
  21. 21.
    Heyer, H., Probability Measures on Locally Compact Groups, Springer-Verlag, New York, 1977.Google Scholar
  22. 22.
    Johnson, O., Suhov, Y., “Entropy and convergence on compact groups,” J. Theoret. Probab., 13(3), pp. 843–857, 2000.MathSciNetMATHCrossRefGoogle Scholar
  23. 23.
    Johnson, O., Information Theory and the Central Limit Theorem, Imperial College Press, London, 2004.Google Scholar
  24. 24.
    Ledoux, M., Concentration of Measure and Logarithmic Sobolev Inequalities, Lecture Notes in Mathematics Vol. 1709, Springer, Berlin, 1999.Google Scholar
  25. 25.
    Ledoux, M., The Concentration of Measure Phenomenon, Mathematical Surveys and Monographs Vol. 89, American Mathematical Society, Providence, RI, 2001.Google Scholar
  26. 26.
    Lieb, E.H., Loss, M., Analysis, 2nd ed., American Mathematical Society, Providence, RI, 2001.Google Scholar
  27. 27.
    Linnik, Y.V., “An information-theoretic proof of the Central Limit Theorem with the Lindeberg condition,” Theory Probab. Applic., 4(3), pp. 288–299, 1959.MathSciNetCrossRefGoogle Scholar
  28. 28.
    Maksimov, V.M., “Necessary and sufficient statistics for the family of shifts of probability distributions on continuous bicompact groups,” Theory Probab. Applic., 12(2), pp. 267–280, 1967.CrossRefGoogle Scholar
  29. 29.
    Otto, F., Villani, C., “Generalization of an inequality by Talagrand and links with the logarithmic Sobolev inequality,” J. Funct. Anal., 173, pp. 361–400, 2000.MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    Roy, K.K., “Exponential families of densities on an analytic group and sufficient statistics,” Sankhya: Indian J. Statist. A, 37(1), pp. 82–92, 1975.MATHGoogle Scholar
  31. 31.
    Stam, A.J., “Some inequalities satisfied by the quantities of information of Fisher and Shannon,” Inform. Control, 2(2), pp. 101–112, 1959.MathSciNetMATHCrossRefGoogle Scholar
  32. 32.
    Sugiura, M., Unitary Representations and Harmonic Analysis, 2nd ed., Elsevier Science Publisher, Amsterdam, 1990.Google Scholar
  33. 33.
    Talagrand, M., “New concentration inequalities in product spaces,” Invent. Math., 126, pp. 505–563, 1996.MathSciNetMATHCrossRefGoogle Scholar
  34. 34.
    Wang, Y., Chirikjian, G.S., “Nonparametric second-order theory of error propagation on the Euclidean group,” Int. J. Robot. Res., 27(1112), pp. 1258–1273, 2008.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Department of Mechanical EngineeringThe Johns Hopkins UniversityBaltimoreUSA

Personalised recommendations