Skip to main content
SpringerLink
Log in

Information Geometry and Its Applications: An Overview

  • Chapter
  • First Online:
Computational Information Geometry

Part of the book series: Signals and Communication Technology ((SCT))

Abstract

We give a personal view of what Information Geometry is, and what it is becoming, by exploring a number of key topics: dual affine families, boundaries, divergences, tensorial structures, and dimensionality. For each, we start with a graphical illustrative example (Sect. 1.1), give an overview of the relevant theory and key references (Sect. 1.2), and finish with a number of applications of the theory (Sect. 1.3). We treat ‘Information Geometry’ as an evolutionary term, deliberately not attempting a comprehensive definition. Rather, we illustrate how both the geometries used and application areas are rapidly developing.

Frank Critchley: This work has been partly funded by EPSRC grant EP/L010429/1

Paul Marriott: This work has been partly funded by NSERC discovery grant ‘Computational Information Geometry and Model Uncertainty’

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Agresti, A. (2013). Categorical data analysis. New Jersey: Wiley.

    Google Scholar 

  • Amari, S.-I. (1985). Differential-geometrical methods in statistics (Vol. 28). Heidelberg: Springer-Verlag.

    MATH  Google Scholar 

  • Amari, S.-I. (1995). Information geometry of the EM and em algorithms for neural networks. Neural Networks, 8(9), 1379–1408.

    Article  Google Scholar 

  • Amari, S.-I. (1997). Information geometry of neural networks - an overview. Mathematics of neural networks (pp. 15–23). Heidelberg: Springer.

    Chapter  Google Scholar 

  • Amari, S.-I. (1998). Natural gradient works efficiently in learning. Neural Computation, 10(2), 251–276.

    Article  Google Scholar 

  • Amari, S.-I. (2015). Information geometry as applied to neural spike data. Encyclopedia of Computational Neuroscience, 1431–1433.

    Google Scholar 

  • Amari, S.-I., Barndorff-Nielsen, O. E., Kass, R., Lauritzen, S., & Rao, C. (1987). Differential geometry in statistical inference. IMS Lecture Notes-Monograph Series, 1–240.

    Google Scholar 

  • Amari, S.-I., & Cichocki, A. (2010). Information geometry of divergence functions. Bulletin of the Polish Academy of Sciences: Technical Sciences, 58(1), 183–195.

    Google Scholar 

  • Amari, S.-I.,  & Kumon, M. (1988). Estimation in the presence of infinitely many nuisance parameters–geometry of estimating functions. The Annals of Statistics, 1044–1068.

    Google Scholar 

  • Amari, S.-I., Kurata, K., & Nagaoka, H. (1992). Information geometry of Boltzmann machines. IEEE Transactions on Neural Networks, 3(2), 260–271.

    Article  Google Scholar 

  • Amari, S.-I., & Nagaoka, H. (2007). Methods of information geometry (Vol. 191). Rhode Island: American Mathematical Society.

    MATH  Google Scholar 

  • Amari, S.-I., & Ohara, A. (2011). Geometry of q-exponential family of probability distributions. Entropy, 13(6), 1170–1185.

    Article  MathSciNet  MATH  Google Scholar 

  • Anaya-Izquierdo, K., Critchley, F., & Marriott, P. (2014). When are first-order asymptotics adequate? a diagnostic. Statistics, 3(1), 17–22.

    Article  Google Scholar 

  • Anaya-Izquierdo, K., Critchley, F., Marriott, P., & Vos, P. (2013a). Computational information geometry: foundations. Geometric science of information (pp. 311–318). Heidelberg: Springer.

    Chapter  Google Scholar 

  • Anaya-Izquierdo, K., Critchley, F., Marriott, P., & Vos, P. (2013b). Computational information geometry in statistics: Mixture modelling. Geometric science of information (pp. 319–326). Heidelberg: Springer.

    Chapter  Google Scholar 

  • Anaya-Izquierdo, K., Critchley, F., Marriott, P., & Vos, P. (2016). The geometry of model sensitivity: An illustration. In Computational information geometry: For image and signal processing. Heidelberg: Springer.

    MATH  Google Scholar 

  • Anaya-Izquierdo, K., Marriott, P. (2007). Local mixture models of exponential families. Bernoulli, 623–640.

    Google Scholar 

  • Arwini, K. A., & Dodson, C. T. J. (2008). Information geometry: Near randomness and near independence. Heidelberg: Springer.

    Book  MATH  Google Scholar 

  • Barndorff-Nielsen, O. (1978). Information and exponential families in statistical theory. New Jersey: Wiley.

    MATH  Google Scholar 

  • Barndorff-Nielsen, O.,& Blaesild, P. (1983). Exponential models with affine dual foliations. The Annals of Statistics, 753–769.

    Google Scholar 

  • Barndorff-Nielsen, O., Cox, D.,& Reid, N. (1986). The role of differential geometry in statistical theory. International Statistical Review/Revue Internationale de Statistique, 83–96.

    Google Scholar 

  • Barndorff-Nielsen, O. E. (1987). Differential geometry and statistics: some mathematical aspects. Indian Journal of Mathematics, 29(3), 335–350.

    MathSciNet  MATH  Google Scholar 

  • Barndorff-Nielsen, O. E., Blaesild, P., & Mora, M. (1989). Generalized higher-order differentiation. Acta Applicandae. Mathematica, 16(3), 243–259.

    MATH  Google Scholar 

  • Barndorff-Nielsen, O. E., & Cox, D. R. (1989). Asymptotic techniques for use in statistics. London: Chapman & Hall.

    Book  MATH  Google Scholar 

  • Barndorff-Nielsen, O. E., & Jupp, P. E. (1997). Statistics, yokes and symplectic geometry. In Annales de la Faculté des sciences de Toulouse: Mathématiques, 6, 389–427.

    Article  MathSciNet  MATH  Google Scholar 

  • Barndorff-Nielsen, O. E., Jupp, P. E., & Kendall, W. S. (1994). Stochastic calculus, statistical asymptotics, Taylor strings and phyla. In Annales de la Faculté des sciences de Toulouse: Mathématiques, 3, 5–62.

    Article  MathSciNet  MATH  Google Scholar 

  • Barndorff-Nielsen, O. E., & Koudou, A. E. (1996). Cuts in natural exponential families. Theory of Probability & Its Applications, 40(2), 220–229.

    Article  MathSciNet  MATH  Google Scholar 

  • Basseville, M. (2013). Divergence measures for statistical data processingan annotated bibliography. Signal Processing, 93(4), 621–633.

    Article  MathSciNet  Google Scholar 

  • Belavkin, R. V. (2013). Optimal measures and Markov transition kernels. Journal of Global Optimization, 55(2), 387–416.

    Article  MathSciNet  MATH  Google Scholar 

  • Betancourt, M. (2013). A general metric for Riemannian manifold Hamiltonian Monte Carlo. Geometric science of information (pp. 327–334). Heidelberg: Springer.

    Chapter  Google Scholar 

  • Betancourt, M., Byrne, S., Livingstone, S.,& Girolami M. (2014). The geometric foundations of Hamiltonian Monte Carlo. arXiv:1410.5110

  • Bhattacharya, A. (2008). Nonparametric statistics on manifolds with applications to shape spaces. ProQuest.

    Google Scholar 

  • Blaesild, P. (1991). Yokes and tensors derived from yokes. Annals of the Institute of Statistical Mathematics, 43(1), 95–113.

    Article  MathSciNet  MATH  Google Scholar 

  • Bregman, L. M. (1967). The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Mathematical Physics, 7(3), 200–217.

    Article  MathSciNet  MATH  Google Scholar 

  • Brody, D. C., & Hughston, L. P. (2001). Interest rates and information geometry. In Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences (Vol. 457, pp. 1343–1363). London: The Royal Society.

    Google Scholar 

  • Brown, L. D. (1986). Fundamentals of statistical exponential families with applications in statistical decision theory. IMS Lecture Notes-monograph Series.

    Google Scholar 

  • Buck, B., & Macaulay, V. A. (1991). Maximum entropy in action: a collection of expository essays. Oxford: Clarendon Press.

    Google Scholar 

  • Chentsov, N. N. (1972). Statistical decision rules and optimal inference (Vol. 53). Rhode Island: American Mathematical Society.

    Google Scholar 

  • Cichocki, A., Zdunek, R., Phan, A. H., & Amari, S.-I. (2009). Nonnegative matrix and tensor factorizations: Applications to exploratory multi-way data analysis and blind source separation. New Jersey: Wiley.

    Book  Google Scholar 

  • Collins, M., Schapire, R. E., & Singer, Y. (2002). Logistic regression. Adaboost and Bregman Distances. Machine Learning, 48(1–3), 253–285.

    Article  MATH  Google Scholar 

  • Cont, A., Dubnov, S., & Assayag, G. (2011). On the information geometry of audio streams with applications to similarity computing. IEEE Transactions on Audio, Speech, and Language Processing, 19(4), 837–846.

    Article  Google Scholar 

  • Copas, J., & Eguchi, S. (2005). Local model uncertainty and incomplete-data bias (with discussion). Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(4), 459–513.

    Article  MathSciNet  MATH  Google Scholar 

  • Cox, D., & Barndorff-Nielsen, O. (1994). Inference and asymptotics (Vol. 52). Florida: CRC Press.

    MATH  Google Scholar 

  • Cox, D. R., & Hinkley, D. V. (1979). Theoretical statistics. Florida: CRC Press.

    MATH  Google Scholar 

  • Cox, D. R., & N. Reid (1987). Parameter orthogonality and approximate conditional inference. Journal of the Royal Statistical Society. Series B (Methodological), 1–39.

    Google Scholar 

  • Cressie, N., & Read, T. R. (1984). Multinomial goodness-of-fit tests. Journal of the Royal Statistical Society. Series B (Methodological), 440–464.

    Google Scholar 

  • Critchley, F., & Marriott, P. (2014a). Computational information geometry in statistics: theory and practice. Entropy, 16, 2454–2471.

    Article  MathSciNet  Google Scholar 

  • Critchley, F., & Marriott, P. (2014b). Computing with Fisher geodesics and extended exponential families. Statistics and Computing, 1–8.

    Google Scholar 

  • Critchley, F., Marriott, P., & Salmon, M. (1994). Preferred point geometry and the local differential geometry of the Kullback-Leibler divergence. The Annals of Statistics, 1587–1602.

    Google Scholar 

  • Critchley, F., Marriott, P., & Salmon, M. (1996). On the differential geometry of the Wald test with nonlinear restrictions. Econometrica: Journal of the Econometric Society, 1213–1222.

    Google Scholar 

  • Csiszár, I. (1975). I-divergence geometry of probability distributions and minimization problems. The Annals of Probability, 146–158.

    Google Scholar 

  • Csiszár, I. (1995). Generalized projections for non-negative functions. Acta Mathematica Hungarica, 68(1–2), 161–186.

    Article  MathSciNet  MATH  Google Scholar 

  • Csiszár, I., et al. (1967). On topological properties of f-divergences. Studia Scientiarum Mathematicarum Hungarica, 2, 329–339.

    MathSciNet  MATH  Google Scholar 

  • Csiszár, I., & Matus, F. (2005). Closures of exponential families. The Annals of Probability, 33(2), 582–600.

    Article  MathSciNet  MATH  Google Scholar 

  • Dodson, C. T. (1987). Geometrization of statistical theory. In: Proceedings of the GST Workshop, University of Lancaster Department of Mathematics, 28–31 October 1987. ULDM Publications.

    Google Scholar 

  • Dodson, C. T., & Poston, T. (2013). Tensor geometry: the geometric viewpoint and its uses (Vol. 130). Heidelberg: Springer Science & Business Media.

    MATH  Google Scholar 

  • Efron, B. (1975). Defining the curvature of a statistical problem (with applications to second order efficiency). The Annals of Statistics, 1189–1242.

    Google Scholar 

  • Eguchi, S. (2006). Information geometry and statistical pattern recognition. Sugaku Expositions, 19(2), 197–216.

    MathSciNet  MATH  Google Scholar 

  • Eguchi, S. (2009). Information divergence geometry and the application to statistical machine learning. Information theory and statistical learning (pp. 309–332). Heidelberg: Springer.

    Chapter  Google Scholar 

  • Eguchi, S., et al. (1985). A differential geometric approach to statistical inference on the basis of contrast functionals. Hiroshima Mathematical Journal, 15(2), 341–391.

    MathSciNet  MATH  Google Scholar 

  • Eguchi, S., et al. (1992). Geometry of minimum contrast. Hiroshima Mathematical Journal, 22(3), 631–647.

    MathSciNet  MATH  Google Scholar 

  • Eguchi, S., & Copas, J. (1998). A class of local likelihood methods and near-parametric asymptotics. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 60(4), 709–724.

    Article  MathSciNet  MATH  Google Scholar 

  • Eguchi, S., Komori, O., & Ohara, A. (2014). Duality of maximum entropy and minimum divergence. Entropy, 16(7), 3552–3572.

    Article  MathSciNet  MATH  Google Scholar 

  • Freund, Y., & Schapire, R. E. (1995). A decision-theoretic generalization of on-line learning and an application to boosting. Computational learning theory (pp. 23–37). Heidelberg: Springer.

    Chapter  Google Scholar 

  • Fu, W., Johnston, M., & Zhang, M. (2013). Gaussian mixture models and information entropy for image segmentation using particle swarm optimisation. 2013 28th International Conference of Image and Vision Computing New Zealand (IVCNZ) (pp. 328–333). New Jersey: IEEE.

    Chapter  Google Scholar 

  • Fukuda, K. (2004). From the zonotope construction to the Minkowski addition of convex polytopes. Journal of Symbolic Computation, 38, 1261–1272.

    Article  MathSciNet  MATH  Google Scholar 

  • Fukumizu, K. (2005). Infinite dimensional exponential families by reproducing kernel Hilbert spaces. In Proceedings of the 2nd International Symposium on Information Geometry and its Applications (pp. 324–333).

    Google Scholar 

  • Geyer, C. J. (2009). Likelihood inference in exponential families and directions of recession. Electronic Journal of Statistics, 3, 259–289.

    Article  MathSciNet  MATH  Google Scholar 

  • Gibilisco, P., & Pistone, G. (1998). Connections on non-parametric statistical manifolds by Orlicz space geometry. Infinite Dimensional Analysis, Quantum Probability and Related Topics, 1(02), 325–347.

    Article  MathSciNet  MATH  Google Scholar 

  • Gibilisco, P., Riccomagno, E., Rogantin, M., & Wynn, H. (2010). Algebraic and Geometric Methods in Statistics. New York, NY: Cambridge University Press.

    MATH  Google Scholar 

  • Girolami, M., & Calderhead, B. (2011). Riemann manifold Langevin and Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(2), 123–214.

    Article  MathSciNet  Google Scholar 

  • Ikeda, S., Tanaka, T., & Amari, S.-I. (2004). Stochastic reasoning, free energy, and information geometry. Neural Computation, 16(9), 1779–1810.

    Article  MATH  Google Scholar 

  • Jaynes, E. T. (1978). Where do we stand on maximum entropy. The maximum entropy formalism, 15–118.

    Google Scholar 

  • Jaynes, E. T. (1982). On the rationale of maximum-entropy methods. Proceedings of the IEEE, 70(9), 939–952.

    Article  Google Scholar 

  • Jordan, M., Sudderth, E. B., Wainwright, M., Willsky, A. S., et al. (2010). Major advances and emerging developments of graphical models [from the guest editors]. Signal Processing Magazine, IEEE, 27(6), 17–138.

    Article  Google Scholar 

  • Jorgensen, B. (1987). Exponential dispersion models. Journal of the Royal Statistical Society. Series B (Methodological), 127–162.

    Google Scholar 

  • Kahle, T., et al. (2010). Neighborliness of marginal polytopes. Contributions to Algebra and Geometry, 51(1), 45–56.

    MathSciNet  MATH  Google Scholar 

  • Karlin, S. (1968). Total positivity (Vol. 1). California: Stanford University Press.

    MATH  Google Scholar 

  • Karlin, S., & Shapley, L. S. (1953). Geometry of moment spaces. Memoirs of the American Mathematical Society 12.

    Google Scholar 

  • Kass, R., Tierney, L., & Kadane, J. (1988). Asymptotics in Bayesian computation. Bayesian statistics, 3, 261–278.

    Google Scholar 

  • Kass, R. E. (1989). The geometry of asymptotic inference. Statistical Science, 188–219.

    Google Scholar 

  • Kass, R. E., Tierney, L., & Kadane, J. B. (1991). Laplace method in Bayesian analysis. Contemporary Mathematics, 115, 89–99.

    Article  MathSciNet  MATH  Google Scholar 

  • Kass, R. E., & Vos, P. W. (2011). Geometrical foundations of asymptotic inference (Vol. 908). New Jersey: Wiley.

    MATH  Google Scholar 

  • Kawakita, M., & Eguchi, S. (2008). Boosting method for local learning in statistical pattern recognition. Neural computation, 20(11), 2792–2838.

    Article  MathSciNet  MATH  Google Scholar 

  • Kolaczyk, E. D., & Nowak, R. D. (2004). Multiscale likelihood analysis and complexity penalized estimation. Annals of Statistics, 500–527.

    Google Scholar 

  • Kolaczyk, E. D., & Nowak, R. D. (2005). Multiscale generalised linear models for nonparametric function estimation. Biometrika, 92(1), 119–133.

    Article  MathSciNet  MATH  Google Scholar 

  • Lauritzen, S. L. (1987). Statistical manifolds. Differential geometry in Statistical Science (pp. 163–216). CA: IMS Hayward.

    Google Scholar 

  • Lauritzen, S. L. (1996). Graphical models. Oxford: Oxford University Press.

    MATH  Google Scholar 

  • Lee, J. A., & Verleysen, M. (2007). Nonlinear dimensionality reduction. Heidelberg: Springer Science & Business Media.

    Book  MATH  Google Scholar 

  • Lenglet, C., Rousson, M., Deriche, R., & Faugeras, O. (2006). Statistics on the manifold of multivariate normal distributions: Theory and application to diffusion tensor MRI processing. Journal of Mathematical Imaging and Vision, 25(3), 423–444.

    Article  MathSciNet  Google Scholar 

  • Li, P., Chen, J., & Marriott, P. (2009). Non-finite fisher information and homogeneity: An em approach. Biometrika, 96(2), 411–426.

    Article  MathSciNet  MATH  Google Scholar 

  • Lindsay, B. (1995). Mixture Models: Theory, Geometry, and Applications. Institute of Mathematical Statistics.

    Google Scholar 

  • Liu, M., Vemuri, B., Amari, S.-I., & Nielsen, F. (2012). Shape retrieval using heirarchical total Bregman soft clustering. IEEE Transactions on pattern analysis and machine intelligence, 34, 2407–2419.

    Article  Google Scholar 

  • Marriott, P. (2002). On the local geometry of mixture models. Biometrika, 89(1), 77–93.

    Article  MathSciNet  MATH  Google Scholar 

  • Marriott, P.,& Salmon, M. (2000). Applications of differential geometry to econometrics. Cambridge University Press.

    Google Scholar 

  • McCullagh, P. (1987). Tensor methods in statistics (Vol. 161). London: Chapman and Hall.

    MATH  Google Scholar 

  • McCullagh, P., & Nelder, J. A. (1989). Generalized linear models (Vol. 37). Florida: CRC Press.

    Book  MATH  Google Scholar 

  • Mio, W., Badlyans, D.,& Liu, X. (2005a). A computational approach to fisher information geometry with applications to image analysis. Proceedings of the EMMCVPR, 18–33.

    Google Scholar 

  • Mio, W., Badlyans, D., & Liu, X. (2005b). A computational approach to fisher information geometry with applications to image analysis. Energy minimization methods in computer vision and pattern recognition (pp. 18–33). Heidelberg: Springer.

    Chapter  Google Scholar 

  • Mio, W., & Liu, X. (2006). Landmark representation of shapes and Fisher-Rao geometry. 2006 IEEE International Conference on Image Processing (pp. 2113–2116). New Jersey: IEEE.

    Chapter  Google Scholar 

  • Morozova, E. A., & Chentsov, N. N. (1991). Natural geometry of families of probability laws. Itogi Nauki i Tekhniki. Seriya “Sovremennye Problemy Matematiki. Fundamental’nye Napravleniya”, 83, 133–265.

    Google Scholar 

  • Murray, M. K., & Rice, J. W. (1993). Differential geometry and statistics (Vol. 48). Florida: CRC Press.

    Book  MATH  Google Scholar 

  • Newton, N. J. (2012). An infinite-dimensional statistical manifold modelled on Hilbert space. Journal of Functional Analysis, 263(6), 1661–1681.

    Article  MathSciNet  MATH  Google Scholar 

  • Nielsen, F. (2014). Geometric Theory of Information. Heidelberg: Springer.

    Book  Google Scholar 

  • Nielsen, F.,& Barbaresco, F. (2014). Proceedings of the 2nd International Symposium on Information Geometry and its Applications. Heidelberg: Springer.

    Google Scholar 

  • Nielsen, F., & Bhatia, R. (2013). Matrix information geometry. Heidelberg: Springer.

    Book  MATH  Google Scholar 

  • Nielsen, F., & Nock, N. (2014a). Optimal interval clustering: Application to Bregman clustering and statistical mixture learning. IEEE Transactions on pattern analysis and machine intelligence, 21(10), 1289–1292.

    Google Scholar 

  • Nielsen, F., & Nock, R. (2014b). On the chi square and higher-order chi distances for approximating \(f\)-divergences. IEEE Signal Processing Letters, 21(1), 10–13.

    Article  Google Scholar 

  • Peter, A., & Rangarajan, A. (2006). Shape analysis using the Fisher-Rao Riemannian metric: Unifying shape representation and deformation. 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2006 (pp. 1164–1167). New Jersey: IEEE.

    Chapter  Google Scholar 

  • Pfanzagl, J. (1973). Asymptotic expansions related to minimum contrast estimators. The Annals of Statistics, 993–1026.

    Google Scholar 

  • Pistone, G. (2013). Nonparametric information geometry. In Geometric Science (Ed.), of Information (pp. 5–36). Heidelberg: Springer.

    Google Scholar 

  • Pistone, G., Riccomagno, E., & Wynn, H. (2000). Algebraic Statistics: Computational Commutative Algebra in Statistics. London: Chapman and Hall.

    Book  MATH  Google Scholar 

  • Pistone, G., Rogantin, M. P., et al. (1999). The exponential statistical manifold: mean parameters, orthogonality and space transformations. Bernoulli, 5(4), 721–760.

    Article  MathSciNet  MATH  Google Scholar 

  • Ramezan, R., Marriott, P., & Chenouri, S. (2014). Multiscale analysis of neural spike trains. Statistics in medicine, 33(2), 238–256.

    Article  MathSciNet  Google Scholar 

  • Rao, C. (1945). Information and accuracy attainable in the estimation of statistical parameters. Bulletin of the Calcutta Mathematical Society, 37(3), 81–91.

    MathSciNet  MATH  Google Scholar 

  • Rao, C. R. (1987). Differential metrics in probability spaces. Differential geometry in statistical inference, 10, 217–240.

    Google Scholar 

  • Read, T. R., & Cressie, N. (2012). Goodness-of-fit statistics for discrete multivariate data. Heidelberg: Springer Science & Business Media.

    MATH  Google Scholar 

  • Rinaldo, A., Feinberg, S., & Zhou, Y. (2009). On the geometry of discrete exponential families with applications to exponential random graph models. Electronic Journal of Statistics, 3, 446–484.

    Article  MathSciNet  MATH  Google Scholar 

  • Rockafellar, R. T. (1997). Convex analysis. Princeton landmarks in mathematics. Princeton: Princeton University Press.

    Google Scholar 

  • Shima, H. (2007). The geometry of Hessian structures (Vol. 1). Singapore: World Scientific.

    Book  MATH  Google Scholar 

  • Simpson, S. L., Hayasaka, S., & Laurienti, P. J. (2011). Exponential random graph modeling for complex brain networks. PLoS One, 6(5), e20039.

    Article  Google Scholar 

  • Skilling, J. (1989). Classic maximum entropy. In Maximum Entropy and Bayesian Methods (pp. 45–52). Heidelberg: Springer.

    Google Scholar 

  • Small, C. G., & McLeish, D. L. (2011). Hilbert space methods in probability and statistical inference (Vol. 920). New Jersey: Wiley.

    MATH  Google Scholar 

  • Sontag, D., & Jaakkola, T. S. (2007). New outer bounds on the marginal polytope. In Advances in Neural Information Processing Systems (NIPS), 20, 1393–1400.

    Google Scholar 

  • Takatsu, A. (2013). Behaviors of \(\varphi \)-exponential distributions in Wasserstein geometry and an evolution equation. SIAM Journal on Mathematical Analysis, 45(4), 2546–2556.

    Article  MathSciNet  MATH  Google Scholar 

  • Takenouchi, T., Eguchi, S., Murata, N., & Kanamori, T. (2008). Robust boosting algorithm against mislabeling in multiclass problems. Neural computation, 20(6), 1596–1630.

    Article  MathSciNet  MATH  Google Scholar 

  • Takenouchi, T., Komori, O., & Eguchi, S. (2012). An extension of the receiver operating characteristic curve and AUC-optimal classification. Neural computation, 24(10), 2789–2824.

    Article  MathSciNet  MATH  Google Scholar 

  • Takenouchi, T., Komori, O.,& Eguchi, S. (2015). A novel boosting algorithm for multi-task learning based on the Itakuda-Saito divergence. In Bayesian inference and Maximum Entropy methods in science and engineering (MAXENT 2014) (Vol. 1641, pp. 230–237). Melville: AIP Publishing.

    Google Scholar 

  • Tatsuno, M., Fellous, J.-M., & Amari, S.-I. (2009). Information-geometric measures as robust estimators of connection strengths and external inputs. Neural computation, 21(8), 2309–2335.

    Article  MathSciNet  MATH  Google Scholar 

  • Tatsuno, M., & Okada, M. (2003). How does the information-geometric measure depend on underlying neural mechanisms? Neurocomputing, 52, 649–654.

    Article  MathSciNet  Google Scholar 

  • Tierney, L., Kass, R. E., & Kadane, J. B. (1989). Fully exponential Laplace approximations to expectations and variances of nonpositive functions. Journal of the American Statistical Association, 84(407), 710–716.

    Article  MathSciNet  MATH  Google Scholar 

  • Tsallis, C. (1988). Possible generalization of Boltzmann-Gibbs statistics. Journal of statistical physics, 52(1–2), 479–487.

    Article  MathSciNet  MATH  Google Scholar 

  • Tsallis, C. (2009). Introduction to nonextensive statistical mechanics. Heidelberg: Springer.

    MATH  Google Scholar 

  • Tuy, H. (1998). Convex analysis and global optimization. London: Klumer academic publishers.

    Book  MATH  Google Scholar 

  • Vos, P. W., & Marriott, P. (2010). Geometry in statistics. Wiley Interdisciplinary Reviews: Computational Statistics, 2(6), 686–694.

    Article  Google Scholar 

  • Wainwright, M. J., & Jordan, M. I. (2003). Variational inference in graphical models: The view from the marginal polytope. In Proceedings of the Annal Allerton Conference on Communication Control and Computing (Vol. 41, pp. 961–971). Citeseer.

    Google Scholar 

  • Wainwright, M. J., & Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 1(1–2), 1–305.

    Article  MATH  Google Scholar 

  • Watanabe, S. (2009). Algebraic geometry and statistical learning theory (Vol. 25). Cambridge University Press.

    Google Scholar 

  • Wong, W. H., & Li, B. (1992). Laplace expansion for posterior densities of nonlinear functions of parameters. Biometrika, 79(2), 393–398.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang, H., Wu, Q., & Nguyen, T. M. (2013). Image segmentation by a robust modified gaussian mixture model. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1478–1482). New Jersey: IEEE.

    Chapter  Google Scholar 

  • Zhang, J. (2004). Divergence function, duality, and convex analysis. Neural Computation, 16(1), 159–195.

    Article  MATH  Google Scholar 

  • Zhang, J. (2006). Referential duality and representational duality on statistical manifolds. In Proceedings of the Second International Symposium on Information Geometry and Its Applications, Tokyo (pp. 58–67).

    Google Scholar 

  • Zhang, J. (2013). Nonparametric information geometry: From divergence function to referential-representational biduality on statistical manifolds. Entropy, 15(12), 5384–5418.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang, J. (2015). Reference duality and representation duality in information geometry. In Bayesian inference and Maximum Entropy methods in science and engineering (MAXENT 2014) (Vol. 1641, pp. 130–146). Melville: AIP Publishing.

    Google Scholar 

  • Zhao, H., & Marriott, P. (2014). Variational Bayes for regime-switching log-normal models. Entropy, 16(7), 3832–3847.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Open University, Walton Hall, Milton Keynes, Buckinghamshire, MK7 6AA, UK

    Frank Critchley

  2. University of Waterloo, 200 University Avenue West, Waterloo, ON, Canada

    Paul Marriott

Authors
  1. Frank Critchley
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul Marriott
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul Marriott .

Editor information

Editors and Affiliations

  1. Laboratoire d’Informatique (LIX),Palaiseau Cedex,France and Sony Computer Science Laboratories, Inc., Tokyo, Japan., Ecole Polytechnique, Palaiseau Cedex, France

    Frank Nielsen

  2. School of Mathematics and Statistics, The Open University, Milton Keynes, United Kingdom

    Frank Critchley

  3. Department of Mathematics, University of Manchester, Manchester, United Kingdom

    Christopher T. J. Dodson

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Critchley, F., Marriott, P. (2017). Information Geometry and Its Applications: An Overview. In: Nielsen, F., Critchley, F., Dodson, C. (eds) Computational Information Geometry. Signals and Communication Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-47058-0_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-47058-0_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-47056-6

  • Online ISBN: 978-3-319-47058-0

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation