Abstract
We apply the functional analytical and differential geometric results of the preceding chapters to the field of statistics and obtain very general versions of the basic classical results. In a narrower sense, the term statistic refers to a mapping from a given sample space \(\varOmega\) to another \(\varOmega'\), and it is called sufficient for a parametric family, if the parameter can be estimated as well from that statistic as from samples taken on the original space \(\varOmega\). More generally, a Markov kernel associates to a sample from \(\varOmega\) a probability measure on \(\varOmega'\). We prove a general version of Chentsov’s theorem, saying that the Fisher metric and the Amari–Chentsov tensor are characterized by their invariance under sufficient statistics. The Cramér–Rao inequality, of which we also prove a new general version, controls the variance of an unbiased estimator from below by the inverse of the Fisher metric. When the estimator is possibly biased, additional terms enter, which we likewise handle in our framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
More precisely, a semiparametric model is a statistical model that has both parametric and nonparametric components. (This distinction is only meaningful when the parametric component is considered to be more important than the nonparametric one.)
References
Amari, S.: Differential geometry of curved exponential families curvature and information loss. Ann. Stat. 10, 357–385 (1982)
Amari, S.: Differential-Geometric Methods in Statistics. Lecture Notes in Statistics, vol. 28. Springer, Heidelberg (1985)
Amari, S.: Differential geometrical theory of statistics. In: Differential Geometry in Statistical Inference, Institute of Mathematical Statistics, California. Lecture Notes–Monograph Series, vol. 10 (1987)
Amari, S., Nagaoka, H.: Methods of Information Geometry. Translations of Mathematical Monographs, vol. 191. Am. Math. Soc./Oxford University Press, Providence/London (2000)
Ay, N., Jost, J., Lê, H.V., Schwachhöfer, L.: Information geometry and sufficient statistics. Probab. Theory Relat. Fields 162, 327–364 (2015)
Ay, N., Jost, J., Lê, H.V., Schwachhöfer, L.: Parametrized measure models. Bernoulli (2015). To appear, arXiv:1510.07305
Ay, N., Jost, J., Lê, H.V., Schwachhöfer, L.: Invariant geometric structures on statistical models. In: Nielsen, F., Barbaresco, F. (eds.) Proceedings of Geometric Science of Information, Second International Conference, GSI 2015, Palaiseau, France, October 28–30, 2015. Springer Lecture Notes in Computer Sciences, vol. 9389 (2015)
Barndorff-Nielsen, O.: Information and Exponential Families in Statistical Theory. Wiley Series in Probability and Mathematical Statistics. John Wiley & Sons, Ltd., Chichester (2014)
Bauer, H.: Probability Theory. de Gruyter, Berlin (1996). Translated from the German by R.B. Burckel
Bickel, P.J., Klaassen, C.A.J., Ritov, Y., Wellner, J.A.: Efficient and Adaptive Estimation for Semiparametric Models. Springer, New York (1998)
Bogachev, V.I.: Measure Theory, vols. I, II. Springer, Berlin (2007)
Borovkov, A.A.: Mathematical Statistics. Gordon & Breach, New York (1998)
Chentsov, N.: Category of mathematical statistics. Dokl. Akad. Nauk SSSR 164, 511–514 (1965)
Chentsov, N.: A systematic theory of exponential families of probability distributions. Teor. Veroâtn. Primen. 11, 483–494 (1966) (Russian)
Chentsov, N.: Statistical Decision Rules and Optimal Inference, vol. 53. Nauka, Moscow (1972) (in Russian); English translation in: Math. Monograph., vol. 53. Am. Math. Soc., Providence (1982)
Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. Wiley, New York (2006)
Cramer, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)
Dudley, R.M.: Real Analysis and Probability. Cambridge University Press, Cambridge (2004)
Fisher, R.A.: On the mathematical foundations of theoretical statistics. Philos. Trans. R. Soc. Lond. Ser. A 222, 309–368 (1922)
Jost, J., Lê, H.V., Schwachhöfer, L.: Cramér–Rao inequality on singular statistical models I. arXiv:1703.09403
Jost, J., Lê, H.V., Schwachhöfer, L.: Cramér–Rao inequality on singular statistical models II. Preprint (2017)
Lauritzen, S.: General exponential models for discrete observations. Scand. J. Stat. 2, 23–33 (1975)
Lê, H.V.: The uniqueness of the Fisher metric as information metric. Ann. Inst. Stat. Math. 69, 879–896 (2017). doi:10.1007/s10463-016-0562-0
Morozova, E., Chentsov, N.: Natural geometry on families of probability laws. Itogi Nauki Tekh. Ser. Sovrem. Mat. Prilozh. Temat. Obz. 83, 133–265 (1991)
Morse, N., Sacksteder, R.: Statistical isomorphism. Ann. Math. Stat. 37, 203–214 (1966)
Neveu, J.: Mathematical Foundations of the Calculus of Probability. Holden-Day Series in Probability and Statistics (1965)
Neyman, J.: Sur un teorema concernente le cosidette statistiche sufficienti. G. Ist. Ital. Attuari 6, 320–334 (1935)
Rao, C.R.: Information and the accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc. 37, 81–89 (1945)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Ay, N., Jost, J., Lê, H.V., Schwachhöfer, L. (2017). Information Geometry and Statistics. In: Information Geometry. Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge / A Series of Modern Surveys in Mathematics, vol 64. Springer, Cham. https://doi.org/10.1007/978-3-319-56478-4_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-56478-4_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56477-7
Online ISBN: 978-3-319-56478-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)