Geometric Theory of Information pp 119-140 | Cite as

# Computational Algebraic Methods in Efficient Estimation

## Abstract

A strong link between information geometry and algebraic statistics is made by investigating statistical manifolds which are algebraic varieties. In particular it is shown how first and second order efficient estimators can be constructed, such as bias corrected Maximum Likelihood and more general estimators, and for which the estimating equations are purely algebraic. In addition it is shown how Gröbner basis technology, which is at the heart of algebraic statistics, can be used to reduce the degrees of the terms in the estimating equations. This points the way to the feasible use, to find the estimators, of special methods for solving polynomial equations, such as homotopy continuation methods. Simple examples are given showing both equations and computations.

## Keywords

Efficient Estimator Exponential Family Algebraic Statistic Information Geometry Asymptotic Efficiency## Notes

### Acknowledgments

This paper has benefited from conversations with and advice from a number of colleagues. We should thank Satoshi Kuriki, Tomonari Sei, Wicher Bergsma and Wilfred Kendall. The first author acknowledges support by JSPS KAKENHI Grant 20700258, 24700288 and the second author acknowledges support from the Institute of Statistical Mathematics for two visits in 2012 and 2013 and from UK EPSRC Grant EP/H007377/1. A first version of this paper was delivered at the WOGAS3 meeting at the University of Warwick in 2011. We thank the sponsors. The authors also thank the referees of the short version in GSI2013 and the referees of the first long version of the paper for insightful suggestions.

## Supplementary material

## References

- 1.Adler, R.J., Taylor, J.E.: Random Fields and Geometry. Springer Monographs in Mathematics. Springer, New York (2007)Google Scholar
- 2.Amari, S., Nagaoka, H.: Methods of Information Geometry, vol. 191. American Mathematical Society, Providence (2007)Google Scholar
- 3.Amari, S.: Differential geometry of curved exponential families-curvatures and information loss. Ann. Stat.
**10**, 357–385 (1982)Google Scholar - 4.Amari, S.: Differential-Geometrical Methods in Statistics. Springer, New York (1985)Google Scholar
- 5.Amari, S., Kumon, M.: Differential geometry of Edgeworth expansions in curved exponential family. Ann. Inst. Stat. Math.
**35**(1), 1–24 (1983)zbMATHMathSciNetGoogle Scholar - 6.Andersson, S., Madsen, J.: Symmetry and lattice conditional independence in a multivariate normal distribution. Ann. Stat.
**26**(2), 525–572 (1998)zbMATHMathSciNetGoogle Scholar - 7.Bergsma, W.P., Croon, M., Hagenaars, J.A.: Marginal Models for Dependent, Clustered, and Longitudinal Categorical Data. Springer, New York (2009)Google Scholar
- 8.Buchberger, B.: Bruno Buchberger’s PhD thesis 1965: an algorithm for finding the basis elements of the residue class ring of a zero dimensional polynomial ideal. J. Symbol. Comput.
**41**(3), 475–511 (2006)Google Scholar - 9.Cox, D.A., Little, J., O’Shea, D.: Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, 3/e (Undergraduate Texts in Mathematics). Springer, New York (2007)Google Scholar
- 10.Dawid, A.P.: Further comments on some comments on a paper by Bradley Efron. Ann. Stat.
**5**(6), 1249 (1977)CrossRefzbMATHMathSciNetGoogle Scholar - 11.Drton, M., Sturmfels B., Sullivant, S.: Lectures on Algebraic Statistics. Springer, New York (2009)Google Scholar
- 12.Drton, M.: Likelihood ratio tests and singularities. Ann. Stat.
**37**, 979–1012 (2009)Google Scholar - 13.Efron, B.: Defining the curvature of a statistical problem (with applications to second order efficiency). Ann. Stat.
**3**, 1189–1242 (1975)CrossRefzbMATHMathSciNetGoogle Scholar - 14.Gehrmann, H., Lauritzen, S.L.: Estimation of means in graphical Gaussian models with symmetries. Ann. Stat.
**40**(2), 1061–1073 (2012)CrossRefzbMATHMathSciNetGoogle Scholar - 15.Gibilisco, P., Riccomagno, E., Rogantin, M.P., Wynn, H.P.: Algebraic and Geometric Methods in Statistics. Cambridge University Press, Cambridge (2009)Google Scholar
- 16.Huber, B., Sturmfels, B.: A polyhedral method for solving sparse polynomial systems. Math. Comput.
**64**, 1541–1555 (1995)CrossRefzbMATHMathSciNetGoogle Scholar - 17.Kuriki, S., Takemura, A.: On the equivalence of the tube and Euler characteristic methods for the distribution of the maximum of Gaussian fields over piecewise smooth domains. Ann. Appl. Probab.
**12**(2), 768–796 (2002)CrossRefzbMATHMathSciNetGoogle Scholar - 18.Lee, T.L., Li, T.Y., Tsai, C.H.: HOM4PS2.0: a software package for solving polynomial systems by the polyhedral homotopy continuation method. Computing
**83**(2–3), 109–133 (2008)CrossRefzbMATHMathSciNetGoogle Scholar - 19.Li, T.Y.: Numerical solution of multivariate polynomial systems by homotopy continuation methods. Acta Numer
**6**(1), 399–436 (1997)CrossRefGoogle Scholar - 20.Naiman, D.Q.: Conservative confidence bands in curvilinear regression. Ann. Stat.
**14**(3), 896–906 (1986)CrossRefzbMATHMathSciNetGoogle Scholar - 21.Pistone, G., Sempi, C.: An infinite-dimensional geometric structure on the space of all the probability measures equivalent to a given one. Ann. Stat.
**23**, 1543–1561 (1995)Google Scholar - 22.Pistone, G., Wynn, H.P.: Generalised confounding with Gröbner bases. Biometrika
**83**, 653–666 (1996)CrossRefzbMATHMathSciNetGoogle Scholar - 23.Rao, R.C.: Information and accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc.
**37**(3), 81–91 (1945)zbMATHMathSciNetGoogle Scholar - 24.Verschelde, J.: Algorithm 795: PHCpack: a general-purpose solver for polynomial systems by homotopy continuation. ACM Trans. Math. Softw. (TOMS)
**25**(2), 251–276 (1999)CrossRefzbMATHGoogle Scholar - 25.Watanabe, S.: Algebraic analysis for singular statistical estimation. In: Watanabe, O., Yokomori, T. (eds.) Algorithmic Learning Theory. Springer, Berlin (1999)Google Scholar
- 26.Weyl, H.: On the volume of tubes. Am. J. Math.
**61**, 461–472 (1939)CrossRefMathSciNetGoogle Scholar