Computational Algebraic Methods in Efficient Estimation

  • Kei KobayashiEmail author
  • Henry P. Wynn
Part of the Signals and Communication Technology book series (SCT)


A strong link between information geometry and algebraic statistics is made by investigating statistical manifolds which are algebraic varieties. In particular it is shown how first and second order efficient estimators can be constructed, such as bias corrected Maximum Likelihood and more general estimators, and for which the estimating equations are purely algebraic. In addition it is shown how Gröbner basis technology, which is at the heart of algebraic statistics, can be used to reduce the degrees of the terms in the estimating equations. This points the way to the feasible use, to find the estimators, of special methods for solving polynomial equations, such as homotopy continuation methods. Simple examples are given showing both equations and computations.


Efficient Estimator Exponential Family Algebraic Statistic Information Geometry Asymptotic Efficiency 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This paper has benefited from conversations with and advice from a number of colleagues. We should thank Satoshi Kuriki, Tomonari Sei, Wicher Bergsma and Wilfred Kendall. The first author acknowledges support by JSPS KAKENHI Grant 20700258, 24700288 and the second author acknowledges support from the Institute of Statistical Mathematics for two visits in 2012 and 2013 and from UK EPSRC Grant EP/H007377/1. A first version of this paper was delivered at the WOGAS3 meeting at the University of Warwick in 2011. We thank the sponsors. The authors also thank the referees of the short version in GSI2013 and the referees of the first long version of the paper for insightful suggestions.

Supplementary material


  1. 1.
    Adler, R.J., Taylor, J.E.: Random Fields and Geometry. Springer Monographs in Mathematics. Springer, New York (2007)Google Scholar
  2. 2.
    Amari, S., Nagaoka, H.: Methods of Information Geometry, vol. 191. American Mathematical Society, Providence (2007)Google Scholar
  3. 3.
    Amari, S.: Differential geometry of curved exponential families-curvatures and information loss. Ann. Stat. 10, 357–385 (1982)Google Scholar
  4. 4.
    Amari, S.: Differential-Geometrical Methods in Statistics. Springer, New York (1985)Google Scholar
  5. 5.
    Amari, S., Kumon, M.: Differential geometry of Edgeworth expansions in curved exponential family. Ann. Inst. Stat. Math. 35(1), 1–24 (1983)zbMATHMathSciNetGoogle Scholar
  6. 6.
    Andersson, S., Madsen, J.: Symmetry and lattice conditional independence in a multivariate normal distribution. Ann. Stat. 26(2), 525–572 (1998)zbMATHMathSciNetGoogle Scholar
  7. 7.
    Bergsma, W.P., Croon, M., Hagenaars, J.A.: Marginal Models for Dependent, Clustered, and Longitudinal Categorical Data. Springer, New York (2009)Google Scholar
  8. 8.
    Buchberger, B.: Bruno Buchberger’s PhD thesis 1965: an algorithm for finding the basis elements of the residue class ring of a zero dimensional polynomial ideal. J. Symbol. Comput. 41(3), 475–511 (2006)Google Scholar
  9. 9.
    Cox, D.A., Little, J., O’Shea, D.: Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, 3/e (Undergraduate Texts in Mathematics). Springer, New York (2007)Google Scholar
  10. 10.
    Dawid, A.P.: Further comments on some comments on a paper by Bradley Efron. Ann. Stat. 5(6), 1249 (1977)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Drton, M., Sturmfels B., Sullivant, S.: Lectures on Algebraic Statistics. Springer, New York (2009)Google Scholar
  12. 12.
    Drton, M.: Likelihood ratio tests and singularities. Ann. Stat. 37, 979–1012 (2009)Google Scholar
  13. 13.
    Efron, B.: Defining the curvature of a statistical problem (with applications to second order efficiency). Ann. Stat. 3, 1189–1242 (1975)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Gehrmann, H., Lauritzen, S.L.: Estimation of means in graphical Gaussian models with symmetries. Ann. Stat. 40(2), 1061–1073 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    Gibilisco, P., Riccomagno, E., Rogantin, M.P., Wynn, H.P.: Algebraic and Geometric Methods in Statistics. Cambridge University Press, Cambridge (2009)Google Scholar
  16. 16.
    Huber, B., Sturmfels, B.: A polyhedral method for solving sparse polynomial systems. Math. Comput. 64, 1541–1555 (1995)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Kuriki, S., Takemura, A.: On the equivalence of the tube and Euler characteristic methods for the distribution of the maximum of Gaussian fields over piecewise smooth domains. Ann. Appl. Probab. 12(2), 768–796 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Lee, T.L., Li, T.Y., Tsai, C.H.: HOM4PS2.0: a software package for solving polynomial systems by the polyhedral homotopy continuation method. Computing 83(2–3), 109–133 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Li, T.Y.: Numerical solution of multivariate polynomial systems by homotopy continuation methods. Acta Numer 6(1), 399–436 (1997)CrossRefGoogle Scholar
  20. 20.
    Naiman, D.Q.: Conservative confidence bands in curvilinear regression. Ann. Stat. 14(3), 896–906 (1986)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Pistone, G., Sempi, C.: An infinite-dimensional geometric structure on the space of all the probability measures equivalent to a given one. Ann. Stat. 23, 1543–1561 (1995)Google Scholar
  22. 22.
    Pistone, G., Wynn, H.P.: Generalised confounding with Gröbner bases. Biometrika 83, 653–666 (1996)CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    Rao, R.C.: Information and accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc. 37(3), 81–91 (1945)zbMATHMathSciNetGoogle Scholar
  24. 24.
    Verschelde, J.: Algorithm 795: PHCpack: a general-purpose solver for polynomial systems by homotopy continuation. ACM Trans. Math. Softw. (TOMS) 25(2), 251–276 (1999)CrossRefzbMATHGoogle Scholar
  25. 25.
    Watanabe, S.: Algebraic analysis for singular statistical estimation. In: Watanabe, O., Yokomori, T. (eds.) Algorithmic Learning Theory. Springer, Berlin (1999)Google Scholar
  26. 26.
    Weyl, H.: On the volume of tubes. Am. J. Math. 61, 461–472 (1939)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.The Institute of Statistical MathematicsTachikawa, TokyoJapan
  2. 2.London School of EconomicsLondonUK

Personalised recommendations