Skip to main content
SpringerLink
Log in

Algebraic exact inference for rater agreement models

  • Statistical Methods
  • Published:
Statistical Methods and Applications Aims and scope Submit manuscript

Abstract

In recent years, a method for sampling from conditional distributions for categorical data has been presented by Diaconis and Sturmfels. Their algorithm is based on the algebraic theory of toric ideals which are used to create so called “Markov Bases”. The Diaconis-Sturmfels algorithm leads to a non-asymptotic Monte Carlo Markov Chain algorithm for exact inference on some classes of models, such as log-linear models. In this paper we apply the Diaconis-Sturmfels algorithm to a set of models arising from the rater agreement problem with special attention to the multi-rater case. The relevant Markov bases are explicitly computed and some results for simplify the computation are presented. An extended example on a real data set shows the wide applicability of this methodology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Agresti A (1992) Modelling patterns of agreement and disagreement. Statistical Methods in Medical Research 1: 201–218

    Google Scholar 

  • Agresti A (2001) Exact inference for categorical data: Recent advances and continuing controversies. Statist. Med. 20: 2709–2722

    Article  Google Scholar 

  • Agresti A (2002) Categorical Data Analysis, 2nd edn. Wiley, New York

    MATH  Google Scholar 

  • Becker MP (1990) Quasisymmetric models for the analysis of square contingency tables. J.R. Statist. Soc. 52(2): 369–378

    Google Scholar 

  • Bigatti A, La Scala R, Robbiano L (1999) Computing toric ideals. J. Symb. Comput. 27: 351–365

    Article  MATH  Google Scholar 

  • Bigatti A, Robbiano L (2001) Toric ideals. Mat. Contemp. 21: 1–25

    MATH  MathSciNet  Google Scholar 

  • Bishop YM, Fienberg S, Holland PW (1975) Discrete multivariate analysis: theory and practice. MIT Press, Cambridge

    MATH  Google Scholar 

  • CDER (2000) Developing Medical Imaging Drugs and Biological products. Guidance for industry. U.S. Department of Health and Human Services, Food and Drug Administration

  • Chib S, Greenberg E (1995) Understanding the Metropolis-Hastings algorithm. Amer. Statist. 49(4): 327–335

    Article  Google Scholar 

  • CoCoATeam (2004) CoCoA: a system for doing Computations in Commutative Algebra, Available at http://cocoa.dima.unige.it

  • Cohen J (1960) A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20: 37–46

    Google Scholar 

  • Darroch JN, McCloud PI (1986) Category distinguishability and observer agreement. Australian Journal of Statistics 28(3): 371–388

    MATH  MathSciNet  Google Scholar 

  • Diaconis P, Sturmfels B (1998) Algebraic algorithms for sampling from conditional distributions. Ann. Statist. 26(1): 363–397

    Article  MATH  MathSciNet  Google Scholar 

  • Dillon WR, Mulani N (1984) A probabilistic latent class model for assessing inter-judge reliability. Multivariate Behavioral Research 19: 438–458

    Article  Google Scholar 

  • Dobra A (2003) Markov bases for decomposable graphical models. Bernoulli 9(6): 1–16

    MathSciNet  Google Scholar 

  • Dobra A, Sullivant S (2002) Markov bases for reducible sets of tables. Workshop on Multidimensional tables: Statistics, Combinatorial Optimization, and Gröbner bases, University of California, Davis, CA

  • Donner A, Klar N (1996) The statistical analysis of kappa statistics in multiple samples. Journal of Clinical Epidemiology 49: 1053–1058

    Article  Google Scholar 

  • Donner A, Shoukri MM, Klar N, Bartfay E (2000) Testing the equality of two dependent kappa statistics. Statist. Med. 19: 373–387.

    Article  Google Scholar 

  • Fienberg S (1980) The Analysis of Cross-Classified Categorical Data. MIT Press, Cambridge

    MATH  Google Scholar 

  • Fleiss JL (1981) Statistical Methods for Rates and Proportions, 2nd edn. Wiley, New York.

    MATH  Google Scholar 

  • Haberman SJ (1974) The Analysis of Frequency Data. The University of Chicago Press, Chicago and London

    MATH  Google Scholar 

  • Hanley JA (1998) Receiver Operating Characteristic (ROC) curves. In: Armitage P, Colton T (eds) Encyclopedia of Biostatistics, 3738–3745, Wiley, New York

    Google Scholar 

  • Klar N, Lipsitz SR, Ibrahim JG (2000) An estimating equation approach for modelling Kappa.Biom. J. 42: 45–58

    Article  MATH  Google Scholar 

  • Kreuzer M, Robbiano L (2000) Computational Commutative Algebra 1. Springer, New York

    Google Scholar 

  • Landis RJ, Koch GG (1975) A review of statistical methods in the analysis of data arising from observer reliability studies, Parts I and II. Statist. Neerlandica 29: 101–123, 151–161

    Article  MATH  MathSciNet  Google Scholar 

  • Pistone G, Riccomagno E, Wynn HP (2001) Algebraic Statistics: Computational Commutative Algebra in Statistics. Chapman&Hall/CRC, Boca Raton

    MATH  Google Scholar 

  • Rapallo F (2003) Algebraic Markov bases and MCMC for two-way contingency tables. Scand. J. Statist. 30(2): 385–397

    Article  MATH  MathSciNet  Google Scholar 

  • Rudas T, Bergsma WP (2002) On generalised symmetry. Technical report, Paul Sabbatier University, Toulouse, France, preprint

    Google Scholar 

  • Smith PWF, Forster JJ, McDonald JW (1996) Monte Carlo exact tests for square contingency tables. J. R. Statist. Soc. 159(2): 309–321

    Article  MathSciNet  Google Scholar 

  • Tanner MA, Young MA (1985) Modelling agreement among raters. J. Am. Statist. Assoc. 80: 175–180

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Genova, via Dodecaneso 35, 16146, Genova, Italy

    Fabio Rapallo

Authors
  1. Fabio Rapallo
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Partially supported by MIUR Cofin03 (G. Consonni) and by INdAM projectAlgebraic Statistics.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rapallo, F. Algebraic exact inference for rater agreement models. Statistical Methods & Applications 14, 45–66 (2005). https://doi.org/10.1007/BF02511574

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02511574

Key words

Search

Navigation