Abstract
Consider an array of ratings or rank orders provided by M judges or respondents on n subjects or products. We propose a new general measure of concordance (agreement) in this context. This measure when applied to rank order data reduces to Kendall’s (1948) measure of concordance. A new measure for rank order data that depends on the average Kendall’s measure of rank correlation is also proposed and is also a special case of our general measure. This particular rank order measure can be considered as an alternative to the well-known Friedman’s statistic for the two-way analysis of variance. The general measure is also compared with another measure proposed by Lin (1989). Relations to the intraclass correlation coefficient of the ratings data are pointed out. Some distributional results are also presented. The proposed concordance measures provide the basis for testing the agreement between two or more methods, instruments or respondents in biometric research or market research surveys.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Choudhary, P. K. and Nagaraja, H. N. (2005). Measuring Agreement in Method Comparison Studies — A Review, In Advances in Ranking and Selection, Multiple Comparisons, and Reliability (Eds., N. Balakrishnan, N. Kannan and H. N. Nagaraja), pp. 215–244, Birkhäuser, Boston.
Cochran, W. G. (1977) Sampling Techniques, Third Edition, John Wiley & Sons, New York.
Daniels, H. E. (1944). The relation between measures of correlation in the universe of sample permutations, Biometrika, 33, 129–135.
Ehrenberg, A. S. C. (1952). On Sampling from a Population of Rankers, Biometrika 39, 82–87.
Fisher, R. A. (1921). On the “probable error” of a coefficient of correlation deduced from a small sample, Metron 1, 3–32 (Also reprinted in Collected Papers of R.A. Fisher, 1 J.H. Bennett (Ed.), (1971) University of Adelaide).
Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance, Journal of the American Statistical Association, 32, 675–701.
Kendall, M. G. (1948). Rank Correlation Methods, Charles Griffin, London.
Kendall, M. G. and Smith, B. B. (1939). The problem of m rankings, Annals of Mathematical Statistics, 10, 275–287.
Kendall, M. G. and Smith, B. B. (1940). On the Method of Paired Comparisons, Biometrika, 31, 324–425.
Lin, L. I. (1989). A concordance correlation coefficient to evaluate reproducibility, Biometrics, 45, 255–268. Corrections: 2000, 56, 324–325.
Lin, L. I. (1992). Assay validation using the concordance correlation coefficient, Biometrics, 48, 599–604.
Lin, L. I., Hedayat, A. S., Sinha, B. and Yang, M. (2002). Statistical methods in assessing agreement: Models, issues, and tools, Journal of the American Statistical Association, 97, 257–270.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Birkhäuser Boston
About this chapter
Cite this chapter
Raghavachari, M. (2005). Measures of Concordance for Assessing Agreement in Ratings and Rank Order Data. In: Balakrishnan, N., Nagaraja, H.N., Kannan, N. (eds) Advances in Ranking and Selection, Multiple Comparisons, and Reliability. Statistics for Industry and Technology. Birkhäuser Boston. https://doi.org/10.1007/0-8176-4422-9_14
Download citation
DOI: https://doi.org/10.1007/0-8176-4422-9_14
Publisher Name: Birkhäuser Boston
Print ISBN: 978-0-8176-3232-8
Online ISBN: 978-0-8176-4422-2
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)