Abstract
This paper investigates the estimation biases of five reliability indices, Cronbach’s α, Guttman’s λ2 and λ4, glb, and McDonald’s ω. The factors included are test dimensionality, population ability distribution, sample size, test length, and test discrimination. It was found that estimation biases of Alpha, λ4, and glb were correlated with the test’s true reliability, whereas λ2 and ω were not. Estimation biases were larger in two-dimensional tests than in unidimensional tests. Alpha overall had the largest estimation bias, but glb displayed similar bias in unidimensional tests. In light of the findings in the simulation study, we recommend McDonald’s ω because it had the smallest estimation bias in most test condition.
Paper presented at 80th annual meeting of Psychometric Society, July 2015.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
N. Bendermacher, Beyond alpha: lower bounds for the reliability of tests. J. Mod. Appl. Stat. Methods 9(1), ii–vi (2010). Article 11. Available at: http://digitalcommons.wayne.edu/jmasm/vol9/iss1/11
P.M. Bentler, A lower-bound method for the dimension-free measurement of internal consistency. Soc. Sci. Res. 1, 343–357 (1972)
P.M. Bentler, J.A. Woodward, Inequalities among lower bounds to reliability: with applications to test construction and factor analysis. Psychometrika 45, 249–267(1980)
T. Benton, An empirical assessment of Guttman’s Lambda 4 reliability coefficient, in Quantitative Psychology Research: The 78th Annual Meeting of the Psychometric Society, ed. by R.E. Millsap, D.M. Bolt, L.A. van der Ark, W.-C. Wang (Springer, New York, 2015), pp. 301–310
K. Sijtsma, On the use, the misuse, and the very limited usefulness of Cronbach’s Alpha. Psychometrika 74, 107–120 (2009)
L. Cronbach, Coefficient alpha and the internal structure of tests. Psychometrika 16, 297–334 (1951)
S.A. Green, Y. Yang, Commentary on coefficient alpha: a cautionary tale. Psychometrika 74, 121–135 (2009)
L. Guttman, A basis for analyzing test-retest reliability. Psychometrika 10, 255–282 (1945)
P. Jackson, C. Agunwamba, Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: I: algebraic lower bounds. Psychometrika 42, 567–578 (1977)
R.P. McDonald, The theoretical foundations of principal factor analysis, canonical factor analysis and alpha factor analysis. Br. J. Math. Psychol. 23, 1–21 (1970)
R.P. McDonald, Test Theory: A Unified Treatment (Erlbaum, Mahwah, NJ, 1999)
T. Raykov, Bias of coefficient α for fixed congeneric measures with correlated errors. Appl. Psychol. Meas. 25, 69–76 (2001)
W. Revelle, R.E. Zinbarg, Coefficients alpha, beta, omega and the glb: comments on Sijtsma. Psychometrika 74, 145–154 (2009)
W. Revelle, psych: Procedures for personality and psychological research. R package version 1.3.2 (2014). http://personality-project.org/r, http://personality-project.org/r/psych-manual.pdf
C. Spearman, The proof and measurement of association between two things. Am. J. Psychol. 15, 72–101 (1904)
W. Tang, Y. Cui, A simulation study for comparing three lower bounds to reliability. Paper presented at the AERA, Vancouver, British Columbia, 2012
J. Ten Berge, G. Socan, The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality. Psychometrika 69, 613–625 (2004)
B. Woodhouse, P. Jackson, Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: II: a search procedure to locate the greatest lower bound. Psychometrika 42, 579–591 (1977)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Sha, S., Ackerman, T. (2017). The Performance of Five Reliability Estimates in Multidimensional Test Situations. In: van der Ark, L.A., Wiberg, M., Culpepper, S.A., Douglas, J.A., Wang, WC. (eds) Quantitative Psychology. IMPS 2016. Springer Proceedings in Mathematics & Statistics, vol 196. Springer, Cham. https://doi.org/10.1007/978-3-319-56294-0_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-56294-0_16
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56293-3
Online ISBN: 978-3-319-56294-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)