Abstract
This chapter gives a succinct introduction to classical test theory, an early attempt to formalize a statistical theory of psychological measurement. The main focus is on reliability. After introducing the true score model, the following reliability coefficients are presented: Cronbach’s α, greatest lower bound, and McDonald’s ω’s. In the second part of this chapter, this simple definition of reliability idea is extended to multiple error sources. This leads to generalizability theory which includes concepts like G-studies and D-studies, as well as generalizability and dependability coefficients.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This is based on an extension of the simple variance sum law: V ar(X + Y ) = V ar(X) + V ar(Y ) + 2Cov(X, Y ).
- 2.
Details on different types of ICCs can be found in Shrout and Fleiss (1979).
- 3.
At the point this book was written, the package did not provide options to specify different n’s explicitly.
References
Algina, J., & Penfield, R. D. (2009). Classical test theory. In: R. E. Millsap & A. Maydeu-Olivares (Eds.), The sage handbook of quantitative methods in psychology (pp. 93–122). Thousand Oaks: Sage.
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01.
Brennan, R. L. (2001). Generalizability theory. New York: Springer.
Cameletti, M., & Caviezel, V. (2012). CMC: Cronbach-Mesbah curve. R package version 1.0. https://CRAN.R-project.org/package=CMC
Coaley, K. (2014). An introduction to psychological assessment & psychometrics (2nd ed.). London: Sage.
Crocker, L. M., & Algina, J. (1986). Introduction to modern and classical test theory. Belmont: Wadsworth Group/Thomson Learning.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.
Cronbach, L. J. (1972). The dependability of behavioral measurements. New York: Wiley.
Diedenhofen, B. (2016). cocron: Statistical comparisons of two or more alpha coefficients. R package version 1.0-1. https://CRAN.R-project.org/package=cocron
Fletcher, T. D. (2010). psychometric: Applied psychometric theory. R package version 2.2. https://CRAN.R-project.org/package=psychometric
Graham, J. M. (2006). Congeneric and (essentially) tau-equivalent estimates of score reliability: What they are and how to use them. Educational and Psychological Measurement, 66, 930–944.
Hoyt, W. T., & Melby, J. N. (1999). Dependability of measurement in counseling psychology: An introduction to generalizability theory. The Counseling Psychologist, 27, 325–352.
Jackson, P., & Agunwamba, C. (1977). Lower bounds for the reliability of the total score on a test composed of nonhomogeneous items: I: Algebraic lower bounds. Psychometrika, 42, 567–578.
Lakes, K. D. (2012). The response to challenge scale (RCS): The development and construct validity of an observer-rated measure of children’s self-regulation. The International Journal of Educational and Psychological Assessment, 10, 83–96.
Lakes, K. D., & Hoyt, W. T. (2004). Promoting self-regulation through school-based martial arts training. Journal of Applied Developmental Psychology, 25, 283–302.
Lakes, K. D., & Hoyt, W. T. (2009). Applications of generalizability theory to clinical child and adolescent psychology research. Journal of Clinical Child & Adolescent Psychology, 38, 144–165.
Mair, P., Hofmann, E., Gruber, K., Zeileis, A., & Hornik, K. (2015). Motivation, values, and work design as drivers of participation in the R open source project for statistical computing. Proceedings of the National Academy of Sciences of the United States of America, 112, 14788–14792.
McDonald, R. P. (1999). Test theory: A unified treatment. Hillsdale: Erlbaum.
Moore, C. T. (2016). gtheory: Apply generalizability theory with R. R package version 0.1.2. https://CRAN.R-project.org/package=gtheory
Revelle, W. (2015). An introduction to psychometric theory with applications in R. Freely available online, http://www.personality-project.org/r/book/
Revelle, W. (2017). psych: Procedures for psychological, psychometric, and personality research. R package version 1.7.8. http://CRAN.R-project.org/package=psych
Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74, 145–154.
Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park: Sage.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.
Sijtmsa, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74, 107–120.
Streiner, D. L. (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80, 99–103.
Ten Berge, J. M. F., & Sočan, G. (2004). The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality. Psychometrika, 69, 613–625.
Willse, J. T. (2014). CTT: Classical test theory functions. R package version 2.1. https://CRAN.R-project.org/package=CTT
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Mair, P. (2018). Classical Test Theory. In: Modern Psychometrics with R. Use R!. Springer, Cham. https://doi.org/10.1007/978-3-319-93177-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-93177-7_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93175-3
Online ISBN: 978-3-319-93177-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)