Advertisement

Psychometrika

, Volume 84, Issue 1, pp 310–322 | Cite as

Optimal Scores: An Alternative to Parametric Item Response Theory and Sum Scores

  • Marie WibergEmail author
  • James O. Ramsay
  • Juan Li
Article
  • 188 Downloads

Abstract

The aim of this paper is to discuss nonparametric item response theory scores in terms of optimal scores as an alternative to parametric item response theory scores and sum scores. Optimal scores take advantage of the interaction between performance and item impact that is evident in most testing data. The theoretical arguments in favor of optimal scoring are supplemented with the results from simulation experiments, and the analysis of test data suggests that sum-scored tests would need to be longer than an optimally scored test in order to attain the same level of accuracy. Because optimal scoring is built on a nonparametric procedure, it also offers a flexible alternative for estimating item characteristic curves that can fit items that do not show good fit to item response theory models.

Notes

Acknowledgements

This research was funded by the Swedish Research Council Grant 2014-578.

Supplementary material

11336_2018_9639_MOESM1_ESM.zip (42 kb)
Supplementary material 1 (zip 42 KB)

References

  1. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental tests (pp. 395–479). Reading, MA: Addison-Wesley.Google Scholar
  2. Lee, Y.-S. (2007). A comparison of methods for nonparametric estimation of item characteristic curves for binary items. Applied Psychological Measurement, 37, 121–134.CrossRefGoogle Scholar
  3. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  4. Mokken, R. J. (1997). Nonparametric models for dichotomous responses. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 351–367). New York: Springer.CrossRefGoogle Scholar
  5. Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56, 611–630.CrossRefGoogle Scholar
  6. Ramsay, J. O. (1997). A functional approach to modeling test data. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 351–367). New York: Springer.Google Scholar
  7. Ramsay, J. O. (2000). TestGraf: A program for the graphical analysis of multiple choice test and questionnaire data [Computer software and manual]. Montreal: Department of Psychology, McGill University.Google Scholar
  8. Ramsay, J. O., Hooker, G., & Graves, S. (2009). Functional data analysis in R and Matlab. New York: Springer.CrossRefGoogle Scholar
  9. Ramsay, J. O., & Silverman, B. W. (2002). Functional models for test items. In J. O. Ramsay & B. W. Silverman Applied functional data analysis. New York: Springer-Verlag.Google Scholar
  10. Ramsay, J. O., & Silverman, B. W. (2005). Functional data analysis. New York: Springer.Google Scholar
  11. Ramsay, J. O., & Wiberg, M. (2017a). A strategy for replacing sum scores. Journal of Educational and Behavioral Statistics, 42(3), 282–307.CrossRefGoogle Scholar
  12. Ramsay, J. O., & Wiberg, M. (2017b). Breaking through the sum scoring barrier. In L. A. van der Ark, M. Wiberg, S. A. Culpepper, J. A. Douglas, & W.-C. Wang (Eds.), Quantitative psychology—81st annual meeting of the psychometric society, Asheville, North Carolina, 2016 (pp. 151–158). New York: Springer.Google Scholar
  13. Rossi, N., Wang, X., & Ramsay, J. O. (2002). Nonparametric item response function estimates with the EM algorithm. Journal of the Behavioral and Educational Sciences, 27, 291–317.Google Scholar
  14. Wiberg, M., & Bränberg, K. (2015). Kernel equating under the non-equivalent groups with covariates design. Applied Psychological Measurement, 39(5), 349–361.CrossRefGoogle Scholar
  15. Woods, C. M. (2006). Ramsay-curve item response theory (RC-IRT) to detect and correct for nonnormal latent variables. Psychological Methods, 11, 253–270.CrossRefGoogle Scholar
  16. Woods, C. M., & Thissen, D. (2006). Item response theory with estimation of the latent population distribution using spline-based densities. Psychometrika, 71, 281–301.CrossRefGoogle Scholar

Copyright information

© The Psychometric Society 2018

Authors and Affiliations

  1. 1.USBE, Department of StatisticsUmeå UniversityUmeåSweden
  2. 2.Department of PsychologyMcGill UniversityMontrealCanada
  3. 3.Department of Mathematics and StatisticsMcGill UniversityMontrealCanada

Personalised recommendations