Abstract
This paper discusses the use of optimal scores as an alternative to sum scores and expected sum scores when analyzing test data. Optimal scores are built on nonparametric methods and use the interaction between the test takers’ responses on each item and the impact of the corresponding items on the estimate of their performance. Both theoretical arguments for optimal score as well as arguments built upon simulation results are given. The paper claims that in order to achieve the same accuracy in terms of mean squared error and root mean squared error, an optimally scored test needs substantially fewer items than a sum scored test. The top-performing test takers and the bottom 5% test takers are by far the groups that benefit most from using optimal scores.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental tests (pp. 395–479). Reading, MA: Addison-Wesley.
Bränberg, K., & Wiberg, M. (2011). Observed score linear equating with covariates. Journal of Educational Measurement, 48, 419–440.
Lee, Y.-S. (2007). A comparison of methods for nonparametric estimation of item characteristic curves for binary items. Applied Psychological Measurement, 37, 121–134.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
Mokken, R. J. (1997). Nonparametric models for dichotomous responses. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 351–367). New York, NY: Springer.
Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56, 611–630.
Ramsay, J. O. (1997). A functional approach to modeling test data. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 351–367). New York, NY: Springer.
Ramsay, J. O. (2000). TestGraf: A program for the graphical analysis of multiple choice test and questionnaire data. [Computer software and manual]. Department of Psychology, McGill University, Montreal Canada.
Ramsay, J. O., Hooker, G., & Graves, S. (2009). Functional data analysis in R and Matlab. New York, NY: Springer.
Ramsay, J. O., & Silverman, B. W. (2002). Functional models for test items. In Applied functional data analysis. New York, NY: Springer.
Ramsay, J. O., & Silverman, B. W. (2005). Functional Data Analysis. New York: Springer.
Ramsay, J. O., & Wiberg, M. (2017a). A strategy for replacing sum scores. Journal of Educational and Behavioral Statistics, 42, 282–307.
Ramsay, J. O., & Wiberg, M. (2017b). Breaking through the sum score barrier. In Paper presented at the International Meeting of the Psychometric Society, July 11–15, Asheville, NC (pp. 151–158).
Rossi, N., Wang, X., & Ramsay, J. O. (2002). Nonparametric item response function estimates with the EM algorithm. Journal of the Behavioral and Educational Sciences, 27, 291–317.
Wallin, G., & Wiberg, M. (2017). Non-equivalent groups with covariates design using propensity scores for kernel equating. In L. A. van der Ark, M. Wiberg, S. A. Culpepper, J. A. Douglas, & W.-C. Wang. (Eds.), Quantitative psychology—81st annual meeting of the psychometric society, Asheville, North Carolina, 2016 (pp. 309–320). New York: Springer.
Wiberg, M., & Bränberg, K. (2015). Kernel equating under the non-equivalent groups with covariates design. Applied Psychological Measurement, 39, 349–361.
Woods, C. M. (2006). Ramsay-curve item response theory (RC-IRT) to detect and correct for nonnormal latent variables. Psychological Methods, 11, 253–270.
Woods, C. M., & Thissen, D. (2006). Item response theory with estimation of the latent population distribution using spline-based densities. Psychometrika, 71, 281–301.
Acknowledgements
This research was funded by the Swedish Research Council grant 2014-578.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Wiberg, M., Ramsay, J.O., Li, J. (2018). Optimal Scores as an Alternative to Sum Scores. In: Wiberg, M., Culpepper, S., Janssen, R., González, J., Molenaar, D. (eds) Quantitative Psychology. IMPS 2017. Springer Proceedings in Mathematics & Statistics, vol 233. Springer, Cham. https://doi.org/10.1007/978-3-319-77249-3_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-77249-3_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-77248-6
Online ISBN: 978-3-319-77249-3
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)