Advertisement

Investigation of the Item Selection Methods in Variable-Length CD-CAT

  • Ya-Hui SuEmail author
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 265)

Abstract

Cognitive diagnostic computerized adaptive testing (CD-CAT) provides useful cognitive diagnostic information for assessment and evaluation. At present, there are only a limited numbers of previous studies investigating how to optimally assemble cognitive diagnostic tests. The cognitive discrimination index (CDI) and attribute-level discrimination index (ADI) are commonly used to select items for cognitive diagnostic tests. The CDI measures an item’s overall discrimination power, and the ADI measures an item’s discrimination power for a specific attribute. Su (Quantitative psychology research. Springer, Switzerland, pp. 41–53, 2018) integrated the constraint-weighted procedure with the posterior-weighted CDI and ADI for item selection in fixed-length CD-CAT, and found examinees yielded different precision. In reality, if the same precision of test results is required for all the examinees, some examinees need to take more items and some need to take fewer items than others do. To achieve the same precision for examinees, this study investigated the performance of the constraint-weighted procedure with the posterior-weighted CDI and ADI for item selection in variable-length CD-CAT through simulations.

Keywords

Cognitive diagnostic computerized adaptive testing Item selection Constraint-weighted procedure Variable-length 

References

  1. Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74, 619–632.MathSciNetCrossRefGoogle Scholar
  2. Cheng, Y., & Chang, H.-H. (2009). The maximum priority index method for severely constrained item selection in computerized adaptive testing. British Journal of Mathematical and Statistical Psychology, 62, 369–383.MathSciNetCrossRefGoogle Scholar
  3. Cheng, Y., Chang, H.-H., Douglas, J., & Guo, F. (2009). Constraint-weighted a-stratification for computerized adaptive testing with nonstatistical constraints: Balancing measurement efficiency and exposure control. Educational and Psychological Measurement, 69, 35–49.MathSciNetCrossRefGoogle Scholar
  4. Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301–321.CrossRefGoogle Scholar
  5. Hartz, S. M. C. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality., (Unpublished doctoral dissertation) Champaign, IL: University of Illinois at Urbana-Champaign.Google Scholar
  6. Henson, R. A., & Douglas, J. (2005). Test construction for cognitive diagnostics. Applied Psychological Measurement, 29, 262–277.MathSciNetCrossRefGoogle Scholar
  7. Henson, R. A., Roussos, L., Douglas, J., & He, X. (2008). Cognitive diagnostic attribute-level discrimination indices. Applied Psychological Measurement, 32, 275–288.MathSciNetCrossRefGoogle Scholar
  8. Huebner, A. (2010). An overview of recent developments in cognitive diagnostic computer adaptive assessments. Practical Assessment, Research & Evaluation, 15(3), 1–7.Google Scholar
  9. Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.MathSciNetCrossRefGoogle Scholar
  10. Kuo, B.-C., Pai, H.-S., & de la Torre, J. (2016). Modified cognitive diagnostic index and modified attribute-level discrimination index for test construction. Applied Psychological Measurement, 40, 315–330.CrossRefGoogle Scholar
  11. Mislevy, R., Almond, R., Yan, D., & Steinberg, L. (2000). Bayes nets in educational assessment: Where do the numbers come from?. Princeton, NJ: CRESST/Educational Testing Service.Google Scholar
  12. Rupp, A. A., Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York, NY: The Guilford Press.Google Scholar
  13. Stocking, M. L., & Swanson, L. (1993). A method for severely constrained item selection in adaptive testing. Applied Psychological Measurement, 17, 277–292.CrossRefGoogle Scholar
  14. Su, Y.-H. (2018). Investigating the constrained-weighted item selection methods for CD-CAT. In M. Wiberg, S. Culpepper, R. Janssen, J. González, & D. Molenaar (Eds.), Quantitative Psychology Research (Vol. 233, pp. 41–53). Switzerland: Springer.  https://doi.org/10.1007/978-3-319-77249-3_4.Google Scholar
  15. Swanson, L., & Stocking, M. L. (1993). A model and heuristic for solving very large item selection problems. Applied Psychological Measurement, 17, 151–166.CrossRefGoogle Scholar
  16. Tatsuoka, C., & Ferguson, T. (2003). Sequential classification on partially ordered sets. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 65, 143–157.MathSciNetCrossRefGoogle Scholar
  17. Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.CrossRefGoogle Scholar
  18. Wang, C. (2013). Mutual information item selection method in cognitive diagnostic computerized adaptive testing with short test length. Educational and Psychological Measurement, 73, 1017–1035.CrossRefGoogle Scholar
  19. Wang, C., Chang, H. H., & Douglas, J. (2012). Combining CAT with cognitive diagnosis: A weighted item selection approach. Behavior Research Methods, 44, 95–109.CrossRefGoogle Scholar
  20. Zheng, C., & Chang, H. (2016). High-efficiency response distribution–based item selection algorithms for short-length cognitive diagnostic computerized adaptive testing. Applied Psychological Measurement, 40, 608–624.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of PsychologyNational Chung Cheng UniversityChiayiTaiwan

Personalised recommendations