Skip to main content
SpringerLink
Log in

Investigating the Constrained-Weighted Item Selection Methods for CD-CAT

  • Conference paper
  • First Online:
Quantitative Psychology (IMPS 2017)

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 233))

Included in the following conference series:

Abstract

Cognitive diagnostic computerized adaptive testing (CD-CAT) not only provides useful cognitive diagnostic information measured in psychological or educational assessments, but also obtains great efficiency brought by computerized adaptive testing. At present, there are only a limited numbers of previous studies examining how to optimally construct cognitive diagnostic tests. The cognitive diagnostic discrimination index (CDI) and attribute-level discrimination index (ADI) have been proposed for item selection in cognitive diagnostic tests. Zheng and Chang (Appl Psychol Measure 40:608–624, 2016) proposed the modified version of these two indices, an extension of the Kullback-Leibler (KL) and posterior-weighted KL (PWKL) methods, and suggested that they could be integrated with the constraint management procedure for item selection in CD-CAT. However, the constraint management procedure hasn’t been investigated in CD-CAT yet. Therefore, the aim of this study is two fold (a) to integrate the indices with the constraint management procedure for item selection, and (b) to investigate the efficiency of these item selection methods in CA-CAT. It was found that the constraint-weighted indices performed much better than those without constraint-weighted procedure in terms of constraint management and exposure control while maintaining similar measurement precision.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Chang, H.-H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213–229.

    Article  Google Scholar 

  • Chang, H.-H., & Ying, Z. (1999). a-stratified multistage computerized adaptive testing. Applied Psychological Measurement, 23, 211–222.

    Article  Google Scholar 

  • Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74, 619–632.

    Article  MathSciNet  Google Scholar 

  • Cheng, Y., & Chang, H.-H. (2009). The maximum priority index method for severely constrained item selection in computerized adaptive testing. British Journal of Mathematical and Statistical Psychology, 62, 369–383.

    Article  MathSciNet  Google Scholar 

  • Cheng, Y., Chang, H.-H., Douglas, J., & Guo, F. (2009). Constraint-weighted a-stratification for computerized adaptive testing with nonstatistical constraints: Balancing measurement efficiency and exposure control. Educational and Psychological Measurement, 69, 35–49.

    Article  MathSciNet  Google Scholar 

  • Cheng, Y., Chang, H.-H., & Yi, Q. (2007). Two-phase item selection procedure for flexible content balancing in CAT. Applied Psychological Measurement, 31, 467–482.

    Article  MathSciNet  Google Scholar 

  • Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301–321.

    Article  Google Scholar 

  • Hartz, S. M. C. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, Champaign, IL.

    Google Scholar 

  • Henson, R. A., & Douglas, J. (2005). Test construction for cognitive diagnostics. Applied Psychological Measurement, 29, 262–277.

    Article  MathSciNet  Google Scholar 

  • Henson, R. A., Roussos, L., Douglas, J., & He, X. (2008). Cognitive diagnostic attribute-level discrimination indices. Applied Psychological Measurement, 32, 275–288.

    Article  MathSciNet  Google Scholar 

  • Huebner, A. (2010). An overview of recent developments in cognitive diagnostic computer adaptive assessments. Practical Assessment, Research and Evaluation, 15(3), 1–7.

    Google Scholar 

  • Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.

    Article  MathSciNet  Google Scholar 

  • Kuo, B.-C., Pai, H.-S., & de la Torre, J. (2016). Modified cognitive diagnostic index and modified attribute-level discrimination index for test construction. Applied Psychological Measurement, 40, 315–330.

    Article  Google Scholar 

  • Mislevy, R., Almond, R., Yan, D., & Steinberg, L. (2000). Bayes nets in educational assessment: Where do the numbers come from? Princeton, NJ: CRESST/Educational Testing Service.

    Google Scholar 

  • Rupp, A. A., Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York, NY: The Guilford Press.

    Google Scholar 

  • Su, Y.-H. (2015). The performance of the modified multidimensional priority index for item selection in variable-length MCAT. In L. A. van der Ark, D. M. Bolt, W.-C. Wang, J. A. Douglas, & S.-M. Chow (Eds.), Quantitative psychology research (Vol. 140, pp. 89–97). Switzerland: Springer.

    Chapter  Google Scholar 

  • Su, Y.-H. (2016). A comparison of constrained item selection methods in multidimensional computerized adaptive testing. Applied Psychological Measurement, 40(5), 346–360.

    Article  Google Scholar 

  • Su, Y.-H., & Huang, Y.-L. (2015). Using a modified multidimensional priority index for item selection under within-item multidimensional computerized adaptive testing. In R. E. Millsap, D. M. Bolt, L. A. van der Ark, & W.-C. Wang (Eds.), Quantitative Psychology Research (Vol. 89, pp. 227–242). Switzerland: Springer.

    Google Scholar 

  • Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.

    Article  Google Scholar 

  • Yao, L. (2011, October). Multidimensional CAT item selection procedures with item exposure control and content constraints. Paper presented at the 2011 International Association of Computer Adaptive Testing (IACAT) Conference. Pacific Grove, CA.

    Google Scholar 

  • Yao, L. (2012). Multidimensional CAT item selection methods for domain scores and composite scores: Theory and Applications. Psychometrika, 77, 495–523.

    Article  MathSciNet  Google Scholar 

  • Yao, L. (2013). Comparing the performance of five multidimensional CAT selection procedures with different stopping rules. Applied Psychological Measurement, 37, 3–23.

    Article  Google Scholar 

  • Zheng, C., & Chang, H.-H. (2016). High-efficiency response distribution–based item selection algorithms for short-length cognitive diagnostic computerized adaptive testing. Applied Psychological Measurement, 40, 608–624.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, National Chung Cheng University, Taiwan, 168 University Rd., Minhsiung Township, Chiayi County, 62102, Taiwan

    Ya-Hui Su

Authors
  1. Ya-Hui Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ya-Hui Su .

Editor information

Editors and Affiliations

  1. Umeå School of Business, Economics and Statistics, Umeå University, Umeå, Sweden

    Marie Wiberg

  2. Department of Statistics, University of Illinois at Urbana-Champaign, Champaign, Illinois, USA

    Steven Culpepper

  3. Faculty of Psychology and Educational Sciences, KU Leuven, Leuven, Belgium

    Rianne Janssen

  4. Faculty of Mathematics, Pontificia Universidad Católica de Chile, Santiago, Chile

    Jorge González

  5. Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands

    Dylan Molenaar

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Su, YH. (2018). Investigating the Constrained-Weighted Item Selection Methods for CD-CAT. In: Wiberg, M., Culpepper, S., Janssen, R., González, J., Molenaar, D. (eds) Quantitative Psychology. IMPS 2017. Springer Proceedings in Mathematics & Statistics, vol 233. Springer, Cham. https://doi.org/10.1007/978-3-319-77249-3_4

Download citation

Publish with us

Policies and ethics

Search

Navigation