An Attribute-Specific Item Discrimination Index in Cognitive Diagnosis
There lacks an item quality index as a measure of item’s correct classification rates of attributes. The purpose of this study is to propose an attribute-specific item discrimination index as a measure of correct classification rate of attributes based on a q-vector, item parameters, and the distribution of attribute patterns. First, an attribute-specific item discrimination index was introduced. Second, a heuristic method was presented using the new index for test construction. The first simulation results showed that the new index performed well in that their values matched closely with the simulated correct classification rates of attributes across different conditions. The second simulation study results showed that the heuristic method based on the sum of the attributes’ indices yielded comparable performance to the famous CDI. The new index provides test developers with a useful tool to evaluate the quality of diagnostic items. It will be valuable to explore the applications and advantages of using the new index for developing an item selection algorithm or a termination rule in cognitive diagnostic computerized adaptive testing.
KeywordsCognitive diagnosis Item discrimination index Correct classification rate Test construction The deterministic inputs Noisy “and” gate model
This research was supported by the Key Project of National Education Science “Twelfth Five Year Plan” of Ministry of Education of China (Grant No. DHA150285). The authors would like to thank the editor Steve Culpepper for reviewing an earlier version of this work.
- Rupp, A. A., Templin, J. L., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York: The Guilford Press.Google Scholar
- von Davier, M. (2005). A general diagnostic model applied to language testing data (ETS RR-05–16). Princeton, NJ: Educational Testing Service.Google Scholar
- Wang, W. Y., Song, L. H., & Ding, S. L. (2018). An item discrimination index and its application in cognitive diagnostic assessment on a classification oriented view. Journal of Psychological Science, 41(2), 475–483. Google Scholar
- Wang, W. Y., Song, L. H., Chen, P., & Ding, S. L. (2019). An item-level expected classification accuracy and its applications in cognitive diagnostic assessment. Journal of Educational Measurement, 56(1), 51–75.Google Scholar