Advertisement

Consistency Theory for the General Nonparametric Classification Method

  • Chia-Yi ChiuEmail author
  • Hans-Friedrich Köhn
Article
  • 26 Downloads

Abstract

Parametric likelihood estimation is the prevailing method for fitting cognitive diagnosis models—also called diagnostic classification models (DCMs). Nonparametric concepts and methods that do not rely on a parametric statistical model have been proposed for cognitive diagnosis. These methods are particularly useful when sample sizes are small. The general nonparametric classification (GNPC) method for assigning examinees to proficiency classes can accommodate assessment data conforming to any diagnostic classification model that describes the probability of a correct item response as an increasing function of the number of required attributes mastered by an examinee (known as the “monotonicity assumption”). Hence, the GNPC method can be used with any model that can be represented as a general DCM. However, the statistical properties of the estimator of examinees’ proficiency class are currently unknown. In this article, the consistency theory of the GNPC proficiency-class estimator is developed and its statistical consistency is proven.

Keywords

cognitive diagnosis Q-matrix DINA model DINO model general DCM G-DINA model nonparametric classification general nonparametric classification method 

Notes

Acknowledgements

Funding was provided by National Science Foundation (Grant No. 1552563).

References

  1. Ayers, E., Nugent, R., & Dean, N. (2008). Skill set profile clustering based on student capability vectors computed from online tutoring data. In R. S. J. de Baker, T. Barnes, & J. E. Beck (Eds.), Educational data mining 2008: Proceedings of the 1st international conference on educational data mining, Montreal, Quebec, Canada (pp. 210–217). Retrieved from http://www.educationaldatamining.org/EDM2008/uploads/proc/full%20proceedings.pdf.
  2. Chiu, C.-Y. (2008). Cluster analysis for cognitive diagnosis: Theory and applications (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3337778).Google Scholar
  3. Chiu, C.-Y., & Douglas, J. A. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response profiles. Journal of Classification, 30, 225–250.CrossRefGoogle Scholar
  4. Chiu, C.-Y., Douglas, J. A., & Li, X. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74, 633–665.CrossRefGoogle Scholar
  5. Chiu, C.-Y., Sun, Y., & Bian, Y. (2018). Cognitive diagnosis for small educational programs: The general nonparametric classification method. Psychometrika, 83, 355–375.CrossRefGoogle Scholar
  6. de la Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34, 115–130.CrossRefGoogle Scholar
  7. de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199.CrossRefGoogle Scholar
  8. de la Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353.CrossRefGoogle Scholar
  9. DiBello, L. V., Roussos, L. A., & Stout, W. F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics: Volume 26. Psychometrics (pp. 979–1030). Amsterdam: Elsevier.Google Scholar
  10. Haberman, S. J., & von Davier, M. (2007). Some notes on models for cognitively based skill diagnosis. In C. R. Rao & S. Sinharay (Eds.), Handbook of Statistics: Volume 26. Psychometrics (pp. 1031–1038). Amsterdam: Elsevier.Google Scholar
  11. Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 333–352.CrossRefGoogle Scholar
  12. Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.CrossRefGoogle Scholar
  13. Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301), 13–30.CrossRefGoogle Scholar
  14. Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.CrossRefGoogle Scholar
  15. Köhn, H.-F., & Chiu, C.-Y. (2016a). Conditions of completeness of the Q-matrix of tests for cognitive diagnosis. In A. L. van der Ark, D. M. Bolt, W.-C. Wang, J. A. Douglas, & M. Wiberg (Eds.), Quantitative psychology research: The 80th annual meeting of the psychometric society, Beijing, 2015 (pp. 255–264). New York: Springer.Google Scholar
  16. Köhn, H.-F., & Chiu, C.-Y. (2016b). A proof of the duality of the DINA model and the DINO model. Journal of Classification, 33, 171–184.CrossRefGoogle Scholar
  17. Köhn, H.-F., & Chiu, C.-Y. (2017). A procedure for assessing completeness of the Q-matrix of cognitive diagnosis models. Psychometrika, 82, 112–132.CrossRefGoogle Scholar
  18. Leighton, J., & Gierl, M. (2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  19. Ma, W., & de la Torre, J. (2019). GDINA: The generalized DINA model framework. R package version 2.3.2. Retrieved from the Comprehensive R Archive Network [CRAN] website https://cran.r-project.org/web/packages/GDINA/.
  20. Macready, G. B., & Dayton, C. M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 33, 379–416.Google Scholar
  21. Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 187–212.CrossRefGoogle Scholar
  22. Nichols, P. D., Chipman, S. E., & Brennan, R. L. (1995). Cognitively diagnostic assessment. Mahwah, NJ: Erlbaum.Google Scholar
  23. Park, Y. S., & Lee, Y.-S. (2011). Diagnostic cluster analysis of mathematics skills. IERI Monograph Series: Issues and Methodologies in Large-Scale Assessments, 4, 75–107.Google Scholar
  24. Robitzsch, A., Kiefer, T., George, A. C., & Uenlue, A. (2018). CDM: Cognitive diagnosis modeling. R package version 7.1-20. Retrieved from the Comprehensive R Archive Network [CRAN] website https://cran.r-project.org/web/packages/CDM/.
  25. Rupp, A. A., Templin, J. L., & Henson, R. A. (2010). Diagnostic Measurement. Theory, Methods, and Applications. New York: Guilford.Google Scholar
  26. Stout, W. (2002). Psychometrics: From practice to theory and back. Psychometrika, 67, 485–518.CrossRefGoogle Scholar
  27. Tatsuoka, K. K. (1985). A probabilistic model for diagnosing misconception in the pattern classification approach. Journal of Educational and Behavioral Statistics, 12, 55–73.CrossRefGoogle Scholar
  28. Tatsuoka, K. K. (2009). Cognitive assessment. An introduction to the rule space method. New York: Routledge/Taylor & Francis.Google Scholar
  29. Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305.CrossRefGoogle Scholar
  30. von Davier, M. (2005). A general diagnostic model applied to language testing data (Research report No. RR-05-16). Princeton, NJ: Educational Testing Service.Google Scholar
  31. von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287–301.CrossRefGoogle Scholar
  32. Wang, S., & Douglas, J. (2015). Consistency of nonparametric classification in cognitive diagnosis. Psychometrika, 80, 85–100.CrossRefGoogle Scholar
  33. Willse, J., Henson, R., & Templin, J. (2007). Using sum scores or IRT in place of cognitive diagnosis models: Can existing or more familiar models do the job? Paper presented at the Annual Meeting of the National Council on Measurement in Education, Chicago, IL.Google Scholar
  34. Zheng, Y., & Chiu, C.-Y. (2014). NPCD: Nonparametric methods for cognitive diagnosis. R package version 1.0-5. Retrieved from the Comprehensive R Archive Network [CRAN] website http://CRAN.R-project.org/package=NPCD.

Copyright information

© The Psychometric Society 2019

Authors and Affiliations

  1. 1.Rutgers, The State University of New JerseyNew BrunswickUSA
  2. 2.University of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations