Advertisement

Exploring Joint Maximum Likelihood Estimation for Cognitive Diagnosis Models

  • Chia-Yi ChiuEmail author
  • Hans-Friedrich Köhn
  • Yi Zheng
  • Robert Henson
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 140)

Abstract

Current methods for fitting cognitive diagnosis models (CDMs) to educational data typically rely on expectation maximization (EM) or Markov chain Monte Carlo (MCMC) for estimating the item parameters and examinees’ proficiency class memberships. However, for advanced, more complex CDMs like the reduced reparameterized unified model (Reduced RUM) and the (saturated) loglinear cognitive diagnosis model (LCDM), EM and Markov chain Monte Carlo (MCMC) have the reputation of often consuming excessive CPU times. Joint maximum likelihood estimation (JMLE) is proposed as an alternative to EM and MCMC. The maximization of the joint likelihood is typically accomplished in a few iterations, thereby drastically reducing the CPU times usually needed for fitting advanced CDMs like the Reduced RUM or the (saturated) LCDM. As another attractive feature, the JMLE algorithm presented here resolves the traditional issue of JMLE estimators—their lack of statistical consistency—by using an external, statistically consistent estimator to obtain initial estimates of examinees’ class memberships as starting values. It can be proven that under this condition the JMLE item parameter estimators are also statistically consistent. The computational performance of the proposed JMLE algorithm is evaluated in two comprehensive simulation studies.

Keywords

Cognitive diagnosis Joint maximum likelihood estimation Nonparametric classification Consistency 

References

  1. Baker, F. B., & Kim, S.-H. (2004). Item response theory: Parameter estimation techniques (2nd ed.). New York: Marcel Dekker.Google Scholar
  2. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Load & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 397–479). Reading, MA: Addison-Wesley.Google Scholar
  3. Chiu, C.-Y., & Douglas, J. A. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response profiles. Journal of Classification, 30, 225–250.MathSciNetCrossRefGoogle Scholar
  4. Chiu, C.-Y., Douglas, J. A., & Li, X. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74, 633–665.MathSciNetCrossRefzbMATHGoogle Scholar
  5. Chiu, C.-Y., Zheng, Y., & Henson, R. (2013) Joint maximum likelihood estimation for cognitive diagnostic models in conjunction with proximity to ideal response patterns. Psychometrika (Manuscript under revision).Google Scholar
  6. de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199.MathSciNetCrossRefzbMATHGoogle Scholar
  7. DiBello, L. V., Roussos, L. A., & Stout, W. F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics: Psychometrics (Vol. 26, pp. 979–1030). Amsterdam: Elsevier.CrossRefGoogle Scholar
  8. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.Google Scholar
  9. Feng, Y., Habing, B. T., & Huebner, A. (2014). Parameter estimation of the reduced RUM using the EM algorithm. Applied Psychological Measurement, 38, 137–150.CrossRefGoogle Scholar
  10. Haberman, S. J., & von Davier, M. (2007). Some notes on models for cognitively based skill diagnosis. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics: Psychometrics (Vol. 26, pp. 1031–1038). Amsterdam: Elsevier.CrossRefGoogle Scholar
  11. Hartz, S. M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. (Doctoral dissertation) Available from ProQuest Dissertations and Theses database (UMI No. 3044108).Google Scholar
  12. Hartz, S. M., & Roussos, L. A. (2008, October). The fusion model for skill diagnosis: Blending theory with practicality (Research report No. RR-08-71). Princeton, NJ: Educational Testing Service.Google Scholar
  13. Henson, R., & Douglas, J. (2005). Test construction for cognitive diagnosis. Applied Psychological Measurement, 29, 262–277.MathSciNetCrossRefGoogle Scholar
  14. Henson, R., Roussos, L. A., Douglas, J., & He, X. (2008). Cognitive diagnostic attribute-level discrimination indices. Applied Psychological Measurement, 32, 275–288.MathSciNetCrossRefGoogle Scholar
  15. Henson, R., Templin, J. L., & Douglas, J. (2007). Using efficient model based sum-scores for conducting skills diagnoses. Journal of Educational Measurement, 44, 361–376.CrossRefGoogle Scholar
  16. Henson, R. A., & Templin, J. (2007, April). Large-scale language assessment using cognitive diagnosis models. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Chicago, IL.Google Scholar
  17. Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.MathSciNetCrossRefzbMATHGoogle Scholar
  18. Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.MathSciNetCrossRefGoogle Scholar
  19. Kim, Y.-H. (2011). Diagnosing EAP writing ability using the reduced reparameterized unified model. Language Testing, 28, 509–541.CrossRefGoogle Scholar
  20. Leighton, J., & Gierl, M. (2007) Cognitive diagnostic assessment for education: Theory and applications. Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
  21. Liu, Y., Douglas, J. A., & Henson, R. A. (2009). Testing person fit in cognitive diagnosis. Applied Psychological Measurement, 33, 579–598.CrossRefGoogle Scholar
  22. Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. (2009). The BUGS project: Evolution, critique, and future directions. Statistics in Medicine, 28, 3049–3067.MathSciNetCrossRefGoogle Scholar
  23. Macready, G. B., & Dayton, C. M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 33, 379–416.Google Scholar
  24. Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 187–212.MathSciNetCrossRefzbMATHGoogle Scholar
  25. Neyman, J., & Scott, E. L. (1948). Consistent estimates based on partially consistent observations. Econometrica, 16, 1–32.MathSciNetCrossRefGoogle Scholar
  26. Robitzsch, A., Kiefer, T., George, A. C., & Uenlue, A. (2014). CDM: Cognitive diagnosis modeling. R package version 3.1-14. Retrieved from the Comprehensive R Archive Network [CRAN] website http://CRAN.R-project.org/package=CDM
  27. Rupp, A. A., & Templin, J. L., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York: Guilford.Google Scholar
  28. Tatsuoka, K. K. (1985). A probabilistic model for diagnosing misconception in the pattern classification approach. Journal of Educational Statistics, 12, 55–73.Google Scholar
  29. Templin, J., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79, 317–339.MathSciNetCrossRefzbMATHGoogle Scholar
  30. Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305.CrossRefGoogle Scholar
  31. Templin, J. L., Henson, R. A., Templin, S. E., & Roussos, L. A. (2008). Robustness of hierarchical modeling of skill association in cognitive diagnosis models. Applied Psychological Measurement, 32, 559–574.MathSciNetCrossRefGoogle Scholar
  32. von Davier, M. (2005, September). A general diagnostic model applied to language testing data (Research report No. RR-05-16). Princeton, NJ: Educational Testing Service.Google Scholar
  33. von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287–301.MathSciNetCrossRefGoogle Scholar
  34. Wang, S., & Douglas, J. (2015). Consistency of nonparametric classification in cognitive diagnosis. Psychometrika, 80, 85–100.MathSciNetCrossRefGoogle Scholar
  35. Zheng, Y., & Chiu, C.-Y. (2014). NPCD: Nonparametric methods for cognitive diagnosis. R package version 1.0-5. Retrieved from the Comprehensive R Archive Network [CRAN] website http://CRAN.R-project.org/package=NPCD

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Chia-Yi Chiu
    • 1
    Email author
  • Hans-Friedrich Köhn
    • 2
  • Yi Zheng
    • 3
  • Robert Henson
    • 4
  1. 1.Rutgers, The State University of New JerseyNew BrunswickUSA
  2. 2.University of Illinois at Urbana-ChampaignChampaignUSA
  3. 3.Arizona State UniversityTempeUSA
  4. 4.University of North CarolinaGreensboroUSA

Personalised recommendations