Abstract
Much of the research and application in cognitive diagnostic assessments to date has been centered on a confirmatory approach where a Q-matrix is pre-identified using content experts’ opinion or test developers’ knowledge on test items. As opposed to the traditional methods, which require prior knowledge about latent dimensions and underlying structure of test items, the approaches described in this chapter attempt to identify a Q-matrix solely relying on the observed test response data and thus avoid probable decision error. There are several important aspects to consider when estimating a Q-matrix from the observed data. First, a fundamental question of identifiability arises, that is, whether and to what extent Q can be estimated from data. The second aspect to consider in learning Q concerns the computational intensity that arises from estimation. The third aspect pertains to the presence of missing data, more precisely, the latent attributes underlying the observed data. The completeness of a Q-matrix, the other important aspect to consider in identifying Q, is beyond the scope of the present chapter.
Jingchen Liu is supported in part by NSF SES-1323977, SES-1826540, IIS-1633360, and Army Research Office grant W911NF-15-1-0159.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, Y., Culpepper, S. A., Chen, Y., & Douglas, J. (2017). Bayesian estimation of the DINA Q matrix. Psychometrika. https://doi.org/10.1007/s11336-017-9579-4
Chen, Y., Liu, J., Xu, G., & Ying, Z. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110, 850–866.
Chiu, C.-Y. (2013). Statistical refinement of the Q-matrix in cognitive diagnosis. Applied Psychological Measurement, 37, 598–618.
Chiu, C.-Y., Douglas, J., & Li, X. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74, 633–665.
DeCarlo, L. T. (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model. Applied Psychological Measurement, 36, 447–468.
de la Torre, J. (2008). An empirically based method of Q-matrix validation for the DINA model: Development and applications. Journal of Educational Measurement, 45, 343–362.
de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199.
de la Torre, J., & Chiu, C.-Y. (2016). A general method of empirical Q-matrix validation. Psychometrika, 81, 253–273.
Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360. Retrieved from http://www.jstor.org/stable/3085904
Fang, G., Liu, J., & Ying, Z. (2017a). Latent variable selection via overlap group LASSO with applications to cognitive assessment. Preprint.
Fang, G., Liu, J., & Ying, Z. (2017b). On the identifiability of diagnostic classification models. ArXiv e-prints. Retrieved from https://arxiv.org/abs/1706.01240
Friedman, J., Hastie, T., & Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33, 1–22.
Haertel, E. H. (1989). Using restricted latent class models to map the attribute structure of achievement items. Journal of Educational Measurement, 26, 333–352.
Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.
Junker, B., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272.
Köhn, H.-F., & Chiu, C.-Y. (2016). A proof of the duality of the dina model and the dino model. Journal of Classification, 33, 171–184.
Köhn, H. F., & Chiu, C.-Y. (2017). A procedure for assessing the completeness of the q-matrices of cognitively diagnostic tests. Psychometrika, 82, 112–132.
Liu, J. (2017). On the consistency of Q-matrix estimation: A commentary. Psychometrika, 82, 523-527.
Liu, J., Xu, G., & Ying, Z. (2012). Data-driven learning of Q-matrix. Applied Psychological Measurement, 36, 548–564.
Liu, J., Xu, G., & Ying, Z. (2013). Theory of self-learning Q-matrix. Bernoulli, 19, 1790–1817.
Rupp, A. A., & Templin, J. L. (2008). The effects of Q-matrix misspecification on parameter estimates and classification accuracy in the DINA model. Educational and Psychological Measurement, 68, 78–96.
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 19, 461–464.
Sun, J., Chen, Y., Liu, J., Ying, Z., & Xin, T. (2016). Latent variable selection for multidimensional item response theory models via L 1 regularization. Psychometrika, 81(4), 921-939. https://doi.org/10.1007/s11336-016-9529-6
Templin, J. L., & Henson, R. A. (2006). A Bayesian method for incorporating uncertainty into Q-matrix estimation in skills assessment. In Symposium Conducted at the Meeting of the American Educational Research Association, San Diego, CA.
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58, 267–288.
Tseng, P. (1988). Coordinate ascent for maximizing nondifferentiable concave functions (Technical Report LIDS-P, 1840). Massachusetts Institute of Technology, Laboratory for Information and Decision Systems.
Tseng, P. (2001). Convergence of block coordinate descent method for nondifferentiable maximization. Journal of Optimization Theory and Applications, 109, 474–494.
von Davier, M. (2005, September). A general diagnostic model applied to language testing data (Research report No. RR-05-16). Princeton, NJ: Educational Testing Service.
von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287–301.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Liu, J., Kang, HA. (2019). Q-Matrix Learning via Latent Variable Selection and Identifiability. In: von Davier, M., Lee, YS. (eds) Handbook of Diagnostic Classification Models. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-030-05584-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-05584-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05583-7
Online ISBN: 978-3-030-05584-4
eBook Packages: EducationEducation (R0)