Abstract
In this chapter, we provide a brief survey of Markov chain Monte Carlo (MCMC) methods used in estimating Cognitive Diagnostic Models (CDMs). MCMC techniques have been widely used for the Bayesian estimation of psychometric models. MCMC algorithms and general purpose MCMC software has been facilitating the development of modern psychometric models that are otherwise difficult to fit (Levy R, J Probab Stat 1–18, 2009. Retrieved from http://www.hindawi.com/journals/jps/2009/537139/, https://doi.org/10.1155/2009/ 537139). We introduce a Gibbs sampler for fitting the saturated Log-linear CDM model (LCDM, Henson RA, Templin JL, Willse JT, Psychometrika, 74(2):191–210, 2009. Retrieved from https://doi.org/10.1007/s11336-008-9089-5). The utility of Bayesian inference is demonstrated by analyzing the Examination for the Certificate of Proficiency in English (ECPE) dataset.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Brooks, S., & Gelman, A. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7(4), 434–455. https://doi.org/10.1080/10618600.1998.10474787
Brooks, S., Gelman, A., Jones, G. L., & Meng, X.-L. (2011). Handbook of Markov chain Monte Carlo. Handbook of Markov chain Monte Carlo. https://doi.org/10.1201/b10905
Cai, L. (2010). High-dimensional exploratory item factor analysis by a Metropolis-Hastings Robbins-Monro algorithm. Psychometrika, 75(1), 33–57. https://doi.org/10.1007/s11336-009-9136-x
Chen, Y., Culpepper, S. A., Chen, Y., & Douglas, J. (2018). Bayesian estimation of the DINA Q matrix. Psychometrika, 83(1), 89–108. Retrieved from https://doi.org/10.1007/s11336-017-9579-4
Chen, Y., Liu, J., Xu, G., & Ying, Z. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110(510), 850–866. Retrieved from https://doi.org/10.1080/01621459.2014.934827
Chung, M.-t. (2014). Estimating the Q-matrix for cognitive diagnosis models in a Bayesian framework. Doctoral dissertation, Columbia University. Retrieved from https://search.proquest.com/docview/1548332406
Culpepper, S. A. (2015) Bayesian estimation of the DINA model with Gibbs sampling. Journal of Educational and Behavioral Statistics, 40(5), 454–476. Retrieved from https://doi.org/10.3102/1076998615595403
DeCarlo, L. T. (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model. Applied Psychological Measurement, 36(6), 447–468. Retrieved from https://doi.org/10.1177/0146621612449069
DeCarlo, L. T., & Kinghorn, B. R. (2016). An Exploratory Approach to the Q-Matrix Via Bayesian Estimation. (Paper presented at the meeting of the National Council on Measurement in Education, Washington, DC)
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B, 39(1), 1–38. Retrieved from http://www.jstor.org/stable/2984875, https://doi.org/10.2307/2984875
de la Torre, J. (2008). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34(1), 115–130. Retrieved from https://doi.org/10.3102/1076998607309474
de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76(2), 179–199. https://doi.org/10.1007/s11336-011-9207-7
de la Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69(3), 333–353. Retrieved from https://doi.org/10.1007/BF02295640
Gelfand, A. E., & Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85(410), 398–409. Retrieved from https://doi.org/10.1080/01621459.1990.10476213
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian data analysis (3 ed.). Boca Raton: Chapman and Hall/CRC.
Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science. https://doi.org/10.1214/ss/1177011136
Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-6(6), 721–741. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/22499653, https://doi.org/10.1109/TPAMI.1984.4767596
George, A. C., Robitzsch, A., Kiefer, T., Groß, J., & Ünlü, A. (2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software, 74(2). Retrieved from http://www.jstatsoft.org/v74/i02/ https://doi.org/10.18637/jss.v074.i02
Geweke, J. (1992). Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments. In J. M. Bernardo, J. O. Berger, A. P. Dawid, & A. F. M. Smith (Eds.), Bayesian statistics 4. Oxford: Oxford University Press.
Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika, 82(4), 711–732. Retrieved from https://doi.org/10.1093/biomet/82.4.711
Hartz, S. M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Doctoral dissertation, University of Illinois at Urbana-Champaign. Retrieved from http://hdl.handle.net/2142/87393
Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1), 97–109. Retrieved from http://www.jstor.org/stable/2334940
Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74(2), 191–210. Retrieved from https://doi.org/10.1007/s11336-008-9089-5
Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272. Retrieved from https://doi.org/10.1177/01466210122032064
Levy, R. (2009). The rise of Markov chain Monte Carlo estimation for psychometric modeling. Journal of Probability and Statistics, 2009, 1–18. Retrieved from http://www.hindawi.com/journals/jps/2009/537139/, https://doi.org/10.1155/2009/537139
Li, F., Cohen, A., Bottge, B., & Templin, J. (2016). A latent transition analysis model for assessing change in cognitive skills. Educational and Psychological Measurement, 76(2), 181–204. Retrieved from https://doi.org/10.1177/0013164415588946
Liu, J., Xu, G., & Ying, Z. (2012). Data-driven learning of Q-matrix. Applied Psychological Measurement, 36(7), 548–564. Retrieved from https://doi.org/10.1177/0146621612456591
Liu, J., Xu, G., & Ying, Z. (2013). Theory of self-learning Q-matrix. Bernoulli, 19(5), 1790–1817. Retrieved from https://doi.org/10.3150/12-BEJ430
Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. (2009) The BUGS project: Evolution, critique and future directions. Statistics in Medicine, 28(25), 3049–3067. Retrieved from https://doi.org/10.1002/sim.3680
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953). Equation of state calculations by fast computing machines. Journal Chemical Physics. https://doi.org/10.1063/1.1699114
Neal, R. M. (1998). Probabilistic inference using Markov chain Monte Carlo methods. Technical Report, 1, 1–144. Retrieved from papers2://publication/uuid/0C88167E-5379-4E4E-A9E4-007ABA4F716D, https://doi.org/10.1021/np100920q
Park, J. Y., Johnson, M. S., & Lee, Y. S. (2015). Posterior predictive model checks for cognitive diagnostic models. International Journal of Quantitative Research in Education, 2(3/4), 244. Retrieved from http://www.inderscience.com/link.php?id=71738, https://doi.org/10.1504/IJQRE.2015.071738
Plummer, M. (2005). JAGS: Just another Gibbs sampler. In Proceedings of the 3rd International Workshop on Distributed Statistical Computing (dsc 2003).
Roberts, G. O., Gelman, A., & Gilks, W. R. (1997). Weak convergence and optimal scaling of random walk Metropolis algorithms. Annals of Applied Probability, 7(1), 110–120. https://doi.org/10.1214/aoap/1034625254
Rubin, D. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. The Annals of Statistics, 12(4), 1151–1172. Retrieved from http://projecteuclid.org/euclid.aos/1176346785, https://doi.org/10.1214/aos/1176346785
Sinharay, S. (2005). Assessing fit of unidimensional item response theory models using a Bayesian approach. Journal of Educational Measurement, 42(4), 375–394. Retrieved from https://doi.org/10.1111/j.1745-3984.2005.00021.x
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & van der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society. Series B, 64(4), 583–639.
Spiegelhalter, D. J., Thomas, A., Best, N., & Lunn, D. (2014). OpenBUGS User Manual (Vol. 164). Retrieved from http://www.openbugs.net/Manuals/Manual.html
Tatsuoka, K. K. (1983) Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20(4), 345–354. Retrieved from https://doi.org/10.1111/j.1745-3984.1983.tb00212.x
Templin, J., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79(2), 317–339. Retrieved from https://doi.org/10.1007/s11336-013-9362-0
Templin, J., & Hoffman, L. (2013). Obtaining diagnostic classification model estimates using Mplus. Educational Measurement: Issues and Practice, 32(2), 37–50. https://doi.org/10.1111/emip.12010
Thomas, A., Spiegelhalter, D. J., & Gilks, W. R. (1992). BUGS: A program to perform Bayesian inference using Gibbs sampling. In J. M. Bernardo, J. O. Berger, A. P. Dawid, & A. F. M. Smith (Eds.), Bayesian statistics 4. Oxford: Oxford University Press.
von Davier, M. (2008). A general diagnostic model applied to language testing data. The British Journal of Mathematical and Statistical Psychology, 61(Pt 2), 287–307. https://doi.org/10.1348/000711007X193957
von Davier, M. (2014). The log-linear cognitive diagnostic model (LCDM) as a special case of the general diagnostic model (GDM). ETS Research Report Series, 2014(2), 1–13. Retrieved from https://doi.org/10.1002/ets2.12043
Xu, G., & Zhang, S. (2016). Identifiability of diagnostic classification models. Psychometrika, 81(3), 625–649. https://doi.org/10.1007/s11336-015-9471-z
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Liu, X., Johnson, M.S. (2019). Estimating CDMs Using MCMC. In: von Davier, M., Lee, YS. (eds) Handbook of Diagnostic Classification Models. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-030-05584-4_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-05584-4_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05583-7
Online ISBN: 978-3-030-05584-4
eBook Packages: EducationEducation (R0)