Advertisement

MT-MCD: A Multi-task Cognitive Diagnosis Framework for Student Assessment

  • Tianyu Zhu
  • Qi Liu
  • Zhenya Huang
  • Enhong ChenEmail author
  • Defu Lian
  • Yu Su
  • Guoping Hu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10828)

Abstract

Student assessment aims to diagnose student latent attributes (e.g., skill proficiency), which is a crucial issue for many educational applications. Existing studies, such as cognitive diagnosis, mainly focus on exploiting students’ scores on questions to mine their attributes from an independent exam. However, in many real-world scenarios, different students usually participate in different exams, where the results obtained from different exams by traditional methods are not comparable to each other. Therefore, the problem of conducting assessments from different exams to obtain precise and comparable results is still underexplored. To this end, in this paper, we propose a Multi Task - Multidimensional Cognitive Diagnosis framework (MT-MCD) for student assessment from different exams simultaneously. In the framework, we first apply a multidimensional cognitive diagnosis model for each independent assessment task. Then, we extract features from the question texts to bridge the connections with each task. After that, we employ a multi-task optimization method for the framework learning. MT-MCD is a general framework where we develop two effective implementations based on two representative cognitive diagnosis models. We conduct extensive experiments on real-world datasets where the experimental results demonstrate that MT-MCD can obtain more precise and comparable assessment results.

Keywords

Student assessment Cognitive diagonosis Item Response Theory Multi-task learning 

Notes

Acknowledgments

This research was partially supported by grants from the National Natural Science Foundation of China (Grants No. 61672483, U1605251 and 91546103), and the Youth Innovation Promotion Association of CAS (No. 2014299).

References

  1. 1.
    Baker, R.S.J.D., Yacef, K.: The state of educational data mining in 2009: a review and future visions. JEDM-J. Educ. Data Min. 1(1), 3–17 (2009)Google Scholar
  2. 2.
    Bansal, T., Belanger, D., McCallum, A.: Ask the GRU: multi-task learning for deep text recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 107–114. ACM (2016)Google Scholar
  3. 3.
    Bickel, S., Bogojeska, J., Lengauer, T., Scheffer, T.: Multi-task learning for HIV therapy screening. In: Proceedings of the 25th International Conference on Machine Learning, pp. 56–63. ACM (2008)Google Scholar
  4. 4.
    Cox, K., Imrie, B.W., Miller, A.: Student Assessment in Higher Education: A Handbook for Assessing Performance. Routledge, London (2014)Google Scholar
  5. 5.
    Cui, Y., Li, J.: Evaluating person fit for cognitive diagnostic assessment. Appl. Psychol. Meas. 39(3), 223–238 (2015)MathSciNetCrossRefGoogle Scholar
  6. 6.
    De La Torre, J., Minchen, N.: Cognitively diagnostic assessments and the cognitive diagnosis model framework. Psicología Educativa 20(2), 89–97 (2014)CrossRefGoogle Scholar
  7. 7.
    DiBello, L.V., Roussos, L.A., Stout, W.: 31A review of cognitively diagnostic assessment and a summary of psychometric models. Handb. Stat. 26, 979–1030 (2006)CrossRefGoogle Scholar
  8. 8.
    DiBello, L.V., Stout, W.: Guest editors’ introduction and overview: IRT-based cognitive diagnostic models and related methods. J. Educ. Meas. 44(4), 285–291 (2007)CrossRefGoogle Scholar
  9. 9.
    Evgeniou, T., Pontil, M.: Regularized multi-task learning. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 109–117. ACM (2004)Google Scholar
  10. 10.
    Fouss, F., Pirotte, A., Renders, J.-M., Saerens, M.: Random-walk computation of similarities between nodes of a graph with application to collaborative recommendation. IEEE Trans. Knowl. Data Eng. 19(3), 355–369 (2007)CrossRefGoogle Scholar
  11. 11.
    Huebner, A.: An overview of recent developments in cognitive diagnostic computer adaptive assessments. Pract. Assess. Res. Eval. 15(3), n3 (2010)Google Scholar
  12. 12.
    Huff, K., Goodman, D.P.: The demand for cognitive diagnostic assessment (2007)Google Scholar
  13. 13.
    Klaus, D., Kubinger, K.D.: On the revival of the rasch model-based LLTM: from constructing tests using item generating rules to measuring item administration effects. Psychol. Sci. 50(3), 311 (2008)Google Scholar
  14. 14.
    Kuncel, N.R., Hezlett, S.A., Ones, D.S.: A comprehensive meta-analysis of the predictive validity of the graduate record examinations: implications for graduate student selection and performance. Psychol. Bull. 127(1), 162 (2001)CrossRefGoogle Scholar
  15. 15.
    Lee, J.: Multidimensional Item Response Theory: An Investigation of Interaction Effects Between Factors on Item Parameter Recovery Using Markov Chain Monte Carlo. Michigan State University, Measurement and Quantitative Methods (2012)Google Scholar
  16. 16.
    Leighton, J., Gierl, M.: Cognitive Diagnostic Assessment for Education: Theory and Applications. Cambridge University Press, Cambridge (2007)CrossRefGoogle Scholar
  17. 17.
    Liu, Q., Runze, W., Chen, E., Guandong, X., Yu, S., Chen, Z., Guoping, H.: Fuzzy cognitive diagnosis for modelling examinee performance. ACM Trans. Intell. Syst. Technol. (TIST) 9(4), 48 (2018)Google Scholar
  18. 18.
    Reckase, M.: Multidimensional Item Response Theory, vol. 150. Springer, New York (2009).  https://doi.org/10.1007/978-0-387-89976-3CrossRefzbMATHGoogle Scholar
  19. 19.
    Romero, C., Ventura, S., Pechenizkiy, M., d Baker, R.S.J.: Handbook of Educational Data Mining. CRC Press, Boca Raton (2010)CrossRefGoogle Scholar
  20. 20.
    Saxon, P.D., Morante, E.A.: Effective student assessment and placement: challenges and recommendations. J. Dev. Educ. 37(3), 24 (2014)Google Scholar
  21. 21.
    Scheuer, O., McLaren, B.M.: Educational data mining. In: Seel, N.M. (ed.) Encyclopedia of the Sciences of Learning, pp. 1075–1079. Springer, Boston (2012).  https://doi.org/10.1007/978-1-4419-1428-6CrossRefGoogle Scholar
  22. 22.
    Serrano-Laguna, Á., Torrente, J., Moreno-Ger, P., Fernández-Manjón, B.: Tracing a little for big improvements: application of learning analytics and videogames for student assessment. Procedia Comput. Sci. 15, 203–209 (2012)CrossRefGoogle Scholar
  23. 23.
    Sheng, Y.: Markov chain Monte Carlo estimation of normal ogive IRT models in MATLAB. J. Stat. Softw. 25(8), 1–15 (2008)CrossRefGoogle Scholar
  24. 24.
    Sheng, Y., Headrick, T.C.: A gibbs sampler for the multidimensional item response model. ISRN Appl. Math. 2012, 14 (2012)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Wu, R., Liu, Q., Liu, Y., Chen, E., Su, Y., Chen, Z., Hu, G.: Cognitive modelling for predicting examinee performance. In: IJCAI, pp. 1017–1024 (2015)Google Scholar
  26. 26.
    Wu, R., Xu, G., Chen, E., Liu, Q., Ng, W.: Knowledge or gaming?: cognitive modelling based on multiple-attempt response. In: Proceedings of the 26th International Conference on World Wide Web Companion, pp. 321–329. International World Wide Web Conferences Steering Committee (2017)Google Scholar
  27. 27.
    Jun, Y., Zhang, B., Kuang, Z., Lin, D., Fan, J.: iPrivacy: image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Trans. Inf. Forensics Secur. 12(5), 1005–1016 (2017)CrossRefGoogle Scholar
  28. 28.
    Zhou, J., Chen, J., Ye, J.: Malsar: multi-task learning via structural regularization. Arizona State University, vol. 21 (2011)Google Scholar
  29. 29.
    Zhou, J., Chen, J., Ye, J.: Multi-task learning: theory, algorithms, and applications. In: https://www.siam.org/meetings/sdm12/zhou_chen_ye.pdf. Citeseer (2012)

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Tianyu Zhu
    • 1
  • Qi Liu
    • 1
  • Zhenya Huang
    • 1
  • Enhong Chen
    • 1
    Email author
  • Defu Lian
    • 2
  • Yu Su
    • 3
  • Guoping Hu
    • 4
  1. 1.Anhui Province Key Laboratory of Big Data Analysis and ApplicationUniversity of Science and Technology of ChinaHefeiChina
  2. 2.University of Electronic Science and Technology of ChinaChengduChina
  3. 3.Anhui UniversityHefeiChina
  4. 4.Anhui USTC IFLYTEK Co., Ltd.HefeiChina

Personalised recommendations