Abstract
Student assessment aims to diagnose student latent attributes (e.g., skill proficiency), which is a crucial issue for many educational applications. Existing studies, such as cognitive diagnosis, mainly focus on exploiting students’ scores on questions to mine their attributes from an independent exam. However, in many real-world scenarios, different students usually participate in different exams, where the results obtained from different exams by traditional methods are not comparable to each other. Therefore, the problem of conducting assessments from different exams to obtain precise and comparable results is still underexplored. To this end, in this paper, we propose a Multi Task - Multidimensional Cognitive Diagnosis framework (MT-MCD) for student assessment from different exams simultaneously. In the framework, we first apply a multidimensional cognitive diagnosis model for each independent assessment task. Then, we extract features from the question texts to bridge the connections with each task. After that, we employ a multi-task optimization method for the framework learning. MT-MCD is a general framework where we develop two effective implementations based on two representative cognitive diagnosis models. We conduct extensive experiments on real-world datasets where the experimental results demonstrate that MT-MCD can obtain more precise and comparable assessment results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Baker, R.S.J.D., Yacef, K.: The state of educational data mining in 2009: a review and future visions. JEDM-J. Educ. Data Min. 1(1), 3–17 (2009)
Bansal, T., Belanger, D., McCallum, A.: Ask the GRU: multi-task learning for deep text recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 107–114. ACM (2016)
Bickel, S., Bogojeska, J., Lengauer, T., Scheffer, T.: Multi-task learning for HIV therapy screening. In: Proceedings of the 25th International Conference on Machine Learning, pp. 56–63. ACM (2008)
Cox, K., Imrie, B.W., Miller, A.: Student Assessment in Higher Education: A Handbook for Assessing Performance. Routledge, London (2014)
Cui, Y., Li, J.: Evaluating person fit for cognitive diagnostic assessment. Appl. Psychol. Meas. 39(3), 223–238 (2015)
De La Torre, J., Minchen, N.: Cognitively diagnostic assessments and the cognitive diagnosis model framework. Psicología Educativa 20(2), 89–97 (2014)
DiBello, L.V., Roussos, L.A., Stout, W.: 31A review of cognitively diagnostic assessment and a summary of psychometric models. Handb. Stat. 26, 979–1030 (2006)
DiBello, L.V., Stout, W.: Guest editors’ introduction and overview: IRT-based cognitive diagnostic models and related methods. J. Educ. Meas. 44(4), 285–291 (2007)
Evgeniou, T., Pontil, M.: Regularized multi-task learning. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 109–117. ACM (2004)
Fouss, F., Pirotte, A., Renders, J.-M., Saerens, M.: Random-walk computation of similarities between nodes of a graph with application to collaborative recommendation. IEEE Trans. Knowl. Data Eng. 19(3), 355–369 (2007)
Huebner, A.: An overview of recent developments in cognitive diagnostic computer adaptive assessments. Pract. Assess. Res. Eval. 15(3), n3 (2010)
Huff, K., Goodman, D.P.: The demand for cognitive diagnostic assessment (2007)
Klaus, D., Kubinger, K.D.: On the revival of the rasch model-based LLTM: from constructing tests using item generating rules to measuring item administration effects. Psychol. Sci. 50(3), 311 (2008)
Kuncel, N.R., Hezlett, S.A., Ones, D.S.: A comprehensive meta-analysis of the predictive validity of the graduate record examinations: implications for graduate student selection and performance. Psychol. Bull. 127(1), 162 (2001)
Lee, J.: Multidimensional Item Response Theory: An Investigation of Interaction Effects Between Factors on Item Parameter Recovery Using Markov Chain Monte Carlo. Michigan State University, Measurement and Quantitative Methods (2012)
Leighton, J., Gierl, M.: Cognitive Diagnostic Assessment for Education: Theory and Applications. Cambridge University Press, Cambridge (2007)
Liu, Q., Runze, W., Chen, E., Guandong, X., Yu, S., Chen, Z., Guoping, H.: Fuzzy cognitive diagnosis for modelling examinee performance. ACM Trans. Intell. Syst. Technol. (TIST) 9(4), 48 (2018)
Reckase, M.: Multidimensional Item Response Theory, vol. 150. Springer, New York (2009). https://doi.org/10.1007/978-0-387-89976-3
Romero, C., Ventura, S., Pechenizkiy, M., d Baker, R.S.J.: Handbook of Educational Data Mining. CRC Press, Boca Raton (2010)
Saxon, P.D., Morante, E.A.: Effective student assessment and placement: challenges and recommendations. J. Dev. Educ. 37(3), 24 (2014)
Scheuer, O., McLaren, B.M.: Educational data mining. In: Seel, N.M. (ed.) Encyclopedia of the Sciences of Learning, pp. 1075–1079. Springer, Boston (2012). https://doi.org/10.1007/978-1-4419-1428-6
Serrano-Laguna, Á., Torrente, J., Moreno-Ger, P., Fernández-Manjón, B.: Tracing a little for big improvements: application of learning analytics and videogames for student assessment. Procedia Comput. Sci. 15, 203–209 (2012)
Sheng, Y.: Markov chain Monte Carlo estimation of normal ogive IRT models in MATLAB. J. Stat. Softw. 25(8), 1–15 (2008)
Sheng, Y., Headrick, T.C.: A gibbs sampler for the multidimensional item response model. ISRN Appl. Math. 2012, 14 (2012)
Wu, R., Liu, Q., Liu, Y., Chen, E., Su, Y., Chen, Z., Hu, G.: Cognitive modelling for predicting examinee performance. In: IJCAI, pp. 1017–1024 (2015)
Wu, R., Xu, G., Chen, E., Liu, Q., Ng, W.: Knowledge or gaming?: cognitive modelling based on multiple-attempt response. In: Proceedings of the 26th International Conference on World Wide Web Companion, pp. 321–329. International World Wide Web Conferences Steering Committee (2017)
Jun, Y., Zhang, B., Kuang, Z., Lin, D., Fan, J.: iPrivacy: image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Trans. Inf. Forensics Secur. 12(5), 1005–1016 (2017)
Zhou, J., Chen, J., Ye, J.: Malsar: multi-task learning via structural regularization. Arizona State University, vol. 21 (2011)
Zhou, J., Chen, J., Ye, J.: Multi-task learning: theory, algorithms, and applications. In: https://www.siam.org/meetings/sdm12/zhou_chen_ye.pdf. Citeseer (2012)
Acknowledgments
This research was partially supported by grants from the National Natural Science Foundation of China (Grants No. 61672483, U1605251 and 91546103), and the Youth Innovation Promotion Association of CAS (No. 2014299).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Zhu, T. et al. (2018). MT-MCD: A Multi-task Cognitive Diagnosis Framework for Student Assessment. In: Pei, J., Manolopoulos, Y., Sadiq, S., Li, J. (eds) Database Systems for Advanced Applications. DASFAA 2018. Lecture Notes in Computer Science(), vol 10828. Springer, Cham. https://doi.org/10.1007/978-3-319-91458-9_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-91458-9_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91457-2
Online ISBN: 978-3-319-91458-9
eBook Packages: Computer ScienceComputer Science (R0)