Abstract
Uncovering the right skills behind question items is a difficult task. It requires a thorough understanding of the subject matter and of the cognitive factors that determine student performance. The skills definition, and the mapping of item to skills, require the involvement of experts. We investigate means to assist experts for this task by using a data driven, matrix factorization approach. The two mappings of items to skills, the expert on one side and the matrix factorization on the other, are compared in terms of discrepancies, and in terms of their performance when used in a linear model of skills assessment and item outcome prediction. Visual analysis shows a relatively similar pattern between the expert and the factorized mappings, although differences arise. The prediction comparison shows the factorization approach performs slightly better than the original expert Q-matrix, giving supporting evidence to the belief that the factorization mapping is valid. Implications for the use of the factorization to design better item to skills mapping are discussed.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Barnes, T.: Novel derivation and application of skill matrices: The Q-matrix method. Handbook on Educational Data Mining (2010)
Cetintas, S., Si, L., Xin, Y.P., Hord, C.: Predicting correctness of problem solving in ITS with a temporal collaborative filtering approach. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010, Part I. LNCS, vol. 6094, pp. 15–24. Springer, Heidelberg (2010)
De La Torre, J.: An empirically based method of q-matrix validation for the dina model: Development and applications. Journal of Educational Measurement 45(4), 343–362 (2008)
DeCarlo, L.T.: On the Analysis of Fraction Subtraction Data: The DINA Model, Classification, Latent Class Sizes, and the Q-Matrix. Applied Psychological Measurement 35, 8–26 (2011)
Desmarais, M.C.: Conditions for effectively deriving a q-matrix from data with non-negative matrix factorization. In: Conati, C., Ventura, S., Calders, T., Pechenizkiy, M. (eds.) 4th International Conference on Educational Data Mining, EDM 2011, Eindhoven, Netherlands, June 6-8, pp. 41–50 (2011)
Desmarais, M.C.: Mapping question items to skills with non-negative matrix factorization. ACM KDD-Explorations 13(2), 30–36 (2011)
Desmarais, M.C., Beheshti, B., Naceur, R.: Item to skills mapping: Deriving a conjunctive Q-matrix from data. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) ITS 2012. LNCS, vol. 7315, pp. 454–463. Springer, Heidelberg (2012)
Koedinger, K.R., McLaughlin, E.A., Stamper, J.C.: Automated student model improvement. In: Proceedings of the 5th International Conference on Educational Data Mining, pp. 17–24 (2012)
Lan, A.S., Waters, A.E., Studer, C., Baraniuk, R.G.: Sparse factor analysis for learning and content analytics. arXiv preprint arXiv:1303.5685 (2013)
Li, N., Cohen, W.W., Matsuda, N., Koedinger, K.R.: A machine learning approach for automatic student model discovery. In: Proceedings of the 4th International Conference on Educational Data Mining, pp. 31–40 (2011)
Liu, J., Xu, G., Ying, Z.: Data-driven learning of q-matrix. Applied Psychological Measurement 36(7), 548–564 (2012), http://apm.sagepub.com/content/36/7/548.abstract
Robitzsch, A., Kiefer, T., George, A., Uenlue, A., Robitzsch, M.: Package CDM (2012), http://cran.r-project.org/web/packages/CDM/index.html
Stamper, J.C., Barnes, T., Croy, M.J.: Extracting student models for intelligent tutoring systems. In: AAAI 2007, pp. 1900–1901. AAAI Press (2007)
Tatsuoka, K.K.: Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement 20, 345–354 (1983)
Tatsuoka, K.: Cognitive Assessment: An Introduction to the Rule Space Method. Routledge Academic (2009)
Tatsuoka, K.: Analysis of errors in fraction addition and subtraction problems. Computer-based Education Research Laboratory, University of Illinois (1984)
Thai-Nghe, N., Drumond, L., Horváth, T., Nanopoulos, A., Schmidt-Thieme, L.: Matrix and tensor factorization for predicting student performance. In: Verbraeck, A., Helfert, M., Cordeiro, J., Shishkov, B. (eds.) CSEDU 2011 - Proceedings of the 3rd International Conference on Computer Supported Education, Noordwijkerhout, Netherlands, May 6-8, vol. 1, pp. 69–78. SciTePress (2011)
Thai-Nghe, N., Horváth, T., Schmidt-Thieme, L.: Factorization models for forecasting student performance. In: Conati, C., Ventura, S., Pechenizkiy, M., Calders, T. (eds.) Proceedings of EDM 2011, The 4th International Conference on Educational Data Mining, Eindhoven, Netherlands, July 6-8, pp. 11–20 (2011), http://www.educationaldatamining.org
Toscher, A., Jahrer, M.: Collaborative filtering applied to educational data mining. Tech. rep., KDD Cup 2010: Improving Cognitive Models with Educational Data Mining (2010)
Winters, T.: Educational Data Mining: Collection and Analysis of Score Matrices for Outcomes-Based Assessment. Ph.D. thesis, University of California Riverside (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Desmarais, M.C., Naceur, R. (2013). A Matrix Factorization Method for Mapping Items to Skills and for Enhancing Expert-Based Q-Matrices. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds) Artificial Intelligence in Education. AIED 2013. Lecture Notes in Computer Science(), vol 7926. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39112-5_45
Download citation
DOI: https://doi.org/10.1007/978-3-642-39112-5_45
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39111-8
Online ISBN: 978-3-642-39112-5
eBook Packages: Computer ScienceComputer Science (R0)