Abstract
Theories of discourse comprehension assume that understanding is a process of making connections between new information (e.g., in a text) and prior knowledge, and that the quality of comprehension is a function of the coherence of the mental representation. When readers are exposed to multiple sources of information, they must make connections both within and between the texts. One challenge is how to represent this coherence and in turn how to predict readers’ levels of comprehension. In this study, we represent coherence using Cohesion Network Analysis (CNA) in which we model a global cohesion graph that semantically links reference texts to different student verbal productions. Our aim is to create an automated model of comprehension prediction based on features extracted from the CNA graph. We examine the cohesion links between the four texts read by 146 students and their (a) self-explanations generated on target sentences and (b) responses to open-ended questions. We analyze the degree to which features derived from the cohesive links from the extended CNA graph are predictive of students’ comprehension scores (on a [0 to 12] scale) using either (a) students’ self-explanations, (b) responses to comprehension questions, or (c) both. We compared the use of Linear Regression, Extra Trees Regressor, Support Vector Regression, and Multi-Layer Perceptron. Our best model used Linear Regression, obtaining a 1.29 mean absolute error when predicting comprehension scores using both sources of verbal responses (i.e., self-explanations and question answers).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Allen, L.K., Jacovina, M., McNamara, D.S.: Cohesive features of deep text comprehension processes. In: 38th Annual Meeting of the Cognitive Science Society, pp. 2681–2686. Cognitive Science Society, Philadelphia, PA (2016)
Dascalu, M., McNamara, D.S., Trausan-Matu, S., Allen, L.K.: Cohesion network analysis of CSCL participation. Behav. Res. Methods 50(2), 604–619 (2018)
Hastings, P., Hughes, S., Magliano, J.P., Goldman, S.R., Lawless, K.: Assessing the use of multiple sources in student essays. Behav. Res. Methods 44(3), 622–633 (2012)
Wasserman, S., Faust, K.: Social Network Analysis: Methods and Applications. Cambridge University Press, Cambridge (1994)
Jurafsky, D., Martin, J.H.: An Introduction to Natural Language Processing. Computational Linguistics, and Speech Recognition. Pearson Prentice Hall, London (2009)
McNamara, D.S., Magliano, J.: Toward a comprehensive model of comprehension. Psychol. Learn. Motiv. 51, 297–384 (2009)
Kintsch, W.: The role of knowledge in discourse comprehension: a construction-integration model. Psychol. Rev. 95(2), 163 (1988)
Wiley, J., Goldman, S.R., Graesser, A.C., Sanchez, C.A., Ash, I.K., Hemmerich, J.A.: Source evaluation, comprehension, and learning in Internet science in-quiry tasks. Am. Educ. Res. J. 46(4), 1060–1106 (2009)
Magliano, J., McCrudden, M.T., Rouet, J.F., Sabatini, J.: The modern reader: should changes to how we read affect research and theory? In: Schober, M.F., Rapp, D.N., Britt, M.A. (eds.) Handbook of Discourse Processes, 2nd Ed. Taylor & Francis, New York (2018)
Goldman, S.R., Braasch, J.L., Wiley, J., Graesser, A.C., Brodowinska, K.: Comprehending and learning from Internet sources: processing patterns of better and poorer learners. Read. Res. Q. 47(4), 356–381 (2012)
Goldman, S.R., Rakestraw, J.A.: Structural aspects of constructing meaning from text. In: Kamil, M.L., Mosenthal, P.B., Pearson, P.D., Barr, R. (eds.) Handbook of Reading Research, vol. 3, pp. 311–335. Erlbaum, Mahwah (2000)
Ozuru, Y., Briner, S., Kurby, C.A., McNamara, D.S.: Comparing text comprehension measured by multiple-choice and open-ended questions. Can. J. Exp. Psychol. 67, 215–227 (2013)
Chi, M.T., De Leeuw, N., Chiu, M.-H., LaVancher, C.: Eliciting self-explanations improves understanding. Cogn. Sci. 18(3), 439–477 (1994)
McNamara, D.S.: Self-explanation and reading strategy training (SERT) improves low-knowledge students’ science course performance. Discourse Processes 54(7), 479–492 (2017)
Dascalu, M., Trausan-Matu, S., McNamara, D.S., Dessus, P.: ReaderBench – automated evaluation of collaboration based on cohesion and dialogism. Int. J. Comput.-Support. Collaborative Learn. 10(4), 395–423 (2015)
Trausan-Matu, S., Dascalu, M., Dessus, P.: Textual complexity and discourse structure in computer-supported collaborative learning. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) ITS 2012. LNCS, vol. 7315, pp. 352–357. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-30950-2_46
Dascalu, M., Trausan-Matu, S., Dessus, P.: Cohesion-based analysis of CSCL conversations: holistic and individual perspectives. In: 10th International Conference on Computer-Supported Collaborative Learning (CSCL 2013), vol. 1, pp. 145–152. ISLS, Madison, USA (2013)
Dascalu, M., Dessus, P., Bianco, M., Trausan-Matu, S., Nardy, A.: Mining texts, learner productions and strategies with ReaderBench. In: Peña-Ayala, A. (ed.) Educational Data Mining. SCI, vol. 524, pp. 345–377. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-02738-8_13
Budanitsky, A., Hirst, G.: Evaluating WordNet-based measures of lexical semantic relatedness. Comput. Linguist. 32(1), 13–47 (2006)
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. 12, 2825–2830 (2011)
Dascalu, M.: Analyzing Discourse and Text Complexity for Learning and Collaborating. Studies in Computational Intelligence, vol. 534. Springer, Switzerland (2014). https://doi.org/10.1007/978-3-319-03419-5
Landauer, T.K., Foltz, P.W., Laham, D.: An introduction to latent semantic analysis. Discourse Processes 25(2/3), 259–284 (1998)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representation in vector space. In: Workshop at ICLR, Scottsdale, AZ (2013)
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3(4–5), 993–1022 (2003)
Wu, Z., Palmer, M.: Verb semantics and lexical selection. In: 32nd Annual Meeting of the Association for Computational Linguistics ACL 1994, pp. 133–138. ACL, New Mexico (1994)
Acknowledgments
This research was supported by the ReadME project “Interactive and Innovative application for evaluating the readability of texts in Romanian Language and for improving users’ writing styles”, contract no. 114/15.09.2017, MySMIS 2014 code 119286, the FP7 2008-212578 LTfLL project, the Institute of Education Sciences (R305A180144 and R305A180261), and the Office of Naval Research (N00014-17-1-2300).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Nicula, B., Perret, C.A., Dascalu, M., McNamara, D.S. (2019). Predicting Multi-document Comprehension: Cohesion Network Analysis. In: Isotani, S., Millán, E., Ogan, A., Hastings, P., McLaren, B., Luckin, R. (eds) Artificial Intelligence in Education. AIED 2019. Lecture Notes in Computer Science(), vol 11625. Springer, Cham. https://doi.org/10.1007/978-3-030-23204-7_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-23204-7_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-23203-0
Online ISBN: 978-3-030-23204-7
eBook Packages: Computer ScienceComputer Science (R0)