Skip to main content
SpringerLink
Log in

Multiple Document Comprehension of University Students

Test Development and Relations to Person and Process Characteristics

  • Chapter
  • First Online:
Student Learning in German Higher Education

Abstract

Multiple document comprehension is the ability to construct an integrated representation of a specific topic based on several sources. It is an important competence for university students; however, there has been so far no established instrument to assess multiple document comprehension in a standardized way. Therefore, we developed a test covering four theory-based cognitive requirements: The corroboration of information across texts, the integration of information across texts, the comparison of sources and source evaluations across texts, and the comparison of source-content links across texts. The developed test comprised 174 items and was empirically examined in a study with 310 university students. Several items had to be excluded due to psychometric misfit and differential item functioning. The resulting final test contains 67 items within 5 units (i.e., test structures of 2–3 texts and related items) and has been shown to fit a unidimensional IRT Rasch model. The test score showed expected relationships to the final school exam grade, the study level (Bachelor/Master), essay performance, sourcing behavior, as well as mental load and mental effort.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), pp. 183–198. https://doi.org/10.1016/j.learninstruc.2006.03.001

  • Anmarkrud, Ø., Bråten, I., & Strømsø, H. I. (2014). Multiple-documents literacy: Strategic processing, source awareness, and argumentation when reading multiple conflicting documents. Learning and Individual Differences, 30, pp. 64–76. https://doi.org/10.1016/j.lindif.2013.01.007

  • Arbeitskreis Deutscher Qualifikationsrahmen (2011). Deutscher Qualifikationsrahmen für lebenslanges Lernen. Retrieved from http://www.dqr.de/media/content/Der_Deutsche_Qualifikationsrahmen_fue_lebenslanges_Lernen.pdf

  • Blossfeld, H.-P., Roßbach, H.-G., & Maurice, J. v. (2011). Education as a lifelong process: The German National Educational Panel Study (NEPS) [Special Issue]. Zeitschrift für Erziehungswissenschaften, 14.

    Google Scholar 

  • Braasch, J. L. G., & Bråten, I. (2017). The discrepancy-induced source comprehension (D-ISC) model: Basic assumptions and preliminary evidence. Educational Psychologist, 52(3), pp. 167–181. https://doi.org/10.1080/00461520.2017.1323219

  • Braasch, J. L. G., Bråten, I., Strømsø, H. I., & Anmarkrud, Ø. (2014). Incremental theories of intelligence predict multiple document comprehension. Learning and Individual Differences, 31, pp. 11–20. https://doi.org/10.1016/j.lindif.2013.12.012

  • Braasch, J. L. G., Rouet, J.-F., Vibert, N., & Britt, M. A. (2012). Readers’ use of source information in text comprehension. Memory & Cognition, 40(3), pp. 450–465. https://doi.org/10.3758/s13421-011-0160-6

  • Bråten, I., Salmerón, L., & Strømsø, H. I. (2016). Who said that? Investigating the Plausibility-Induced Source Focusing assumption with Norwegian undergraduate readers. Contemporary Educational Psychology, 46, pp. 253–262. https://doi.org/10.1016/j.cedpsych.2016.07.004

  • Bråten, I., Ferguson, L. E., Strømsø, H. I., & Anmarkrud, Ø. (2014). Students working with multiple conflicting documents on a scientific issue: Relations between epistemic cognition while reading and sourcing and argumentation in essays. British Journal of Educational Psychology, 84(1), pp. 58–85. https://doi.org/10.1111/bjep.12005

  • Britt, M. A., & Rouet, J.-F. (2012). Learning with multiple documents: Component skills and their acquisition. In J. R. Kirby & M. J. Lawson (Eds.), Enhancing the quality of learning: Dispositions, instruction, and learning processes (pp. 276–314). New York: Cambridge University Press.

    Google Scholar 

  • Britt, M. A., & Sommer, J. (2004). Facilitating textual integration with macro-structure focusing tasks. Reading Psychology, 25(4), pp. 313–339. https://doi.org/10.1080/02702710490522658

  • Britt, M. A., & Aglinskas, C. (2002). Improving students’ ability to identify and use source information. Cognition and Instruction, 20(4), pp. 485–522. https://doi.org/10.1207/s1532690xci2004_2

  • Britt, M. A., Perfetti, C. A., Sandak, R., & Rouet, J.-F. (1999). Content integration and source separation in learning from multiple texts. In S. R. Goldman, A. C. Graesser, & P. Van den Broek (Eds.), Narrative, comprehension, causality, and coherence: Essays in honor of Tom Trabasso (pp. 209–233). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Cerdán, R., & Vidal-Abarca, E. (2008). The effects of tasks on integrating information from multiple documents. Journal of Educational Psychology, 100(1), pp. 209–222. https://doi.org/10.1037/0022-0663.100.1.209

  • DBV (2009). Standards der Informationskompetenz für Studierende. Retrieved from www.bibliotheksverband.de/fileadmin/user_upload/Kommissionen/Kom_Dienstleistung/Publikationen/Standards_Infokompetenz_03.07.2009_endg.pdf

  • Ferguson, L. E. (2015). Epistemic beliefs and their relation to multiple-text comprehension: A Norwegian program of research. Scandinavian Journal of Educational Research, 59(6), pp. 731–752. https://doi.org/10.1080/00313831.2014.971863

  • Gil, L., Bråten, I., Vidal-Abarca, E., & Strømsø, H. I. (2010a). Summary versus argument tasks when working with multiple documents: Which is better for whom? Contemporary Educational Psychology, 35(3), pp. 157–173. https://doi.org/10.1016/j.cedpsych.2009.11.002

  • Gil, L., Bråten, I., Vidal-Abarca, E., & Strømsø, H. I. (2010b). Understanding and integrating multiple science texts: Summary tasks are sometimes better than argument tasks. Reading Psychology, 31(1), pp. 30–68. https://doi.org/10.1080/02702710902733600

  • Goldhammer, F., Naumann, J., & Keßel, Y. (2013). Assessing individual differences in basic computer skills. Psychometric characteristics of an interactive performance measure. European Journal of Psychological Assessment, 29(4), pp. 263–275. https://doi.org/10.1027/1015-5759/a000153

  • Goldman, S. R., & Scardamalia, M. (2013). Managing, understanding, applying, and creating knowledge in the information age: Next-generation challenges and opportunities. Cognition and Instruction, 31(2), pp. 255–269. https://doi.org/10.1080/10824669.2013.773217

  • Goldman, S. R., Britt, M. A., Brown, W., Cribb, G., George, M., Greenleaf, C., . . . Project, R. (2016). Disciplinary literacies and learning to read for understanding: A conceptual framework for disciplinary literacy. Educational Psychologist, 51(2), pp. 219–246. https://doi.org/10.1080/00461520.2016.1168741

  • Griffin, T. D., Wiley, J., Britt, M. A., & Salas, C. R. (2017). The role of clear thinking in learning science from multiple-document ınquiry tasks. International Electronic Journal of Elementary Education, 5(1), pp. 63–78.

    Google Scholar 

  • Gruenbaum, E. A. (2012). Common literacy struggles with college students: Using the reciprocal teaching technique. Journal of College Reading and Learning, 42(2), pp. 109–116. https://doi.org/10.1080/10790195.2012.10850357

  • Guthrie, J. T., Wigfield, A., Metsala, J. L., & Cox, K. E. (1999). Motivational and cognitive predictors of text comprehension and reading amount. Scientific Studies of Reading, 3(3), pp. 231–256.

    Google Scholar 

  • Hahnel, C., Kroehne, U., Goldhammer, F., Schoor, C., Mahlow, N., & Artelt, C. (2019). Validating process variables of sourcing in an assessment of multiple document comprehension. British Journal of Educational Psychology, 89(3), pp. 524–537. https://doi.org/10.1111/bjep.12278

  • Hahnel, C., Schoor, C., Kröhne, U., Goldhammer, F., Mahlow, N., & Artelt, C. (2019). The role of cognitive load for university students’ comprehension of multiple documents. Zeitschrift für Pädagogische Psychologie, 33(2), pp. 105–118. https://doi.org/10.1024/1010-0652/a000238

  • Homann, B. (2000). Das Dynamische Modell der Informationskompetenz (DYMIK) als Grundlage für bibliothekarische Schulungen. In G. Knorz & R. Kuhlen (Eds.), Informationskompetenz – Basiskompetenz in der Informationsgesellschaft. Proceedings des 7. Internationale Symposiums für Informationswissenschaft (ISI 2000), Darmstadt, 8.–10. November 2000 (pp. 195–206). Konstanz: UVK Verlag.

    Google Scholar 

  • Kammerer, Y., & Gerjets, P. (2014). Quellenbewertungen und Quellenverweise bei Lesen und Zusammenfassen wissensbezogener Informationen aus multiplen Webseiten [Source evaluations and source references when reading and summarizing science-related information from multiple web pages]. Unterrichtswissenschaft, 42(1), pp. 7–23.

    Google Scholar 

  • Kammerer, Y., Kalbfell, E., & Gerjets, P. (2016). Is this information source commercially biased? How contradictions between web pages stimulate the consideration of source information. Discourse Processes, 53(5–6), pp. 430–456. https://doi.org/10.1080/0163853x.2016.1169968

  • Keck, D., Kammerer, Y., & Starauschek, E. (2015). Reading science texts online: Does source information influence the identification of contradictions within texts? Computers & Education, 82, pp. 442–449. https://doi.org/10.1016/j.compedu.2014.12.005

  • Kintsch, W. (1998). Comprehension. A paradigm for cognition. Cambridge: Cambridge University Press.

    Google Scholar 

  • Krell, M. (2015). Evaluating an instrument to measure mental load and mental effort using Item Response Theory. Science Education Review Letters, 2015, pp. 1–6.

    Google Scholar 

  • Kultusministerkonferenz (2012). Bildungsstandards im Fach Deutsch für die Allgemeine Hochschulreife (Beschluss der Kultusministerkonferenz vom 18.10.2012) [Educational standards in the subject German for the general qualification for university entrance (decision of the Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany of 18.10.2012)]. Retrieved from www.kmk.org/fileadmin/veroeffentlichungen_beschluesse/2012/2012_10_18-Bildungsstandards-Deutsch-Abi.pdf

  • Lau, J. (2006). Guidelines on information literacy for lifelong learning. Retrieved from https://www.archive.ifla.org/VII/s42/pub/IL-Guidelines2006.pdf

  • List, A., & Alexander, P. A. (2017). Cognitive affective engagement model of multiple source use. Educational Psychologist, 52(3), pp. 182–199. https://doi.org/10.1080/00461520.2017.1329014

  • Locher, F. M., & Pfost, M. (2019). Erfassung des Lesevolumens in Large-Scale Studien. Diagnostica, 65(1), pp. 26–36. https://doi.org/10.1026/0012-1924/a000203

  • Maier, J., & Richter, T. (2013). Text belief consistency effects in the comprehension of multiple texts with conflicting information. Cognition and Instruction, 31(2), pp. 151–175. https://doi.org/10.1080/07370008.2013.769997

  • Mühlen, S. v. d., Richter, T., Schmid, S., Schmidt, E. M., & Berthold, K. (2016). The use of source-related strategies in evaluating multiple psychology texts: A student–scientist comparison. Reading and Writing, 29(8), pp. 1677–1698. https://doi.org/10.1007/s11145-015-9601-0

  • Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84(4), pp. 429–434.

    Google Scholar 

  • Paul, J., Macedo-Rouet, M., Rouet, J.-F., & Stadtler, M. (2017). Why attend to source information when reading online? The perspective of ninth grade students from two different countries. Computers & Education, 113, pp. 339–354. https://doi.org/10.1016/j.compedu.2017.05.020

  • Perfetti, C. A., Rouet, J.-F., & Britt, M. A. (1999). Toward a theory of documents representation. In H. van Oostendorp & S. R. Goldman (Eds.), The construction of mental representations during reading (pp. 99–122). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Peter, T. (2019, June 18). „Es gibt gravierende Mängel, was die Studierfähigkeit zahlreicher Abiturienten angeht“. Leipziger Volkszeitung. Retrieved from https://www.lvz.de/Nachrichten/Politik/Praesident-der-Hochschulrektorenkonferenz-Es-gibt-gravierende-Maengel-was-die-Studierfaehigkeit-zahlreicher-Abiturienten-angeht

  • Primor, L., & Katzir, T. (2018). Measuring Multiple Text Integration: A Review. Frontiers in Psychology, 9(2294). https://doi.org/10.3389/fpsyg.2018.02294

  • Rölke, H. (2012). The ItemBuilder: A graphical authoring system for complex item development. In T. Bastiaens & G. Marks (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2012 (Vol. 2012, pp. 344–353). Chesapeake, VA: AACE.

    Google Scholar 

  • Rouet, J.-F., Britt, M. A., & Durik, A. M. (2017). RESOLV: Readers’ representation of reading contexts and tasks. Educational Psychologist, 52(3), pp. 200–215. https://doi.org/10.1080/00461520.2017.1329015

  • Rouet, J.-F., Favart, M., Britt, M. A., & Perfetti, C. A. (1997). Studying and using multiple documents in history: Effects of discipline expertise. Cognition and Instruction, 15(1), pp. 85–106. https://doi.org/10.1207/s1532690xci1501_3.

  • Salmerón, L., Gil, L., Bråten, I., & Strømsø, H. (2010). Comprehension effects of signalling relationships between documents in search engines. Computers in Human Behavior, 26(3), pp. 419–426. https://doi.org/10.1016/j.chb.2009.11.013

  • Scharrer, L., & Salmerón, L. (2016). Sourcing in the reading process [Special issue]. Reading and Writing, 29(8).

    Google Scholar 

  • Schmalhofer, F., & Glavanov, D. (1986). Three components of understanding a programmer’s manual: Verbatim, propositional, and situational representations. Journal of Memory and Language, 25(3), pp. 279–294. https://doi.org/10.1016/0749-596x(86)90002-1

  • Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representation. Learning and Instruction, 13(2), pp. 141–156.

    Google Scholar 

  • Schoor, C., Hahnel, C., Artelt, C., Reimann, D., Kröhne, U., & Goldhammer, F. (2020). Entwicklung und Skalierung eines Tests zur Erfassung des Verständnisses multipler Dokumente von Studierenden [Developing and scaling a test of multiple document comprehension in university students]. Diagnostica. https://doi.org/10.1026/0012-1924/a000231

  • Seufert, T. (2009). Lernen mit multiplen Repräsentationen – Gestaltungs- und Verarbeitungsstrategien [Learning with multiple representations – Design and processing strategies]. In R. Plötzner, T. Leuders, & A. Wichert (Eds.), Lernchance Computer. Strategien für das Lernen mit digitalen Medienverbünden (pp. 45–66). Münster: Waxmann.

    Google Scholar 

  • Spinath, B., Stiensmeier-Pelster, J., Schöne, C., & Dickhäuser, O. (2002). SELLMO: Skalen zur Erfassung der Lern- und Leistungsmotivation [Learning and Achievement Motivation Scales]. Göttingen: Hogrefe.

    Google Scholar 

  • Stadtler, M., & Bromme, R. (2014). The content–source integration model: A taxonomic description of how readers comprehend conflicting scientific information. In D. N. Rapp & J. L. Braasch (Eds.), Processing inaccurate information: Theoretical and applied perspectives from cognitive science and the educational sciences (pp. 379–402). Cambridge, MA: MIT Press.

    Google Scholar 

  • Stadtler, M., Scharrer, L., Skodzik, T., & Bromme, R. (2014). Comprehending multiple documents on scientific controversies: Effects of reading goals and signaling rhetorical relationships. Discourse Processes, 51(1–2), pp. 93–116. https://doi.org/10.1080/0163853x.2013.855535

  • Strømsø, H. I., Bråten, I., Britt, M. A., & Ferguson, L. E. (2013). Spontaneous sourcing among students reading multiple documents. Cognition and Instruction, 31(2), pp. 176–203. https://doi.org/10.1080/07370008.2013.769994

  • Sweller, J. (2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22(2), pp. 123–138. https://doi.org/10.1007/s10648-010-9128-5

  • Warm, T. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54(3), pp. 427–450. https://doi.org/10.1007/bf02294627

  • Wineburg, S. S. (1991). Historical problem solving: A study of the cognitive processes used in the evaluation of documentary and pictorial evidence. Journal of Educational Psychology, 83(1), pp. 73–87. https://doi.org/10.1037/0022-0663.83.1.73

Download references

Author information

Authors and Affiliations

  1. Universität Bamberg, Bamberg, Germany

    C. Schoor

  2. DIPF | Leibniz-Institut für Bildungsforschung und Bildungsinformation, Frankfurt am Main, Germany

    C. Hahnel, J. Klagges, U. Kroehne & F. Goldhammer

  3. Leibniz-Institut für Bildungsverläufe, Bamberg, Germany

    N. Mahlow & C. Artelt

Authors
  1. C. Schoor
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. Hahnel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. N. Mahlow
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. Klagges
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. U. Kroehne
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. F. Goldhammer
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. C. Artelt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to C. Schoor .

Editor information

Editors and Affiliations

  1. Johannes Gutenberg University Mainz, Mainz, Germany

    Olga Zlatkin-Troitschanskaia

  2. Humboldt University of Berlin, Berlin, Germany

    Hans Anand Pant

  3. Johannes Gutenberg University Mainz, Mainz, Germany

    Miriam Toepper

  4. Humboldt University of Berlin, Berlin, Germany

    Corinna Lautenbach

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Fachmedien Wiesbaden GmbH, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Schoor, C. et al. (2020). Multiple Document Comprehension of University Students. In: Zlatkin-Troitschanskaia, O., Pant, H.A., Toepper, M., Lautenbach, C. (eds) Student Learning in German Higher Education. Springer VS, Wiesbaden. https://doi.org/10.1007/978-3-658-27886-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-658-27886-1_11

  • Published:

  • Publisher Name: Springer VS, Wiesbaden

  • Print ISBN: 978-3-658-27885-4

  • Online ISBN: 978-3-658-27886-1

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation