Skip to main content
SpringerLink
Log in
Book cover

The Psychology of Digital Learning pp 1–19Cite as

Learning from Multimedia: Cognitive Processes and Instructional Support

  • Chapter
  • First Online:

Abstract

Multimedia materials have become ubiquitously available as educational resources in the digital age. In multimedia learning environments, verbal and pictorial representations are presented jointly for explaining and illustrating a subject matter. In order to benefit from multimedia instruction, learners need to construct a coherent mental model from text and pictures by mentally integrating information from both external sources. In the present chapter, we will first review findings that aim at providing a more thorough description of the cognitive underpinnings of learning from multimedia and of the integration process in particular. Second, we will introduce empirical evidence illustrating the effectiveness of instructional interventions aimed at supporting the integration process in multimedia learning. The interlinking of these two lines of research is seen as an example for contributing to use-inspired basic research.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   119.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Anglin, G. J., Vaez, H., & Cunningham, K. L. (2004). Visual representations and learning: The role of static and animated graphics. In D. Jonassen (Ed.), Handbook of research on educational communications and technology (Vol. 2, pp. 865–916). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Arndt, J., Schüler, A., & Scheiter, K. (2015). Text-picture integration: How delayed testing moderates recognition of pictorial information in multimedia learning. Applied Cognitive Psychology, 29, 702–712.

    Article  Google Scholar 

  • Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and its control processes. In K. W. Spence & J. T. Spence (Eds.), The psychology of learning and motivation (Vol. 2, pp. 89–195). London: Academic Press.

    Google Scholar 

  • Baddeley, A. D. (1999). Essentials of human memory. Hove, UK: Psychology Press.

    Google Scholar 

  • Bartholomé, T., & Bromme, R. (2009). Coherence formation when learning from text and pictures: What kind of support for whom? Journal of Educational Psychology, 101, 282–293.

    Article  Google Scholar 

  • Berthold, K., Eysink, T. H., & Renkl, A. (2009). Assisting self-explanation prompts are more effective than open prompts when learning with multiple representations. Instructional Science, 37, 345–363.

    Article  Google Scholar 

  • Berthold, K., & Renkl, A. (2009). Instructional aids to support a conceptual understanding of multiple representations. Journal of Educational Psychology, 101, 70–87.

    Article  Google Scholar 

  • Bodemer, D., & Faust, U. (2006). External and mental referencing of multiple representations. Computers in Human Behavior, 22, 27–42.

    Article  Google Scholar 

  • Butcher, K. R. (2006). Learning from text with diagrams: Promoting mental model development and inference generation. Journal of Educational Psychology, 98, 182–197.

    Article  Google Scholar 

  • Cierniak, G., Scheiter, K., & Gerjets, P. (2009). Explaining the split-attention effect: Is the reduction of extraneous cognitive load accompanied by an increase in germane cognitive load? Computers in Human Behavior, 25, 315–324.

    Article  Google Scholar 

  • De Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Paas, F. (2009). Towards a framework for attention cueing in instructional animations: Guidelines for research and design. Educational Psychology Review, 21, 113–140.

    Article  Google Scholar 

  • Eitel, A. (2016). How repeated studying and testing affects multimedia learning: Evidence for adaptation to task demands. Learning and Instruction, 41, 70–84.

    Article  Google Scholar 

  • Eitel, A., Scheiter, K., & Schüler, A. (2012). The time course of information extraction from instructional diagrams. Perceptual and Motor Skills, 115, 677–701.

    Article  Google Scholar 

  • Eitel, A., Scheiter, K., & Schüler, A. (2013). How inspecting a picture affects processing of text in multimedia learning. Applied Cognitive Psychology, 27, 451–461.

    Google Scholar 

  • Eitel, A., Scheiter, K., Schüler, A., Nyström, M., & Holmqvist, K. (2013). How a picture facilitates the process of learning from text: Evidence for scaffolding. Learning and Instruction, 28, 48–63.

    Google Scholar 

  • Gentner, D., & Loftus, E. F. (1979). Integration of verbal and visual information as evidenced by distortions in picture memory. The American Journal of Psychology, 92, 363–375.

    Article  Google Scholar 

  • Ginns, P. (2006). Integrating information: A meta-analysis of the spatial contiguity and temporal contiguity effects. Learning and Instruction, 16, 511–525.

    Article  Google Scholar 

  • Glenberg, A., & Langston, W. E. (1992). Comprehension of illustrated text: Pictures help to build mental models. Journal of Memory and Language, 31, 129–151.

    Article  Google Scholar 

  • Gollwitzer, P. M. (1999). Implementation intentions: Strong effects of simple plans. American Psychologist, 54, 493–503.

    Article  Google Scholar 

  • Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta-analysis of effects and processes. In M. P. Zanna (Ed.), Advances in experimental social psychology (Vol. 38, pp. 69–119). San Diego, CA: Elsevier Academic Press.

    Google Scholar 

  • Greene, M. R., & Oliva, A. (2009). The briefest of glances: The time course of natural scene understanding. Psychological Science, 20, 464–472.

    Article  Google Scholar 

  • Hannus, M., & Hyönä, J. (1999). Utilization of illustrations during learning of science textbook passages among low- and high-ability children. Contemporary Educational Psychology, 24, 95–123.

    Article  Google Scholar 

  • Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32, 717–742.

    Article  Google Scholar 

  • Jamet, E. (2014). An eye-tracking study of cueing effects in multimedia learning. Computers in Human Behavior, 32, 47–53.

    Article  Google Scholar 

  • Jarodzka, H., Balslev, T., Holmqvist, K., Nyström, M., Scheiter, K., Gerjets, P., & Eika, B. (2012). Conveying clinical reasoning based on visual observation via eye-movement modelling examples. Instructional Science, 40, 813–827.

    Article  Google Scholar 

  • Jarodzka, H., van Gog, T., Dorr, M., Scheiter, K., & Gerjets, P. (2013). Learning to see: Guiding students' attention via a model's eye movements fosters learning. Learning and Instruction, 25, 62–70.

    Article  Google Scholar 

  • Johnson, C. I., & Mayer, R. E. (2012). An eye movement analysis of the spatial contiguity effect in multimedia learning. Journal of Experimental Psychology: Applied, 18, 178–191.

    Google Scholar 

  • Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science of language, inference, and consciousness. Cambridge: Cambridge University Press.

    Google Scholar 

  • Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329–354.

    Article  Google Scholar 

  • Kombartzky, U., Plötzner, R., Schlag, S., & Metz, B. (2010). Developing and evaluating a strategy for learning from animations. Learning and Instruction, 20, 424–433.

    Article  Google Scholar 

  • Levin, J. R., Anglin, G. J., & Carney, R. N. (1987). On empirically validating functions of pictures in prose. In D. M. Willows & H. A. Houghton (Eds.), The psychology of illustration (Vol. 1, pp. 51–85). New York: Springer.

    Chapter  Google Scholar 

  • Mason, L., Pluchino, P., & Tornatora, M. C. (in press). Using eye-tracking technology as an indirect instruction tool to improve text and picture processing and learning. British Journal of Educational Technology. doi:10.1111/bjet.12271.

  • Mason, L., Pluchino, P., & Tornatora, M. C. (2015). Eye-movement modeling of integrative reading of an illustrated text: Effects on processing and learning. Contemporary Educational Psychology, 41, 172–187.

    Google Scholar 

  • Mason, L., Pluchino, P., & Tornatora, M. C. (2013). Effects of picture labeling on science text processing and learning: Evidence from eye movements. Reading Research Quarterly, 48, 199–214.

    Google Scholar 

  • Mason, L., Tornatora, M. C., & Pluchino, P. (2013). Do fourth graders integrate text and picture in processing and learning from an illustrated science text? Evidence from eye-movement patterns. Computers & Education, 60, 95–109.

    Google Scholar 

  • Mason, L., Tornatora, M. C., & Pluchino, P. (2015). Integrative processing of verbal and graphical information during re-reading predicts learning from illustrated text: An eye-movement study. Reading and Writing, 28, 851–872.

    Google Scholar 

  • Mayer, R. E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32, 1–19.

    Article  Google Scholar 

  • Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York, NY: Cambridge University Press.

    Book  Google Scholar 

  • McNamara, T. P., Halpin, J. A., & Hardy, J. K. (1992). The representation and integration in memory of spatial and nonspatial information. Memory & Cognition, 20, 519–532.

    Article  Google Scholar 

  • O’Keefe, P. A., Letourneau, S. M., Homer, B. D., Schwartz, R. N., & Plass, J. L. (2014). Learning from multiple representations: An examination of fixation patterns in a science simulation. Computers in Human Behavior, 35, 234–242.

    Article  Google Scholar 

  • Oliva, A., & Torralba, A. (2001). Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision, 42, 145–175.

    Article  Google Scholar 

  • Ozcelik, E., Arslan-Ari, I., & Cagiltay, K. (2010). Why does signaling enhance multimedia learning? Evidence from eye movements. Computers in Human Behavior, 26, 110–117.

    Article  Google Scholar 

  • Ozcelik, E., Karakus, T., Kursun, E., & Cagiltay, K. (2009). An eye-tracking study of how color coding affects multimedia learning. Computers & Education, 53, 445–453.

    Article  Google Scholar 

  • Paivio, A. (1991). Dual coding theory: Retrospect and current status. Canadian Journal of Psychology, 45, 255–287.

    Article  Google Scholar 

  • Renkl, A., & Scheiter, K. (in press). Studying visual displays: How to instructionally support learning. Educational Psychology Review. doi:10.1007/s10648-015-9340-4.

  • Rey, G. D. (2012). How seductive are decorative elements in learning material? Journal of Educational Multimedia and Hypermedia, 21, 257–283.

    Google Scholar 

  • Richter, J., Scheiter, K., & Eitel, A. (2016). Signaling text-picture relations in multimedia learning: A comprehensive meta-analysis. Educational Research Review, 17, 19–36.

    Article  Google Scholar 

  • Rummer, R., Schweppe, J., Scheiter, K., & Gerjets, P. (2008). Lernen mit Multimedia: Die kognitiven Grundlagen des Modalitätseffekts. Psychologische Rundschau, 59, 98–107.

    Google Scholar 

  • Scheiter, K., & Eitel, A. (2015). Signals foster multimedia learning by supporting integration of highlighted text and diagram elements. Learning and Instruction, 36, 11–26.

    Article  Google Scholar 

  • Schmidt-Weigand, F., Kohnert, A., & Glowalla, U. (2010). A closer look at split visual attention in system-and self-paced instruction in multimedia learning. Learning and Instruction, 20, 100–110.

    Article  Google Scholar 

  • Schmidt-Weigand, F., & Scheiter, K. (2011). The role of spatial descriptions in learning from multimedia. Computers in Human Behavior, 27, 22–28.

    Article  Google Scholar 

  • Schnotz, W. (2002). Commentary: Towards an integrated view of learning from text and visual displays. Educational Psychology Review, 14, 101–120.

    Article  Google Scholar 

  • Schnotz, W. (2005). An integrated model of text and picture comprehension. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 49–69). New York, NY: Cambridge University Press.

    Chapter  Google Scholar 

  • Schnotz, W., Ludewig, U., Ullrich, M., Horz, H., McElvany, N., & Baumert, J. (2014). Strategy shifts during learning from texts and pictures. Journal of Educational Psychology, 106, 974–989.

    Article  Google Scholar 

  • Schubert, C., Scheiter, K., & Schüler, A. (2014). Modeling eye movements to support multimedia learning. Poster presented at the Meeting of the EARLI SIG2 Text and Graphics Comprehension. Rotterdam, The Netherlands.

    Google Scholar 

  • Schüler, A., Arndt, J., & Scheiter, K. (2015). Processing multimedia material: Does integration of text and pictures result in a single or two interconnected mental representations? Learning and Instruction, 35, 62–72.

    Article  Google Scholar 

  • Schüler, A., Scheiter, K., & Gerjets, P. (2013). Is spoken text always better? Investigating the modality and redundancy effect with longer text presentation. Computers in Human Behavior, 29, 1590–1601.

    Article  Google Scholar 

  • Schüler, A., Scheiter, K., & van Genuchten, E. (2011). The role of working memory in multimedia instruction: Is working memory working during learning from text and pictures? Educational Psychology Review, 23, 389–411.

    Article  Google Scholar 

  • Schwan, S. (this volume). Digital pictures, videos, and beyond: Knowledge acquisition with realistic images.

    Google Scholar 

  • Serra, M. J., & Dunlosky, J. (2010). Metacomprehension judgements reflect the belief that diagrams improve learning from text. Memory, 18, 698–711.

    Article  Google Scholar 

  • Seufert, T. (2003). Supporting coherence formation in learning from multiple representations. Learning and Instruction, 13, 227–237.

    Article  Google Scholar 

  • Stalbovs, K., Scheiter, K., & Gerjets, P. (2015). Implementation intentions during multimedia learning: Using if-then plans to facilitate cognitive processing. Learning and Instruction, 35, 1–15.

    Article  Google Scholar 

  • Stokes, D. E. (1997). Pasteur’s quadrant—Basic science and technological innovation. Washington, DC: Brookings Institution Press.

    Google Scholar 

  • Ullrich, M., Schnotz, W., Horz, H., McElvany, N., Schroeder, S., & Baumert, J. (2015). Kognitionspsychologische Aspekte eines Kompetenzmodells zur Bild-Text-Integration. Psychologische Rundschau, 63, 11–17.

    Article  Google Scholar 

  • Van Gog, T. (2014). The signaling (or cueing) principle in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 263–278). New York: Cambridge University Press.

    Google Scholar 

  • Veenman, M. V. J., Hout-Wolters, B. H. A. M., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1, 3–14.

    Article  Google Scholar 

  • Zahn, C. (this volume). Digital design and learning.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Leibniz-Institut für Wissensmedien, Schleichstraße 6, 72076, Tübingen, Germany

    Katharina Scheiter & Anne Schüler

  2. Albert-Ludwigs-Universität Freiburg, Institut für Psychologie, Engelbergerstraße 41, 79085, Freiburg, Germany

    Alexander Eitel

Authors
  1. Katharina Scheiter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anne Schüler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Eitel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Katharina Scheiter .

Editor information

Editors and Affiliations

  1. Realistic Depictions, Leibniz-Institut für Wissensmedien, Tübingen, Germany

    Stephan Schwan

  2. Knowledge Construction, Leibniz-Institut für Wissensmedien, Tübingen, Germany

    Ulrike Cress

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Scheiter, K., Schüler, A., Eitel, A. (2017). Learning from Multimedia: Cognitive Processes and Instructional Support. In: Schwan, S., Cress, U. (eds) The Psychology of Digital Learning. Springer, Cham. https://doi.org/10.1007/978-3-319-49077-9_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-49077-9_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-49075-5

  • Online ISBN: 978-3-319-49077-9

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation