Skip to main content
SpringerLink
Log in

The Modality Effect

  • Chapter
  • First Online:
Cognitive Load Theory

Abstract

The modality effect is closely related to the split-attention effect (Chapter 9). According to cognitive load theory, the split-attention effect occurs when learners must process separate but related sources of information that cannot be understood without mental integration. The cognitive resources required to effect this integration are unavailable for learning and may exceed the available capacity of working memory. This chapter describes an alternative way of dealing with split-attention conditions by engaging both auditory and visual channels of information in working memory rather than just the visual channel. For example, rather than presenting a diagram and written text that rely entirely on the visual channel, a diagram and spoken text relying on both auditory and visual modalities are used.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology, 94, 416–427.

    Article  Google Scholar 

  • Atkinson, R. K., Mayer, R. E., & Merrill, M. M. (2005). Fostering social agency in multimedia learning: Examining the impact of an animated agent’s voice. Contemporary Educational Psychology, 30, 117–139.

    Article  Google Scholar 

  • Baddeley, A. (1986). Working memory. New York: Oxford University Press.

    Google Scholar 

  • Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs: Prentice-Hall.

    Google Scholar 

  • Brünken, R., Plass, J. L., & Leutner, D. (2004). Assessment of cognitive load in multimedia learning with dual-task methodology: Auditory load and modality effects. Instructional Science, 32, 115–132.

    Article  Google Scholar 

  • Clark, R. C., & Mayer, R. E. (2003). E-learning and the science of instruction. San Francisco: Jossey-Bass/Pfeiffer.

    Google Scholar 

  • De Westelinck, K., Valcke, M., De Craene, B., & Kirschner, P. (2005). Multimedia learning in social sciences: Limitations of external graphical representations. Computers in Human Behavior, 21, 555–573.

    Article  Google Scholar 

  • Dillon, A., & Jobst, J. (2005). Multimedia learning with hypermedia. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 569–588). New York: Cambridge University Press.

    Google Scholar 

  • Dunsworth, Q., & Atkinson, R. K. (2007). Fostering multimedia learning of science: Exploring the role of an animated agent’s image. Computers and Education, 49, 677–690.

    Article  Google Scholar 

  • Dutke, S., & Rinck, M. (2006). Multimedia learning: Working memory and the learning of word and picture diagrams. Learning and Instruction, 16, 526–537.

    Article  Google Scholar 

  • Gerjets, P., Scheiter, K., Opfermann, M., Hesse, F. W., & Eysink, T. H. S. (2009). Learning with hypermedia: The influence of representational formats and different levels of learner control on performance and learning behavior. Computers in Human Behavior, 25, 360–370.

    Article  Google Scholar 

  • Gerjets, P., Scheiter, K., & Schuh, J. (2008). Information comparisons in example-based hypermedia environments: Supporting learners with processing prompts and an interactive comparison tool. Educational Technology Research and Development, 56, 73–92.

    Article  Google Scholar 

  • Ginns, P. (2005b). Meta-analysis of the modality effect. Learning and Instruction, 15, 313–331.

    Article  Google Scholar 

  • Goolkasian, P. (2000). Pictures, words, and sounds: From which format are we best able to reason? The Journal of General Psychology, 127, 439–459.

    Article  Google Scholar 

  • Harskamp, E. G., Mayer, R. E., & Suhre, C. (2007). Does the modality principle for multimedia learning apply to science classrooms? Learning and Instruction, 17, 465–477.

    Article  Google Scholar 

  • Jeung, H.-J., Chandler, P., & Sweller, J. (1997). The role of visual indicators in dual sensory mode instruction. Educational Psychology, 17, 329–343.

    Article  Google Scholar 

  • Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351–371.

    Article  Google Scholar 

  • Kalyuga, S., Chandler, P., & Sweller, J. (2000). Incorporating learner experience into the design of multimedia instruction. Journal of Educational Psychology, 92, 126–136.

    Article  Google Scholar 

  • Lowe, R. K. (1999). Extracting information from an animation during complex visual learning. European Journal of Psychology of Education, 14, 225–244.

    Article  Google Scholar 

  • Mayer, R. E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32, 1–19.

    Article  Google Scholar 

  • Mayer, R. E. (Ed.). (2005). Cambridge handbook of multimedia learning. New York: Cambridge University Press.

    Google Scholar 

  • Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge University Press.

    Google Scholar 

  • Mayer, R. E., & Anderson, R. B. (1991). Animations need narrations: An experimental test of a dual-coding hypothesis. Journal of Educational Psychology, 83, 484–490.

    Article  Google Scholar 

  • Mayer, R. E., & Anderson, R. B. (1992). The instructive animation: Helping students build connections between words and pictures in multimedia learning. Journal of Educational Psychology, 84, 444–452.

    Article  Google Scholar 

  • Mayer, R. E., Dow, G. T., & Mayer, S. (2003). Multimedia learning in an interactive self-explaining environment: What works in the design of agent-based microworlds? Journal of Educational Psychology, 95, 806–812.

    Article  Google Scholar 

  • Mayer, R. E., & Johnson, C. I. (2008). Revising the redundancy principle in multimedia learning. Journal of Educational Psychology, 100, 380–386.

    Article  Google Scholar 

  • Mayer, R. E., & Moreno, R. (1998). A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90, 312–320.

    Article  Google Scholar 

  • Mayer, R. E., & Moreno, R. (2002). Aids to computer-based multimedia learning. Learning and Instruction, 12, 107–119.

    Article  Google Scholar 

  • Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38, 43–52.

    Article  Google Scholar 

  • Mayer, R. E., Sobko, K., & Mautone, P. D. (2003). Social cues in multimedia learning: Role of speaker’s voice. Journal of Educational Psychology, 95, 419–425.

    Article  Google Scholar 

  • Moreno, R., & Durán, R. (2004). Do multiple representations need explanations? The role of verbal guidance and individual differences in multimedia mathematics learning. Journal of Educational Psychology, 96, 492–503.

    Article  Google Scholar 

  • Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91, 358–368.

    Article  Google Scholar 

  • Moreno, R., Mayer, R. E., Spires, H. A., & Lester, J. C. (2001). The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19, 177–213.

    Article  Google Scholar 

  • Mousavi, S. Y., Low, R., & Sweller, J. (1995). Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology, 87, 319–334.

    Article  Google Scholar 

  • O’Neil, H. F., & Chuang, S. S. (2007). An audio/text after-action review in a computer-based collaborative problem solving task. Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing (CRESST). University of California.

    Google Scholar 

  • Penney, C. G. (1989). Modality effects and the structure of short-term verbal memory. Memory and Cognition, 17, 398–422.

    Article  Google Scholar 

  • Schneider, W., & Detweiler, M. (1987). A connectionist/control architecture for working memory. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 21, pp. 53–119). New York: Academic.

    Google Scholar 

  • Schnotz, W. (2005). An integrated model of text and picture comprehension. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 49–69). Cambridge, UK: Cambridge University Press.

    Google Scholar 

  • Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representation. Learning and Instruction, 13, 141–156.

    Article  Google Scholar 

  • Tabbers, H. K., Martens, R. L., & van Merriënboer, J. J. G. (2004). Multimedia instructions and cognitive load theory: Effects of modality and cueing. The British Journal of Educational Psychology, 74, 71–81.

    Article  Google Scholar 

  • Tindall-Ford, S., Chandler, P., & Sweller, J. (1997). When two sensory modes are better than one. Journal of Experimental Psychology: Applied, 3, 257–287.

    Article  Google Scholar 

  • Wouters, P., Paas, F., & van Merriënboer, J. J. G. (2009). Observational learning from animated models: Effects of modality and reflection on transfer. Contemporary Educational Psychology, 34, 1–8.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Education, University of New South Wales, Sydney, NSW, Australia, 2052

    John Sweller, Paul Ayres & Slava Kalyuga

Authors
  1. John Sweller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul Ayres
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Slava Kalyuga
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John Sweller .

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Sweller, J., Ayres, P., Kalyuga, S. (2011). The Modality Effect. In: Cognitive Load Theory. Explorations in the Learning Sciences, Instructional Systems and Performance Technologies, vol 1. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-8126-4_10

Download citation

Publish with us

Policies and ethics

Search

Navigation