Skip to main content
SpringerLink
Log in
Book cover

Understanding Multimedia Documents pp 249–265Cite as

How Should We Evaluate Multimedia Learning Environments?

  • Chapter

Abstract

Early research with multimedia environments questioned whether these environments are effective in supporting learning. More recently it has been acknowledged that this question should really be about the specific conditions and reasons why multimedia is effective. However, while the argument has become more sophisticated, the techniques for evaluating learning with multimedia environments have not always followed suit. The dominant approach at present involves factorial designs with novices as participants, learning something for a short period of time with outcomes tested by an immediate pen and paper post-test. In this chapter, the positive aspects of this approach are reviewed, but it will be argued that such an approach limits the questions that can be answered. Four important such questions about learning with multimedia are proposed and then the chapter describes a range of methodologies that can be used to answer them.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183–198.

    Article  Google Scholar 

  • Ainsworth, S. E., Bibby, P., & Wood, D. (2002). Examining the effects of different multiple representational systems in learning primary mathematics. Journal of the Learning Sciences, 11(1), 25–61.

    Article  Google Scholar 

  • Ainsworth, S. E., & Loizou, A. T. (2003). The effects of self-explaining when learning with text or diagrams. Cognitive Science, 27(4), 669–681.

    Article  Google Scholar 

  • Buckley, B. C. (2000). Interactive multimedia and model-based learning in biology. International Journal of Science Education, 22(9), 895–935.

    Article  Google Scholar 

  • Chandler, P., & Sweller, J. (1992). The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology, 62, 233–246.

    Google Scholar 

  • Card, S., Moran, T., & Newell, A. (1983). The psychology of human-computer interaction. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Cheng, P. C. H. (2002). Electrifying diagrams for learning: principles for complex representational systems. Cognitive Science, 26(6), 685–736.

    Article  Google Scholar 

  • Daily, L. Z., Lovett, M. C., & Reder, L. M. (2001). Modelling individual differences in working memory performance: a source activation account. Cognitive Science, 25(3), 315–353.

    Article  Google Scholar 

  • de Jong, T., Ainsworth, S. E., Dobson, M., van der Hulst, A., Levonen, J., Reimann, P., et al. (1998). Acquiring knowledge in science and math: The use of multiple representations in technology based learning environments. In M. W. Van Someren, P. Reimann, H. P. A. Boshuizen, & T. de Jong (Eds.), Learning with multiple representations (pp. 9–40). Amsterdam: Elsevier Science.

    Google Scholar 

  • diSessa, A. A. (2004). Metarepresentation: Native competence and targets for instruction. Cognition and Instruction, 22(3), 293–331.

    Article  Google Scholar 

  • Goldman, S. R. (2003). Learning in complex domains: When and why do multiple representations help? Learning and Instruction, 13(2), 239–244.

    Article  Google Scholar 

  • Guercin, F. (2001). Can children process complex information from different media? In J.-F. Rouet, J. L. Levonen, & A. Biardeau (Eds.), Multimedia learning (pp. 59–64). Amsterdam: Pergamon.

    Google Scholar 

  • Howe, C., Tolmie, A., Anderson, A., & Mackenzie, M. (1992). Conceptual knowledge in physics: The role of group interaction in computer-supported teaching. Learning and Instruction, 2, 161–183.

    Article  Google Scholar 

  • Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13(4), 351–371.

    Article  Google Scholar 

  • Klein, P. D. (2003). Rethinking the multiplicity of cognitive resources and curricular representations: alternatives to ‘learning styles’ and ‘multiple intelligences’. Journal of Curriculum Studies, 35(1), 45–81.

    Google Scholar 

  • Kozma, R., Chin, E., Russell, J., & Marx, N. (2000). The roles of representations and tools in the chemistry laboratory and their implications for chemistry learning. Journal of the Learning Sciences, 9(2), 105–143.

    Article  Google Scholar 

  • Lane, P. C. R., Cheng, P. C. -H., & Gobet, F. (2000). CHREST+: A simulation of how humans learn to solve problems using diagrams. AISB Quarterly, 103, 24–30.

    Google Scholar 

  • Lewalter, D. (2003). Cognitive strategies for learning from static and dynamic visuals. Learning and Instruction, 13(2), 177–189.

    Article  Google Scholar 

  • Lowe, R. K. (2003). Animation and learning: selective processing of information in dynamic graphics. Learning and Instruction, 13(2), 157–176.

    Article  Google Scholar 

  • Mayer, R. E. (2001). Multimedia Learning. Cambridge: Cambridge University Press.

    Google Scholar 

  • Mayer, R. E. (2003). The promise of multimedia learning: Using the same instructional design methods across different media. Learning and Instruction, 13(2), 125–139.

    Article  Google Scholar 

  • Mayer, R. E., & Gallini, J. K. (1990). When is an illustration worth ten thousand words? Journal of Educational Psychology, 82(4), 715–726.

    Article  Google Scholar 

  • Mayer, R. E., & Moreno, R. (1998). Split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90(2), 312–320.

    Article  Google Scholar 

  • Mayer, R. E., & Sims, V. K. (1994). For whom is a picture worth 1000 Words – extensions of a dual-coding theory of multimedia learning. Journal of Educational Psychology, 86(3), 389–401.

    Article  Google Scholar 

  • Moreno, R., & Mayer, R. E. (2000). A coherence effect in multimedia learning: The case for minimizing irrelevant sounds in the design of multimedia instructional messages. Journal of Educational Psychology, 92(1), 117–125.

    Article  Google Scholar 

  • Najjar, L. (1998). Principles of educational multi-media user interface design. Human Factors, 40(2), 311–323.

    Article  Google Scholar 

  • Pane , J. F., Corbett , A. T., & John , B. E. (1996). Assessing dynamics in computer-based instruction., Proceedings of ACM CHI'96 Conference on Human Factors in Computing Systems. Vancouver.

    Google Scholar 

  • Price , S. J. (2002). Diagram representation: The cognitive basis for understanding animation in education (Technical Report CSRP 553): School of Computing and Cognitive Sciences, University of Sussex.

    Google Scholar 

  • Reimann, P. (2003). Multimedia learning: beyond modality. Learning and Instruction, 13(2), 245–252.

    Article  Google Scholar 

  • Rieber, L. P. (1990). Animation in Computer-Based Instruction. Educational Technology Research and Development, 38(1), 77–86.

    Article  Google Scholar 

  • Roberts, M. J., Gilmore, D. J., & Wood, D. J. (1997). Individual differences and strategy selection in reasoning. British Journal of Psychology, 88, 473–492.

    Google Scholar 

  • Robson, C.(2002) Real world research. Oxford: Blackwell Publishing.

    Google Scholar 

  • Romero , P., du Boulay , B., Lutz , R., & Cox , R. (2003). The effects of graphical and textual visualisations in multi-representational debugging environments., Proceedings of 2003 IEEE Symposia on Human Centric Computing Languages and Environments.

    Google Scholar 

  • Roth, W.-M., & Bowen, G. M. (2001). Professionals read graphs: A semiotic analysis. Journal for Research in Mathematics Education, 32, 159–194.

    Article  Google Scholar 

  • Rouet, J.-F. & Passerault, J.-M. (1999). Analyzing learner-hypermedia interaction: An overview of online methods. Instructional Science, 27(3/4), 201–219.

    Article  Google Scholar 

  • Schnotz, W. (2001). Sign systems, technologies and the acquisition of knowledge. In J.-F. Rouet, J. J. Levonen & A. Biardeau (Eds.), Multimedia learning: Cognitive and instructional Issues (pp. 9–30). Amsterdam: Pergamon.

    Google Scholar 

  • Schoenfeld, A. H., Smith, J. P., & Arcavi, A. (1993). Learning: The microgenetic analysis of one student's evolving understanding of a complex subject matter domain. In R. Glaser (Ed.), Advances in instructional psychology (Vol. volume 4). Hillsdale, NJ: LEA.

    Google Scholar 

  • Seufert, T. (2003). Supporting coherence formation in learning from multiple representations. Learning and Instruction, 13(2), 227–237.

    Article  Google Scholar 

  • Snow, R. E., & Yalow, E. (1982). Education and intelligence. In R. J. Stenberg (Ed.), A handbook of human intelligence (pp. 493–585). Cambridge: Cambridge University Press.

    Google Scholar 

  • Siegler, R. S. (1995). How does change occur: A microgenetic study of number conservation. Cognitive Psychology, 25, 225–273.

    Google Scholar 

  • Stern, E., Aprea, C., & Ebner, H. G. (2003). Improving cross-content transfer in text processing by means of active graphical representation. Learning and Instruction, 13(2), 191–203.

    Article  Google Scholar 

  • Sweller, J., van Merrienboer, J. J. G., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296...

    Article  Google Scholar 

  • Tabachneck-Schijf, H. J. M., Leonardo, A. M., & Simon, H. A. (1997). CaMeRa: A computational model of multiple representations. Cognitive Science, 21(3), 305–350.

    Article  Google Scholar 

  • Tabbers, H. K., Martens, R. L., & Van Merriënboer, J. J. G. (2001). The modality effect in multimedia instructions. In J. D. Moore & K. Stenning (Eds.), Proceedings of the 23rd annual conference of the Cognitive Science Society (pp. 1024–1029). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Tsui, C. -Y. & Treagust, D. F. (2003). Genetics reasoning with multiple external representations. Research in Science Education, 33, 111–135.

    Google Scholar 

  • Van Someren, M. W., & Tabbers, H. (1998). The role of prior knowledge qualitative knowledge in inductive learning. In M. W. Van Someren, P. Reimann, H. P. A. Boshuizen, & T. de Jong (Eds.), Learning with multiple representations (pp. 102–119). Amsterdam: Pergamon.

    Google Scholar 

  • Waldrip, B. G., & Prain, V. (2004, April). Enhancing learning through using multi-modal representations of concepts. A paper presented at the Annual Meeting of the American Educational Research Association, San Diego.

    Google Scholar 

  • Winn, B. (1987). Charts, graphs and diagrams in educational materials. In D. M. Willows & H. A. Houghton (Eds.), The psychology of illustration: I. Basic research (pp. 152–198). New York: Springer.

    Google Scholar 

  • Yerushalmy, M. (1991). Student perceptions of aspects of algebraic function using multiple representation software. Journal of Computer Assisted Learning, 7, 42–57.

    Article  Google Scholar 

  • Zahn, C., Barquero, B., &, S. (2004). Learning with hyperlinked videos – design criteria and efficient strategies for using audiovisual hypermedia. Learning and Instruction, 14(3), 275–291.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Psychology and Learning Sciences Research Institute, University of Nottingham, University Park, Nottingham, NG7 2RD, UK

    Shaaron Ainsworth

Authors
  1. Shaaron Ainsworth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaaron Ainsworth .

Editor information

Editors and Affiliations

  1. University of Poitiers, France

    Jean-François Rouet

  2. Curtin University, Perth, WA, Australia

    Richard Lowe

  3. University of Landau, Germany

    Wolfgang Schnotz

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Ainsworth, S. (2008). How Should We Evaluate Multimedia Learning Environments?. In: Rouet, JF., Lowe, R., Schnotz, W. (eds) Understanding Multimedia Documents. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-73337-1_13

Download citation

Publish with us

Policies and ethics

Search

Navigation