Multimedia Effect in Problem Solving: a Meta-analysis

  • Liru HuEmail author
  • Gaowei Chen
  • Pengfei Li
  • Jing Huang


Pictures are commonly used to represent problems. However, it remains unclear how the addition of pictures affects students’ problem-solving performance. The multimedia effect in problem solving describes the phenomenon whereby an individual’s problem-solving performance is enhanced when equivalent pictures are added to illustrate or replace part of the problem text. Using meta-analytic techniques, this study seeks to determine the overall size of the multimedia effect in problem solving and its boundary conditions (k = 40, N = 38,570). The results demonstrated a significant small-to-medium-sized multimedia effect on response accuracy (Hedges’ g = 0.25) and a significant medium-sized multimedia effect on students’ response certainty (Hedges’ g = 0.48), but no significant multimedia effect on response time. Function types of pictures significantly moderated the multimedia effect on response accuracy. Yet, only organizational pictures had a significant positive impact on response accuracy (Hedges’ g = 0.46), while representational, informational or decorative pictures did not produce any significant effects on an individual’s response accuracy. Problem difficulty was another significant moderator. The addition of pictures significantly improved students’ response accuracy on difficult problems (Hedges’ g = 0.17), whereas the significant effect all but disappeared for easy problems. These findings suggest that the multimedia effect in problem solving is diversified and limited by multiple boundary conditions. More primary studies are needed to further investigate the multimedia effect in problem solving.


Representation Multimedia effect Visual Picture Problem solving Testing 



We would like to thank Ms. Xiao, Nan in the University of Hong Kong for her precious comments on an earlier version of the manuscript.

Funding Information

This research was supported by Hong Kong RGC grant no. 27606915 and Hong Kong PICO grant no. 2017.A8.073.18C.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.


  1. Agathangelou, S., Gagatsis, A., & Papakosta, V. (2008). The role of verbal description, representational and decorative picture in mathematical problem solving. In A. Gagatsis (Ed.), Research in Mathematics education: conference of five cities: Nicosia, Rhodes, Bologna, Palermo, Locarno (pp. 39–56). Cyprus: University of Cyprus.Google Scholar
  2. Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183–198. Scholar
  3. Baddeley, A. D. (1992). Working memory. Science, 255(5044), 556–559.CrossRefGoogle Scholar
  4. Baddeley, A. D., Allen, R. J., & Hitch, G. J. (2011). Binding in visual working memory: the role of the episodic buffer. Neuropsychologia, 49(6), 1393–1400. Scholar
  5. Baujat, B., Mahé, C., Pignon, J. P., & Hill, C. (2002). A graphical method for exploring heterogeneity in meta-analyses: application to a meta-analysis of 65 trials. Statistics in Medicine, 21(18), 2641–2652.CrossRefGoogle Scholar
  6. Berends, I. E., & van Lieshout, E. C. D. M. (2009). The effect of illustrations in arithmetic problem-solving: effects of increased cognitive load. Learning and Instruction, 19(4), 345–353. Scholar
  7. Brase, G. L. (2008). Pictorial representations in statistical reasoning. Applied Cognitive Psychology, 23(3), 369–381.Google Scholar
  8. Beveridge, M., & Parkins, E. (1987). Visual representation in analogical problem solving. Memory & Cognition, 15(3), 230–237.CrossRefGoogle Scholar
  9. Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: beliefs, techniques, and illusions. Annual Review of Psychology, 64(1), 417–444. Scholar
  10. Bodner, M. G., & Domin, D. S. (2000). Mental models: the role of representations in problem solving in chemistry. University Chemistry Education, 4(1), 24–30.Google Scholar
  11. Boonen, A. J. H., van Wesel, F., Jolles, J., & van der Schoot, M. (2014). The role of visual representation type, spatial ability, and reading comprehension in word problem solving: an item-level analysis in elementary school children. International Journal of Educational Research, 68, 15–26.CrossRefGoogle Scholar
  12. Butcher, K. R. (2014). The multimedia principle. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 174–205). Cambridge: Cambridge University Press. Scholar
  13. Canham, M., & Hegarty, M. (2010). Effects of knowledge and display design on comprehension of complex graphics. Learning and Instruction, 20(2), 155–166.
  14. Carney, R., & Levin, J. (2002). Pictorial illustrations still improve students’ learning from text. Educational Psychology Review, 14(1), 5–26. Scholar
  15. Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8(4), 293–332.CrossRefGoogle Scholar
  16. Chen, O., Kalyuga, S., & Sweller, J. (2015). The worked example effect, the generation effect, and element interactivity. Journal of Educational Psychology, 107(3), 689–704. Scholar
  17. Chen, O., Kalyuga, S., & Sweller, J. (2016). Relations between the worked example and generation effects on immediate and delayed tests. Learning and Instruction, 45, 20–30. Scholar
  18. Chuah, J., Zhang, J., & Johnson, T. R. (2000). The representational effect in complex systems: a distributed representation approach. Proceedings of the 22nd Annual Conference of the Cognitive Science Society (pp. 633–638). Hillsdale: Erlbaum.Google Scholar
  19. Crisp, V., & Sweiry, E. (2006). Can a picture ruin a thousand words? The effects of visual resources in exam questions. Educational Research, 48(2), 139–154. Scholar
  20. Daniel, K. L., Bucklin, C. J., Leone, E. A., & Idema, J. (2018). Towards a Definition of Representational Competence. In K. L. Daniel (Ed.), Towards a framework for representational competence in science education (pp. 3–11). Cham: Springer.CrossRefGoogle Scholar
  21. Del Re, A. C. (2013). compute effect sizes. R package version 0.2-2. Retrieved May 22, 2019, from Accessed 22 May 2019
  22. Dewolf, T. (2014). Get the picture? Are representational illustrations effective in helping pupils to solve mathematical word problems realistically? Unpublished doctoral dissertation. University of Leuven.Google Scholar
  23. Dewolf, T., Van Dooren, W., Ev Cimen, E., & Verschaffel, L. (2014). The impact of illustrations and warnings on solving mathematical word problems realistically. Journal of Experimental Education, 82(1), 103–120. Scholar
  24. Dewolf, T., Van Dooren, W., Hermens, F., & Verschaffel, L. (2015). Do students attend to representational illustrations of non-standard mathematical word problems, and, if so, how helpful are they? Instructional Science, 43(1), 147–171. Scholar
  25. Dewolf, T., Van Dooren, W., & Verschaffel, L. (2017). Can visual aids in representational illustrations help pupils to solve mathematical word problems more realistically? European Journal of Psychology of Education, 32(3), 335–351. Scholar
  26. Dindar, M., Yurdakul, I. K., & Dönmez, F. I. (2013). Multimedia in test items: animated questions vs. static graphics questions. Procedia - Social and Behavioral Sciences, 106, 1876–1882. Scholar
  27. Eitel, A. (2016). How repeated studying and testing affects multimedia learning: evidence for adaptation to task demands. Learning and Instruction, 41, 70–84. Scholar
  28. Eitel, A., Bender, L., & Renkl, A. (2019). Are seductive details seductive only when you think they are relevant? An experimental test of the moderating role of perceived relevance. Applied Cognitive Psychology, 33(1), 20–30. Scholar
  29. Elia, I., & Philippou, G. (2004). The functions of pictures in problem solving. In M. J. Hoines & A. B. Fuglestad (Eds.), Proceedings of the 28th Conference of the International Group for the Psychology of Mathematics Education: Vol. 2 (pp. 327–334). Bergen: PME.Google Scholar
  30. Elia, I., Gagatsis, A., & Demetriou, A. (2007). The effects of different modes of representation on the solution of one-step additive problems. Learning and Instruction, 17(6), 658–672. Scholar
  31. Folker, S., Ritter, H., & Sichelschmidt, L. (2005). Processing and integrating multimodal material—the influence of color-coding. In B. G. Bara, L. Barsalou, & &. M. Bucciarelli (Eds.), Proceedings of the 27th Annual Conference of the Cognitive Science Society (pp. 690–695). Mahwah: Erlbaum.Google Scholar
  32. Fuchs, L. S., Fuchs, D., Compton, D. L., Hamlett, C. L., & Wang, A. Y. (2015). Is word-problem solving a form of text comprehension? Scientific Studies of Reading, 19(3), 204–223. Scholar
  33. Funke, J., Fischer, A., & Holt, D. V. (2018). Competencies for complexity: problem solving in the twenty-first century. In Assessment and teaching of 21st century skills (pp. 41–53). Cham: Springer.CrossRefGoogle Scholar
  34. Gagatsis, A., & Elia, E. (2004). The effects of different modes of representation on mathematical problem solving. In M. J. Hoines & A. B. Fuglestad (Eds.), Proceedings of the 28th Conference of the International Group of the Psychology of Mathematics Education. Vol. 2 (pp. 447–454). Bergen: PME.Google Scholar
  35. Garcia-Retamero, R., Galesic, M., & Gigerenzer, G. (2010). Do icon arrays help reduce denominator neglect? Medical Decision Making, 30(6), 672–684.
  36. Garcia-Retamero, R., & Hoffrage, U. (2013). Visual representation of statistical information improves diagnostic inferences in doctors and their patients. Social Science & Medicine, 83, 27–33.CrossRefGoogle Scholar
  37. Garrett, A. J. (2008). The role of picture perception in children’s performance on a picture vocabulary test (Doctoral dissertation). Retrieved from ProQuest Dissertations Publishing. (Accession No. 3324639).Google Scholar
  38. Gentner, D., Loewenstein, J., & Thompson, L. (2003). Learning and transfer: a general role for analogical encoding. Journal of Educational Psychology, 95(2), 393–408.CrossRefGoogle Scholar
  39. Gibson, J. J. (1966). The senses considered as perceptual systems. New York: Houghton Mifflin.Google Scholar
  40. Gibson, J. J. (1979). The ecological approach to visual perception. New York: Houghton Mifflin.Google Scholar
  41. Gick, M. L., & Holyoak, K. J. (1983). Schema induction and analogical transfer. Cognitive Psychology, 15(1), 1–38.
  42. Ginther, A. (2001). Effects of the presence and absence of visuals on performance on TOEFL CBT Listening-Comprehension Stimuli. (TOEFL research report no. 66). Princeton: Educational Testing Service.Google Scholar
  43. Goldhammer, F., Naumann, J., Stelter, A., Tóth, K., Rölke, H., & Klieme, E. (2014). The time on task effect in reading and problem solving is moderated by task difficulty and skill: insights from a computer-based large-scale assessment. Journal of Educational Psychology, 106(3), 608–626. Scholar
  44. Goldhammer, F., Naumann, J., & Greiff, S. (2015). More is not always better: the relation between item response and item response time in Raven’s matrices. Journal of Intelligence, 3(1), 21–40. Scholar
  45. Goolkasian, P. (1996). Picture-word differences in a sentence verification task. Memory & Cognition, 24(5), 584–594. Scholar
  46. Greco, T., Zangrillo, A., Biondi-Zoccai, G., & Landoni, G. (2013). Meta-analysis: pitfalls and hints. Heart, Lung and Vessels, 5(4), 219–225. Scholar
  47. Greiff, S., Wüstenberg, S., Holt, D. V., Goldhammer, F., & Funke, J. (2013). Computer-based assessment of complex problem solving: concept, implementation, and application. Educational Technology Research and Development, 61(3), 407–421. Scholar
  48. Hao, Y. (2010). Does multimedia help students answer test items? Computers in Human Behavior, 26(5), 1149–1157. Scholar
  49. Hardy-Vallée, B., & Payette, N. (Eds.). (2009). Beyond the brain: embodied, situated and distributed cognition. Newcastle, UK: Cambridge Scholars Publishing.Google Scholar
  50. Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: a theory of cognitive interest in science learning. Journal of Educational Psychology, 90(3), 414–434. Scholar
  51. Hartmann, S., & Leutner, D. (2013). Die Rolle von Leseverständnis und Lesegeschwindigkeit beim Zustandekommen der Leistungen in schriftlichen Tests zur Erfassung naturwissenschaftlicher Kompetenz (Doctoral dissertation, University of Duisburg-Essen. Retrieved from http://duepublico. uni-duisburg-essen. de/servlets/DerivateServlet/Derivate-33260/hartmann_diss. pdf).Google Scholar
  52. Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32(6), 717–742. Scholar
  53. Hollan, J., Hutchins, E., & Kirsh, D. (2000). Distributed cognition: toward a new foundation for human-computer interaction research. ACM Transactions on Computer-Human Interaction (TOCHI), 7(2), 174–196.CrossRefGoogle Scholar
  54. Hoogland, K., de Koning, J., Bakker, A., Pepin, B. E. U., & Gravemeijer, K. (2018a). Changing representation in contextual mathematical problems from descriptive to depictive: the effect on students’ performance. Studies in Educational Evaluation, 58(June), 122–131. Scholar
  55. Hoogland, K., Pepin, B., de Koning, J., Bakker, A., & Gravemeijer, K. (2018b). Word problems versus image-rich problems: an analysis of effects of task characteristics on students’ performance on contextual mathematics problems. Research in Mathematics Education, 20(1), 37–52. Scholar
  56. Huinker, D. (2015). Representational competence: a renewed focus for classroom practice in mathematics. Wisconsin Teacher of Mathematics, 67(2), 4–8.Google Scholar
  57. Isberner, M. B., Richter, T., Maier, J., Knuth-Herzig, K., Horz, H., & Schnotz, W. (2013). Comprehending conflicting science-related texts: graphs as plausibility cues. Instructional Science, 41(5), 849–872. Scholar
  58. Jarodzka, H., Janssen, N., Kirschner, P. A., & Erkens, G. (2015). Avoiding split attention in computer-based testing: is neglecting additional information facilitative? British Journal of Educational Technology, 46(4), 803–817. Scholar
  59. Johnson-Laird, P. N. (2005). Mental models and thought. In K. J. Holyoak and R. G. Morrison (Eds.), The Cambridge handbook of thinking and reasoning (185–208). New York: Cambridge University Press.Google Scholar
  60. Johnstone, A. H. (1982). Macro and microchemistry. School Science Review, 64, 377–379.Google Scholar
  61. Kirschner, P., Park, B., Malone, S., & Jarodzka, H. (2017). Towards a cognitive theory of multimedia assessment (CTMMA). In J. M. Spector, B. B. Lockee, & M. Childress (Eds.), Learning, design, and technology: an international compendium of theory, research, practice, and policy (pp. 1–23). Cham: Springer.Google Scholar
  62. Lenzner, A., Schnotz, W., & Müller, A. (2013). The role of decorative pictures in learning. Instructional Science, 41(5), 811–831. Scholar
  63. Lesh, R., Post, T., & Behr, M. (1987). Representations and translations among representations in mathematics learning and problem solving. In C. Janvier (Ed.), Problems of representations in the teaching and learning of mathematics (pp. 33–40). Hillsdale: Lawrence Erlbaum.Google Scholar
  64. Levin, J. R. (1981). On the functions of pictures in prose. In F. J. Pirozzolo & M. C. Wittrock (Eds.), Neuropsychological and cognitive processes in reading (pp. 203–228). San Diego: Academic Press.CrossRefGoogle Scholar
  65. Levin, J. R., Anglin, G. J., & Carney, R. N. (1987). On empirically validating functions of pictures in prose. In D. M. Willows & H. A. Houghton (Eds.), The psychology of illustration: vol. 1. Basic research (pp. 51–85). New York: Springer-Verlag.CrossRefGoogle Scholar
  66. Light, R. J., & Pillemer, D. B. (1984). Summing up: the science of reviewing research. Cambridge: Harvard University Press.Google Scholar
  67. Lin, Y.-H., Wilson, M., & Cheng, C.-L. (2013). An investigation of the nature of the influences of item stem and option representation on student responses to a mathematics test. European Journal of Psychology of Education, 28(4), 1141–1161. Scholar
  68. Lindner, M. A., Ihme, J. M., Saß, S., & Köller, O. (2016). How representational pictures enhance students’ performance and test-taking pleasure in low-stakes assessment. European Journal of Psychological Assessment., 34(6), 376–385. Scholar
  69. Lindner, M. A., Eitel, A., Strobel, B., & Köller, O. (2017a). Identifying processes underlying the multimedia effect in testing: an eye-movement analysis. Learning and Instruction, 47, 91–102. Scholar
  70. Lindner, M. A., Lüdtke, O., Grund, S., & Köller, O. (2017b). The merits of representational pictures in educational assessment: evidence for cognitive and motivational effects in a time-on-task analysis. Contemporary Educational Psychology, 51, 482–492. Scholar
  71. Lindner, M. A., Eitel, A., Barenthien, J., & Köller, O. (2018). An integrative study on learning and testing with multimedia: effects on students’ performance and metacognition. Learning and Instruction.
  72. Magner, U. I. E., Schwonke, R., Aleven, V., Popescu, O., & Renkl, A. (2014). Triggering situational interest by decorative illustrations both fosters and hinders learning in computer-based learning environments. Learning and Instruction, 29, 141–152. Scholar
  73. Maries, A. (2013). Role of multiple representations in physics problem solving (Doctoral dissertation). Retrieved May 21, 2019, from: Accessed 21 May 2019
  74. Martin, M. O., Mullis, I. V. S., Foy, P., & Hooper, M. (2016). TIMSS 2015 international results in science. International Association for the Evaluation of Educational Achievement. Retrieved May 21, 2019, from Accessed 21 May 2019
  75. Mayer, R. E. (1987). Educational psychology: a cognitive approach. Boston: Little, Brown.Google Scholar
  76. Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press.CrossRefGoogle Scholar
  77. Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge University Press.CrossRefGoogle Scholar
  78. Mayer, R. E. (2014). Cognitive theory of multimedia learning. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 31–48). Cambridge: Cambridge University Press. Scholar
  79. McCabe, D. P., & Castel, A. D. (2008). Seeing is believing: the effect of brain images on judgments of scientific reasoning. Cognition, 107(1), 343–352. Scholar
  80. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Prisma Group. (2009). Reprint-preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Physical Therapy, 89(9), 873–880.Google Scholar
  81. Moreno, R. (2006). Does the modality principle hold for different media? A test of the method-affects-learning hypothesis. Journal of Computer Assisted Learning, 22(3), 149–158. Scholar
  82. Moreno, R. (2009). Learning from animated classroom exemplars: the case for guiding student teachers’ observations with metacognitive prompts. Educational Research and Evaluation, 15(5), 487–501. Scholar
  83. Moreno, R., & Mayer, R. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19(3), 309–326. Scholar
  84. Nathan, M. J., Kintsch, W., & Young, E. (1992). A theory of algebra-word-problem comprehension and its implications for the design of learning environments. Cognition and Instruction, 9(4), 329–389.CrossRefGoogle Scholar
  85. Newell, A. (1990). Unified theories of cognition. Cambridge: Harvard University Press.Google Scholar
  86. Nickerson, R. S. (1965). Short-term memory for complex meaningful visual configurations: a demonstration of capacity. Canadian Journal of Psychology, 19(2), 155–160. Scholar
  87. Ögren, M., Nyström, M., & Jarodzka, H. (2017). There’s more to the multimedia effect than meets the eye: is seeing pictures believing? Instructional Science, 45(2), 263–287. Scholar
  88. Ott, N., Brünken, R., Vogel, M., & Malone, S. (2018). Multiple symbolic representations: The combination of formula and text supports problem solving in the mathematical field of propositional logic. Learning and Instruction, 58(December 2016), 88–105.
  89. Paivio, A. (1986). Mental representations. New York: Oxford University Press.Google Scholar
  90. Pande, P., & Chandrasekharan, S. (2017). Representational competence: towards a distributed and embodied cognition account. Studies in Science Education, 53(1), 1–43. Scholar
  91. Park, B., Moreno, R., Seufert, T., & Brünken, R. (2011). Does cognitive load moderate the seductive details effect? A multimedia study. Computers in Human Behavior, 27(1), 5–10. Scholar
  92. Pennock-Roman, M., & Rivera, C. (2011). Mean effects of test accommodations for ELLs and non-ELLs: a meta-analysis of experimental studies. Educational Measurement: Issues and Practice, 30(3), 10–28. Scholar
  93. OECD (2007). PISA 2006: science competencies for tomorrow’s world: volume 1: analysis. PISA, OECD Publishing, Paris.
  94. Ramjan, L. M. (2011). Contextualism adds realism: Nursing students’ perceptions of and performance in numeracy skills tests. Nurse Education Today, 31(8), e16–e21.Google Scholar
  95. Reusser, K. (1996). From cognitive modeling to the design of pedagogical tools. In S. Vosniadou, E. De Corte, R. Glaser, & H. Mandl (Eds.), International perspectives on the design of technology supported learning environments (pp. 81–104). Mahwah: Lawrence Erlbaum Associates, Publishers.Google Scholar
  96. Rop, G. (2017). Effects of task experience on attention to extraneous information during multimedia learning effects of task experience on attention (Doctoral dissertation). Retrieved from:
  97. Rop, G., Verkoeijen, P. P. J. L., & van Gog, T. (2017). With task experience students learn to ignore the content, not just the location of irrelevant information. Journal of Cognitive Psychology, 29(5), 599–606. Scholar
  98. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. Scholar
  99. Rosenthal, R. (1991). Meta-analytic procedures for social research. Newbury Park: Sage. Scammacca.CrossRefGoogle Scholar
  100. Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.). (2005). Publication bias in meta-analysis: prevention, assessment, and adjustments. New York: Wiley. Scholar
  101. Saß, S., & Schütte, K. (2016). Helping poor readers demonstrate their science competence: item characteristics supporting text-picture integration. Journal of Psychoeducational Assessment, 34(1), 91–96. Scholar
  102. Saß, S., Wittwer, J., Senkbeil, M., & Köller, O. (2012). Pictures in test items: effects on response time and response correctness. Applied Cognitive Psychology, 26(1), 70–81.CrossRefGoogle Scholar
  103. Saß, S., Schütte, K., & Lindner, M. A. (2017). Test-takers’ eye movements: effects of integration aids and types of graphical representations. Computers and Education, 109, 85–97. Scholar
  104. Schneider, W., & Chein, J. M. (2003). Controlled & automatic processing: behavior, theory, and biological mechanisms. Cognitive Science, 27(3), 525–559. Scholar
  105. Schneider, S., Nebel, S., & Rey, G. D. (2016). Decorative pictures and emotional design in multimedia learning. Learning and Instruction, 44, 65–73. Scholar
  106. Schnotz, W. (2014). Integrated model of text and picture comprehension. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 72–103). Cambridge: Cambridge University Press. Scholar
  107. Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representation. Learning and Instruction, 13(2), 141–156. Scholar
  108. Scherer, R., Greiff, S., & Hautamäki, J. (2015). Exploring the relation between time on task and ability in complex problem solving. Intelligence, 48, 37–50.
  109. Schüler, A., Pazzaglia, F., & Scheiter, K. (2019). Specifying the boundary conditions of the multimedia effect: the influence of content and its distribution between text and pictures. British Journal of Psychology, 110(1), 126–150.CrossRefGoogle Scholar
  110. Schwert, P. M. (2007). Using sentence and picture clues to solve verbal insight problems. Creativity Research Journal, 19(2–3), 293–306.CrossRefGoogle Scholar
  111. Serra, M. J., & Dunlosky, J. (2010). Metacomprehension judgements reflect the belief that diagrams improve learning from text. Memory, 18(7), 698–711. Scholar
  112. Shepard, R. N. (1967). Recognition memory for words, sentences, and pictures. Journal of Verbal Learning and Verbal Behavior, 6(1), 156–163. Scholar
  113. Shiffrin, R. M., & Schneider, W. (1977). Controlled and automatic human information processing II: perceptual learning, automatic attending and a general theory. Psychological Review, 84(2), 127–189. Scholar
  114. Solano-Flores, G., Wang, C., Kachchaf, R., Soltero-Gonzalez, L., & Nguyen-Le, K. (2014). Developing testing accommodations for English language learners: illustrations as visual supports for item accessibility. Educational Assessment, 19(4), 267–283.CrossRefGoogle Scholar
  115. Solano-Flores, G., Wang, C., & Shade, C. (2016). International semiotics: item difficulty and the complexity of science item illustrations in the PISA-2009 international test comparison. International Journal of Testing, 16(3), 205–219.CrossRefGoogle Scholar
  116. Sterne, J. A. C., & Egger, M. (2001). Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. Journal of Clinical Epidemiology, 54(10), 1046–1055. Scholar
  117. Sweller, J. (1988). Cognitive load during problem solving: effects on learning. Cognitive Science, 12(2), 257–285. Scholar
  118. Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295–312. Scholar
  119. Verschaffel, L., Greer, B., & De Corte, E. (2000). Making sense of word problems. Lisse: Swets & Zeitling.Google Scholar
  120. Viechtbauer, W. (2010). Conducting meta-analysis in R with the metafor package. Journal of Statistical Software, 36(3), 1–48.CrossRefGoogle Scholar
  121. Viechtbauer, W., & Cheung, M. W. L. (2010). Outlier and influence diagnostics for meta-analysis. Research Synthesis Methods, 1(2), 112–125.CrossRefGoogle Scholar
  122. Whitley, K. N., Novick, L. R., & Fisher, D. (2006). Evidence in favor of visual representation for the dataflow paradigm: an experiment testing LabVIEW’s comprehensibility. International Journal of Human-Computer Studies, 64(4), 281–303. Scholar
  123. Wiley, J., Sanchez, C. A., & Jaeger, A. J. (2014). The individual differences in working memory capacity principle in multimedia learning. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 598–620). Cambridge: Cambridge University Press.
  124. Winter, P. C., Kopriva, R. J., Chen, C. S., & Emick, J. E. (2006). Exploring individual and item factors that affect assessment validity for diverse learners: results from a large-scale cognitive lab. Learning and Individual Differences, 16(4), 267–276. Scholar
  125. Wise, S. L., Pastor, D. A., & Kong, X. J. (2009). Correlates of rapid-guessing behavior in low-stakes testing: implications for test development and measurement practice. Applied Measurement in Education, 22(2), 185–205. Scholar
  126. Wu, H. K., Kuo, C. Y., Jen, T. H., & Hsu, Y. S. (2015). What makes an item more difficult? Effects of modality and type of visual information in a computer-based assessment of scientific inquiry abilities. Computers and Education, 85, 35–48. Scholar
  127. Yang, D. C., & Huang, F. Y. (2004). Relationships among computational performance, pictorial representation, symbolic representation and number sense of sixth-grade students in Taiwan. Educational Studies, 30(4), 373–389.
  128. Zahner, D., & Corter, J. E. (2010). The process of probability problem solving: use of external visual representations. Mathematical Thinking and Learning, 12(2), 177–204. Scholar
  129. Zhang, J. (1997). The nature of external representations in problem solving. Cognitive Science, 21(2), 179–217. Scholar
  130. Zhang, J., & Norman, D. A. (1994). Representations in distributed cognitive tasks. Cognitive Science, 18(1), 87–122.CrossRefGoogle Scholar
  131. Zhang, J., & Norman, D. A. (1995). A representational analysis of numeration systems. Cognition, 57(3), 271–295.CrossRefGoogle Scholar
  132. Zhang, J., & Patel, V. L. (2006). Distributed cognition, representation, and affordance. Pragmatics & Cognition, 14(2), 333–341. Scholar
  133. Zhao, F., Schnotz, W., Wagner, I., & Gaschler, R. (2014). Eye tracking indicators of reading approaches in text-picture comprehension. Frontline Learning Research, 6, 46–66. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Faculty of EducationThe University of Hong KongHong KongChina
  2. 2.School of EducationShaanxi Normal UniversityXi’anChina
  3. 3.Office of the Vice President (Academic)The Education University of Hong KongHong KongChina

Personalised recommendations