Abstract
We investigated strategies used by readers when reading a science article with a diagram and assessed whether semantic and spatial representations were constructed while reading the diagram. Seventy-one undergraduate participants read a scientific article while tracking their eye movements and then completed a reading comprehension test. Our results showed that the text-diagram referencing strategy was commonly used. However, some readers adopted other reading strategies, such as reading the diagram or text first. We found all readers who had referred to the diagram spent roughly the same amount of time reading and performed equally well. However, some participants who ignored the diagram performed more poorly on questions that tested understanding of basic facts. This result indicates that dual coding theory may be a possible theory to explain the phenomenon. Eye movement patterns indicated that at least some readers had extracted semantic information of the scientific terms when first looking at the diagram. Readers who read the scientific terms on the diagram first tended to spend less time looking at the same terms in the text, which they read after. Besides, presented clear diagrams can help readers process both semantic and spatial information, thereby facilitating an overall understanding of the article. In addition, although text-first and diagram-first readers spent similar total reading time on the text and diagram parts of the article, respectively, text-first readers had significantly less number of saccades of text and diagram than diagram-first readers. This result might be explained as text-directed reading.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Afflerbach P, Pearson P, Paris SG (2008) Clarifying differences between reading skills and reading strategies. Read Teach 61(5):364–373. doi:10.1598/RT.61.5.1
Ainsworth S (1999) The functions of multiple representations. Comput Educ 33:131–152. doi:10.1016/S0360-1315(99)00029-9
Andrews S, Miller B, Rayner K (2004) Eye movements and morphological segmentation of compound words: there is a mouse in mousetrap. Eur J Cogn Psychol 16(1/2):285–311. doi:10.1080/09541440340000123
Boucheix JM, Lowe RK (2010) An eye tracking comparison of external pointing cues and internal continuous cues in learning with complex animations. Learn Instr 20:123–135. doi:10.1016/j.learninstruc.2009.02.015
Brusnighan MB, Folk JR (2012) Combining contextual and morphemic cues is beneficial during incidental vocabulary acquisition: semantic transparency in novel compound word processing. Read Res Q 47(2):172–190. doi:10.1002/RRQ.015
Carney RN, Levin JR (2002) Pictorial illustrations still improve students’ learning from text. Educ Psychol Rev 14(1):5–26. doi:10.1023/A%3A1013176309260
Cook MP (2006) Visual representations in science education: the influence of prior knowledge and cognitive load theory on instructional design principles. Sci Educ 90:1073–1091. doi:10.1002/sce.20164
De Koning BB, Tabbers HK, Rikers RMJP, Paas F (2007) Attention cueing as a mean to enhance learning from an animation. Appl Cognit Psychol 21:731–746. doi:10.1002/acp.1346
De Koning BB, Tabbers HK, Rikers RMJP, Paas F (2010) Attention guidance in learning from a complex animation: seeing is understanding? Learn Instr 20(2):111–122. doi:10.1016/j.learninstruc.2009.02.010
Ferk V, Vrtacnik M, Blejec A, Gril A (2003) Students’ understanding of molecular structure representations. Int J Sci Educ 25(10):1227–1245. doi:10.1080/0950069022000038231
Hannus M, Hyönä J (1999) Utilization of illustrations during learning of science textbook passages among low- and high-ability children. Contemp Educ Psychol 24:95–123. doi:10.1006/ceps.1998.0987
Hegarty M (1992) Mental animation: Inferring motion from static displays of mechanical systems. J Exp Psychol Learn Mem Cognit 18:1084–1102. doi:10.1037//0278-7393.18.5.1084
Hegarty M, Just MA (1993) Constructing mental models of machines from text and diagrams. J Mem Lang 32:717–742. doi:10.1006/jmla.1993.1036
Hegarty M, Carpenter PA, Just MA (1991) Diagrams in the comprehension of scientific texts. In: Barr R, Kamil ML, Mosenthal PB, Pearson PD (eds) Handbook of reading research, vol II. Longman, New York, pp 641–668
Heiser J, Tversky B (2006) Arrows in comprehending and producing mechanical diagrams. Cognit Sci 30:581–592. doi:10.1207/s15516709cog0000_70
Holsanova J, Holmberg N, Holmqvist K (2009) Reading information graphics: the role of spatial contiguity and dual attentional guidance. Appl Cognit Psychol 23:1215–1226. doi:10.1002/acp.1525
Hyönä J (2010) The use of eye movements in the study of multimedia learning. Learn Instr 20(2):172–176. doi:10.1016/j.learninstruc.2009.02.013
Jian YC, Chen ML, Ko HW (2013) Context effects in processing of Chinese academic words: an eye-tracking investigation. Read Res Quart 48(4):403–413. doi:10.1002/rrq.56
Jian YC, Wu CJ (2011) Using eye tracking to investigate the function of diagram in scientific article (in Chinese). Oral presentation at the 28th conference on Chinese Science Education, Kaohsiung, Taiwan
Jian YC, Wu CJ (2012) The effect of arrows in an illustration when reading scientific text: evidence from eye movements (in Chinese). Chinese J Psychol 54:385–402
Johnson CI, Mayer RE (2012) An eye movement analysis of the spatial contiguity effect in multimedia learning. J Exp Psychol Appl 18:178–191. doi:10.1037/a0026923
Kress G, van Leeuwen T (1996) Reading images: the grammar of the visual design. Routledge, New York
LeDoux J (1994) Emotion, memory and the brain. Sci Am 270:50–57
Ledoux K, Gordon PC, Camblin CC, Swaab TY (2007) Coreference and lexical repetition: mechanisms of discourse integration. Mem Cognit 35(4):801–815. doi:10.3758/BF03193316
Liversedge SP, Pickering MJ, Clayes EL, Branigan HP (2003) Thematic processing of adjuncts: evidence from an eye-tracking experiment. Psychon Bull Rev 10(3):667–675. doi:10.3758/BF03196530
Mason L, Pluchino P, Tornatora MC, Ariasi N (2013a) An eye-tracking study of learning from science text with concrete and abstract illustrations. J Exp Educ 81:356–384. doi:10.1080/00220973.2012.727885
Mason L, Tornatora MC, Pluchino P (2013b) Do fourth graders integrated text and picture in processing and learning from an illustrated science text? Evidence from eye-movement patterns. Comput Educ 60:95–109. doi:10.1016/j.compedu.2012.07.011
Mayer RE (1989) Systematic thinking fostered by illustrations in scientific text. J Educ Psychol 81:240–246. doi:10.1037/0022-0663.81.2.240
Mayer RE (2001) Multimedia learning. Cambridge University Press, Cambridge
Mayer RE (2005) Cognitive theory of multimedia learning. In: Mayer RE (ed) Cambridge handbook of multimedia learning. Cambridge University Press, New York, pp 31–48
Mayer RE, Gallini JK (1990) When is an illustration worth ten thousand words? J Educ Psychol 82:715–726. doi:10.1037/0022-0663.82.4.715
Mayer RE, Moreno R (2003) Nine ways to reduce cognitive load in multimedia learning. Educ Psychol 38:43–52. doi:10.1207/S15326985EP3801_6
Ozcelik E, Arslan-Ari I, Cagiltay K (2010) Why does signaling enhance multimedia learning? Evidence from eye movements. Comput Hum Behav 26:110–117
Paivio A (1990) Dual coding theory. In: Paivio A (ed) Mental representations: a dual coding approach. Oxford University Press, New York, pp 53–83
Peeck J (1993) Increasing picture effects in learning from illustrated text. Learn Instr 3:227–238. doi:10.1016/0959-4752(93)90006-L
Pozzer LL, Roth WM (2003) Prevalence, function, and structure of photographs in highschool biology textbooks. J Res Sci Teach 40(10):1089–1114. doi:10.1002/tea.10122
Ratwani RM, Trafton JG, Boehm-Davis DA (2008) Thinking graphically: connecting vision and cognition during graph comprehension. J Exp Psychol Appl 14(1):36–49. doi:10.1037/1076-898X.14.1.36
Rayner K (1998) Eye movements in reading and information processing: 20 years of research. Psychol Bull 124(3):372–422. doi:10.1037/0033-2909.124.3.372
Rayner K, Rotello CM, Stewart AJ, Keir J, Duffy SA (2001) Integrating text and pictorial information: eye movements when looking at print advertisements. J Exp Psychol Appl 7:219–226. doi:10.1037//1076-898X.7.3.219
Roth WM, Pozzer-Ardenghi L, Han J (2005) Critical graphicacy: Understanding visual representation practices in school science. Springer, Dordrecht
Schnotz W (2002) Toward an integrated view of learning from text and visual displays. Educ Psychol Rev 14(1):101–120. doi:10.1023/A:1013136727916
She HC, Chen YZ (2009) The impact of multimedia effect on science learning: evidence from eye movements. Comput Educ 53:1297–1307. doi:10.1016/j.compedu.2009.06.012
Slough S, McTigue E (2010) Introduction to the integration of verbal and visual information in science texts. Read Psychol 31(3):206–212. doi:10.1080/02702710903241397
Slough S, McTigue EM, Kim S, Jennings S (2010) Science textbook’s use of graphical representation: a descriptive analysis of four sixth-grade science texts. Read Psychol 31:301–325. doi:10.1080/02702710903256502
Unsworth L (2001) Teaching multiliteracies across the curriculum: changing contexts of text and image in classroom practice. Open University Press, Buckingham
Yang Yu-Fen (2006) Reading strategies or comprehension monitoring strategies? Read Psychol 27(4):313–343. doi:10.1080/02702710600846852
Acknowledgments
This study is supported by the grant NSC101-2511-S-003-015 from the National Science Council in Taiwan.
Author information
Authors and Affiliations
Corresponding author
Appendix: The Reading Material in the Present Study (English Edition)
Appendix: The Reading Material in the Present Study (English Edition)
Rights and permissions
About this article
Cite this article
Jian, YC., Wu, CJ. Using Eye Tracking to Investigate Semantic and Spatial Representations of Scientific Diagrams During Text-Diagram Integration. J Sci Educ Technol 24, 43–55 (2015). https://doi.org/10.1007/s10956-014-9519-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10956-014-9519-3