Using Eye Tracking to Investigate Semantic and Spatial Representations of Scientific Diagrams During Text-Diagram Integration
- 542 Downloads
We investigated strategies used by readers when reading a science article with a diagram and assessed whether semantic and spatial representations were constructed while reading the diagram. Seventy-one undergraduate participants read a scientific article while tracking their eye movements and then completed a reading comprehension test. Our results showed that the text-diagram referencing strategy was commonly used. However, some readers adopted other reading strategies, such as reading the diagram or text first. We found all readers who had referred to the diagram spent roughly the same amount of time reading and performed equally well. However, some participants who ignored the diagram performed more poorly on questions that tested understanding of basic facts. This result indicates that dual coding theory may be a possible theory to explain the phenomenon. Eye movement patterns indicated that at least some readers had extracted semantic information of the scientific terms when first looking at the diagram. Readers who read the scientific terms on the diagram first tended to spend less time looking at the same terms in the text, which they read after. Besides, presented clear diagrams can help readers process both semantic and spatial information, thereby facilitating an overall understanding of the article. In addition, although text-first and diagram-first readers spent similar total reading time on the text and diagram parts of the article, respectively, text-first readers had significantly less number of saccades of text and diagram than diagram-first readers. This result might be explained as text-directed reading.
KeywordsEye movement Scientific diagrams Text-diagram reading strategies Semantic representation Spatial representation
This study is supported by the grant NSC101-2511-S-003-015 from the National Science Council in Taiwan.
- Andrews S, Miller B, Rayner K (2004) Eye movements and morphological segmentation of compound words: there is a mouse in mousetrap. Eur J Cogn Psychol 16(1/2):285–311. doi: 10.1080/09541440340000123
- Hegarty M, Carpenter PA, Just MA (1991) Diagrams in the comprehension of scientific texts. In: Barr R, Kamil ML, Mosenthal PB, Pearson PD (eds) Handbook of reading research, vol II. Longman, New York, pp 641–668Google Scholar
- Jian YC, Chen ML, Ko HW (2013) Context effects in processing of Chinese academic words: an eye-tracking investigation. Read Res Quart 48(4):403–413. doi: 10.1002/rrq.56
- Jian YC, Wu CJ (2011) Using eye tracking to investigate the function of diagram in scientific article (in Chinese). Oral presentation at the 28th conference on Chinese Science Education, Kaohsiung, TaiwanGoogle Scholar
- Jian YC, Wu CJ (2012) The effect of arrows in an illustration when reading scientific text: evidence from eye movements (in Chinese). Chinese J Psychol 54:385–402Google Scholar
- Kress G, van Leeuwen T (1996) Reading images: the grammar of the visual design. Routledge, New YorkGoogle Scholar
- Rayner K (1998) Eye movements in reading and information processing: 20 years of research. Psychol Bull 124(3):372–422. doi: 10.1037/0033-2909.124.3.372
- Roth WM, Pozzer-Ardenghi L, Han J (2005) Critical graphicacy: Understanding visual representation practices in school science. Springer, DordrechtGoogle Scholar
- Unsworth L (2001) Teaching multiliteracies across the curriculum: changing contexts of text and image in classroom practice. Open University Press, BuckinghamGoogle Scholar