Skip to main content
Log in

Using Eye-Tracking Technology to Explore the Impact of Instructional Multimedia on CFL Learners’ Chinese Character Recognition

  • Regular Article
  • Published:
The Asia-Pacific Education Researcher Aims and scope Submit manuscript

Abstract

This study investigated differences in Chinese as a Foreign Language (CFL) students’ learning achievement, cognitive load level, and eye-movement patterns when learning three types of Chinese characters: pictographs, compound ideographs, and phono-semantic compounds with multimedia slides. Thirty CFL students participated in this study. Additional eye-tracking data, such as participants’ total fixation duration, mean fixation duration, and fixation count while viewing instructional slides, were recorded. The results showed that CFL learners learned pictograph characters with less cognitive load, compared with that involved in learning compound ideograph and phono-semantic compound characters. Participants were found to fixate more on the character area when inspecting the pictograph than on the other two categories of characters. Meanwhile, fewer fixations were placed on the enlarged character area when text passages served as the major resource for understanding compound ideograph and phono-semantic compound characters. The illustrative nature of pictograph characters seemed to make this character form more explicit than the other categories of Chinese characters. However, using only text information to understand abstract meanings of compound ideographs and phono-semantic compounds may place an additional cognitive demand on CFL learners, resulting in learners spending less time inspecting the character shape. Replacing the text description with videos or animation may provide scenarios in which ambiguity in the meaning of compound ideographs and phono-semantic compounds can be removed, to promote better understanding of these characters for CFL learners. Future studies are recommended to determine if the use of voice-over and videos/animation explanations improve the ability to learn Chinese characters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Reference

  • Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183–198.

    Article  Google Scholar 

  • Beebe, L. (1985). Input: Choosing the right stuff. In S. Gass & C. Madden (Eds.), Input in second language acquisition (pp. 404–414). Rowley, MA: Newbury House.

    Google Scholar 

  • Block, D. (2003). The social turn in second language acquisition. Washington, D.C.: George-town University Press.

    Google Scholar 

  • Chen, H.-Y., & Liu, K.-Y. (2008). Web-based synchronized multimedia lecture system design for teaching/learning Chinese as second language. Computers & Education, 50(3), 693–702. https://doi.org/10.1016/j.compedu.2006.07.010.

    Article  Google Scholar 

  • Chinese character classification. (2018). In Wikipedia. Retrieved November 13, 2018, from https://en.wikipedia.org/wiki/Chinese_character_classification.

  • Chuang, H.-Y., & Ku, H.-Y. (2011). The effect of computer-based multimedia instruction with Chinese character recognition. Educational Media International, 48(1), 27–41. https://doi.org/10.1080/09523987.2011.549676.

    Article  Google Scholar 

  • Cook, V. (2003). Introduction: The changing LI in the L2 user's mind. In V. Cook (Ed.), The effects of learning a second language on the first: The case of increased metalinguistic awareness (pp. 1–18). Clevedon: Multilingual Matters.

    Chapter  Google Scholar 

  • de Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Paas, F. (2010). Attention guidance in learning from a complex animation: Seeing is understanding? Learning and Instruction, 20(2), 111–122.

    Article  Google Scholar 

  • Du, J., Zhai, J.F., Hu, J.S., Zhu, B., Wei, S., & Dai, L.R. (2015). Writer adaptive feature extraction based on convolutional neural networks for online handwritten Chinese character recognition. Paper presented at 13th international conference in Tubis.

  • Glaholt, M. G., Wu, M.-C., & Reingold, E. M. (2010). Evidence for top-down control of eye movements during visual decision making. Journal of Vision, 10, 1–10.

    Article  Google Scholar 

  • Hegarty, M. (1992). The mechanics of comprehension and comprehension of mechanics. In K. Rayner (Ed.), Eye movements and visual cognition. Springer series in neuropsychology (pp. 428–443). New York: Springer.

    Chapter  Google Scholar 

  • Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32, 717–742.

    Article  Google Scholar 

  • Hyönä, J. (2010). The use of eye movements in the study of multimedia learning. Learning and Instruction, 20(2), 172–176.

    Article  Google Scholar 

  • Jian, Y-C., Chen, M. L., & Ko, H. W. (2013). Context effects in processing of Chinese academic words: An eye-tracking investigation. Reading Research Quarterly48(4), 403–413. https://doi.org/10.1002/rrq.56.

    Article  Google Scholar 

  • Jiang, X. (2008). Research on words of teaching Chinese as a second language and learning to read. Beijing: Beijing Language and Culture University Press.

    Google Scholar 

  • Kikas, E. (2006). The effect of verbal and visuo-spatial abilities on the development of knowledge of the Earth. Research in Science Education, 36(3), 269.

    Article  Google Scholar 

  • Krashen, S. D. (1985). The input hypothesis: Issues and implications. London: Longman.

    Google Scholar 

  • Kriz, S., & Hegarty, M. (2007). Top-down and bottom-up influences on learning from animations. International Journal of Human-Computer Studies, 65(11), 911–930.

    Article  Google Scholar 

  • Kuo, M-L., & Hooper, S. (2004). The effects of visual and verbal coding mnemonics on learning Chinese characters in computer-based instruction. Educational Technology Research and Development, 52, 23–34. https://doi.org/10.1007/BF02504673.

    Article  Google Scholar 

  • Lee, C.-P., Shen, C.-W., & Lee, D. (2008). The effect of multimedia instruction for Chinese learning. Learning, Media and Technology, 33(2), 127–138. https://doi.org/10.1080/17439880802097667.

    Article  Google Scholar 

  • Mason, A., & Zhang, W. (2017). An exploration of the use of mobile applications to support the learning of Chinese characters employed by students of Chinese as a foreign language. https://doi.org/10.14705/rpnet.2017.mooc2016.674

  • Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press.

    Book  Google Scholar 

  • Ozcelik, E., Karakus, T., Kursun, E., & Cagiltay, K. (2009). An eye-tracking study of how color coding affects multimedia learning. Computers & Education, 53(2), 445–453.

    Article  Google Scholar 

  • Paas, F. G. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84(4), 429–434

    Article  Google Scholar 

  • Paas, F. G. W. C., & Van Merriënboer, J. J. G. (1994). Variability of worked examples and transfer of geometrical problem-solving skills: A cognitive-load approach. Journal of Educational Psychology, 86(1), 122–133.

    Article  Google Scholar 

  • Rayner, K. (1990). Eye movements and visual cognition: Introduction (pp. 1–7). New York: Springer-Verlag New York Inc.

    Google Scholar 

  • Rayner, K. (1998). Eye movements in reading and information processing. 20 years of research. Psychological Bulletin, 124(3), 372–422.

    Article  Google Scholar 

  • Rayner, K., Abbott, M., Schotter, E., Belanger, N., Higgins, E., Leinenger, M., et al. (2015). Keith Rayner eye movements in reading data collection. UC San Diego Library Digital Collections. https://doi.org/10.6075/J0JW8BSV.

    Article  Google Scholar 

  • Renshaw, J. A., Finlay, J. E., Tyfa, D., & Ward, R. D. (2004). Understanding visual influence in graph design through temporal and spatial eye movement characteristics. Interacting with Computers, 16, 557–578.

    Article  Google Scholar 

  • Rites of Zhou. (2018). In Wikipedia. Retrieved December 10, 2018, from https://en.wikipedia.org/wiki/Rites_of_Zhou

  • Shen, H. (2004). Level of cognitive processing: Effects on character learning among non-native learners of Chinese as a foreign language. Language and Education, 18(2), 167–182.

    Article  Google Scholar 

  • Shi, D., Damper, R. I., & Gunn, S. R. (2003). An approach to off-line handwritten Chinese character recognition based on hierarchical radical decomposition. Journal of Quantitative Linguistics, 10(1), 41–69.

    Article  Google Scholar 

  • Schnotz, W., & Kürschner, C. (2007). A reconsideration of cognitive load theory. Educational Psychology Review, 19, 469–508. https://doi.org/10.1007/s10648-007-9053-4.

    Article  Google Scholar 

  • Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295–312.

    Article  Google Scholar 

  • Underwood, G., & Radach, R. (1998). Eye guidance and visual information processing: Reading, visual search, picture perception and driving. In G. Underwood (Ed.), Eye guidance in reading and scene perception (pp. 1–27). Oxford: Elsevier.

    Google Scholar 

  • van Gog, T., & Scheiter, K. (2010). Eye tracking as a tool to study and enhance multimedia learning. Learning and Instruction, 20(2), 95–99.

    Article  Google Scholar 

  • Wen, Y. (2018). Chinese character composition game with the augment paper. Educational Technology & Society, 21(3), 132–145.

    Google Scholar 

  • Wieger, L. S. J. (1965). Chinese characters: Their origin, etymology, history, classification and signification. A thorough study from Chinese documents 2nd ed. English and revision according to the 4th French ed. New York: Paragon Book Reprint Corp.

  • Ye, L.-J. (2011). Teaching and learning Chinese as a foreign language in the United States: To delay or not to delay the character introduction (Doctoral dissertations, College of Art and Sciences of Georgia State University. Retrieved November 17, 2018, from https://scholarworks.gsu.edu/cgi/viewcontent.cgi?article=1021&context=alesl_diss

  • Zhen, Z. (2015). Advantages and thinking on design of Chinese characters’ graphics. Studies in Literature and Language, 11(2), 82–87. https://doi.org/10.3968/7447.

    Article  Google Scholar 

  • Zhou, P., Su, Y. E, Crain, S., Gao, L., & Zhan, L. (2012). Children’s use of phonological information in ambiguity resolution: A review from Mandarin Chinese. Journal of Child Language, 39, 687–730.

    Article  Google Scholar 

Download references

Acknowledgements

This research project was supported in part by the Ministry of Science and Technology, Taiwan; Grant No. MOST 106-2511-S-415-009-.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Han-Chin Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

figure a
figure b

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, HC. Using Eye-Tracking Technology to Explore the Impact of Instructional Multimedia on CFL Learners’ Chinese Character Recognition. Asia-Pacific Edu Res 30, 33–46 (2021). https://doi.org/10.1007/s40299-020-00512-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40299-020-00512-2

Keywords

Navigation