Investigating Eye Movements in Natural Language and C++ Source Code - A Replication Experiment

  • Patrick Peachock
  • Nicholas Iovino
  • Bonita SharifEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10284)


Natural language text and source code are very different in their structure and semantics. Source code uses words from natural language such as English mainly in comments and identifier names. Is there an inherent difference in the way programmers read natural language text compared to source code? Does expertise play a role in the reading behavior of programmers? In order to start answering these questions, we conduct a controlled experiment with novice and non-novice programmers while they read small short snippets of natural language text and C++ source code. This study is a replication of an earlier study by Busjahn et al. [1] but uses C++ instead of Java source code. The study was conducted with 33 students, who were each given ten tasks: a set of seven programs, and three natural language texts. They were asked one of three random comprehension questions after each task. Using several linearity metrics presented in an earlier study [1], we analyze the eye movements on source code and natural language. The results indicate that novices and non-novices both read source code less linearly than natural language text. We did not find any differences between novices and non-novices between natural language text and source code. We compare our results to the Busjahn study and provide directions for future work.


Eye tracking study C++ Program comprehension Natural language 


  1. 1.
    Busjahn, T., Bednarik, R., Begel, A., Crosby, M., Paterson, J.H., Schulte, C., Sharif, B., Tamm, S.: Eye movements in code reading: relaxing the linear order. In: Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension, ICPC 2015, Piscataway, NJ, USA, pp. 255–265. IEEE Press (2015).
  2. 2.
    Rayner, K., Chace, K.H., Slattery, T.J., Ashby, J.: Eye movements as reflections of comprehension processes in reading. Sci. Stud. Read. 10, 241–255 (2006)CrossRefGoogle Scholar
  3. 3.
    Brooks, R.: Towards a theory of the comprehension of computer programs. Int. J. Man-Mach. Stud. 18(6), 543–554 (1983). CrossRefGoogle Scholar
  4. 4.
    Raina, S., Bernard, L., Taylor, B., Kaza, S.: Using eye-tracking to investigate content skipping: a study on learning modules in cybersecurity. In: 2016 IEEE Conference on Intelligence and Security Informatics (ISI), pp. 261–266, September 2016Google Scholar
  5. 5.
    Beelders, T., du Plessis, J.-P.: The influence of syntax highlighting on scanning and reading behaviour for source code. In: Proceedings of the Annual Conference of the South African Institute of Computer Scientists and Information Technologists, SAICSIT 2016, pp. 5:1–5:10. ACM, New York (2016).
  6. 6.
    Sarkar, A.: The impact of syntax colouring on program comprehension. In: PPIG, July 2015Google Scholar
  7. 7.
    Crosby, M.E.: Natural versus computer languages: a reading comparison. Ph.D. dissertation, University of Hawaii at Manoa (1986)Google Scholar
  8. 8.
    Fan, Q.: The effects of beacons, comments, and tasks on program comprehension process in software maintenance. Ph.D. dissertation, Catonsville, MD, USA (2010)Google Scholar
  9. 9.
    Turner, R., Falcone, M., Sharif, B., Lazar, A.: An eye-tracking study assessing the comprehension of C++ and Python source code. In: Proceedings of the Symposium on Eye Tracking Research and Applications, ETRA 2014, pp. 231–234. ACM, New York (2014).
  10. 10.
    Sharif, B., Maletic, J.I.: An eye tracking study on camelcase and under_score identifier styles. In: Proceedings of the 2010 IEEE 18th International Conference on Program Comprehension, ICPC 2010, Washington, DC, USA, pp. 196–205. IEEE Computer Society (2010).
  11. 11.
    Binkley, D., Davis, M., Lawrie, D., Maletic, J., Morrell, C., Sharif, B.: The impact of identifier style on effort and comprehension. Empir. Softw. Eng. J. (Invit. Submiss.) 18(2), 219–276 (2013)CrossRefGoogle Scholar
  12. 12.
    Uwano, H., Nakamura, M., Monden, A., Matsumoto, K.-I.: Analyzing individual performance of source code review using reviewers’ eye movement. In: Proceedings of the 2006 Symposium on Eye Tracking Research Applications, ETRA 2006, pp. 133–140. ACM, New York (2006).
  13. 13.
    Schulte, C., Clear, T., Taherkhani, A., Busjahn, T., Paterson, J.H.: An introduction to program comprehension for computer science educators. In: Proceedings of the 2010 ITiCSE Working Group Reports, ITiCSE-WGR 2010, pp. 65–86. ACM, New York (2010).
  14. 14.
    Busjahn, T., Schulte, C.: The use of code reading in teaching programming. In: Proceedings of the 13th Koli Calling International Conference on Computing Education Research, Koli Calling 2013, pp. 3–11. ACM, New York (2013).
  15. 15.
    Marter, T., Babucke, P., Lembken, P., Hanenberg, S.: Lightweight programming experiments without programmers and programs: an example study on the effect of similarity and number of object identifiers on the readability of source code using natural texts. In: Proceedings of the 2016 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward! 2016, pp. 1–14. ACM, New York (2016).
  16. 16.
    Hansen, M.: GitHub - synesthesiam/eyecode-tools: a collection of tools for analyzing data from my eyeCode experiment.
  17. 17.
    Cristino, F., Mathôt, S., Theeuwes, J., Gilchrist, I.D.: ScanMatch: a novel method for comparing fixation sequences. Behav. Res. Methods 42(3), 692–700 (2010). CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Patrick Peachock
    • 1
  • Nicholas Iovino
    • 1
  • Bonita Sharif
    • 1
    Email author
  1. 1.Youngstown State UniversityYoungstownUSA

Personalised recommendations