Advertisement

Investigating Eye Movements in Natural Language and C++ Source Code - A Replication Experiment

  • Patrick Peachock
  • Nicholas Iovino
  • Bonita Sharif
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10284)

Abstract

Natural language text and source code are very different in their structure and semantics. Source code uses words from natural language such as English mainly in comments and identifier names. Is there an inherent difference in the way programmers read natural language text compared to source code? Does expertise play a role in the reading behavior of programmers? In order to start answering these questions, we conduct a controlled experiment with novice and non-novice programmers while they read small short snippets of natural language text and C++ source code. This study is a replication of an earlier study by Busjahn et al. [1] but uses C++ instead of Java source code. The study was conducted with 33 students, who were each given ten tasks: a set of seven programs, and three natural language texts. They were asked one of three random comprehension questions after each task. Using several linearity metrics presented in an earlier study [1], we analyze the eye movements on source code and natural language. The results indicate that novices and non-novices both read source code less linearly than natural language text. We did not find any differences between novices and non-novices between natural language text and source code. We compare our results to the Busjahn study and provide directions for future work.

Keywords

Eye tracking study C++ Program comprehension Natural language 

References

  1. 1.
    Busjahn, T., Bednarik, R., Begel, A., Crosby, M., Paterson, J.H., Schulte, C., Sharif, B., Tamm, S.: Eye movements in code reading: relaxing the linear order. In: Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension, ICPC 2015, Piscataway, NJ, USA, pp. 255–265. IEEE Press (2015). http://dl.acm.org/citation.cfm?id=2820282.2820320
  2. 2.
    Rayner, K., Chace, K.H., Slattery, T.J., Ashby, J.: Eye movements as reflections of comprehension processes in reading. Sci. Stud. Read. 10, 241–255 (2006)CrossRefGoogle Scholar
  3. 3.
    Brooks, R.: Towards a theory of the comprehension of computer programs. Int. J. Man-Mach. Stud. 18(6), 543–554 (1983). http://www.sciencedirect.com/science/article/pii/S0020737383800315 CrossRefGoogle Scholar
  4. 4.
    Raina, S., Bernard, L., Taylor, B., Kaza, S.: Using eye-tracking to investigate content skipping: a study on learning modules in cybersecurity. In: 2016 IEEE Conference on Intelligence and Security Informatics (ISI), pp. 261–266, September 2016Google Scholar
  5. 5.
    Beelders, T., du Plessis, J.-P.: The influence of syntax highlighting on scanning and reading behaviour for source code. In: Proceedings of the Annual Conference of the South African Institute of Computer Scientists and Information Technologists, SAICSIT 2016, pp. 5:1–5:10. ACM, New York (2016). http://doi.acm.org/10.1145/2987491.2987536
  6. 6.
    Sarkar, A.: The impact of syntax colouring on program comprehension. In: PPIG, July 2015Google Scholar
  7. 7.
    Crosby, M.E.: Natural versus computer languages: a reading comparison. Ph.D. dissertation, University of Hawaii at Manoa (1986)Google Scholar
  8. 8.
    Fan, Q.: The effects of beacons, comments, and tasks on program comprehension process in software maintenance. Ph.D. dissertation, Catonsville, MD, USA (2010)Google Scholar
  9. 9.
    Turner, R., Falcone, M., Sharif, B., Lazar, A.: An eye-tracking study assessing the comprehension of C++ and Python source code. In: Proceedings of the Symposium on Eye Tracking Research and Applications, ETRA 2014, pp. 231–234. ACM, New York (2014). http://doi.acm.org/10.1145/2578153.2578218
  10. 10.
    Sharif, B., Maletic, J.I.: An eye tracking study on camelcase and under_score identifier styles. In: Proceedings of the 2010 IEEE 18th International Conference on Program Comprehension, ICPC 2010, Washington, DC, USA, pp. 196–205. IEEE Computer Society (2010). http://dx.doi.org/10.1109/ICPC.2010.41
  11. 11.
    Binkley, D., Davis, M., Lawrie, D., Maletic, J., Morrell, C., Sharif, B.: The impact of identifier style on effort and comprehension. Empir. Softw. Eng. J. (Invit. Submiss.) 18(2), 219–276 (2013)CrossRefGoogle Scholar
  12. 12.
    Uwano, H., Nakamura, M., Monden, A., Matsumoto, K.-I.: Analyzing individual performance of source code review using reviewers’ eye movement. In: Proceedings of the 2006 Symposium on Eye Tracking Research Applications, ETRA 2006, pp. 133–140. ACM, New York (2006). http://doi.acm.org/10.1145/1117309.1117357
  13. 13.
    Schulte, C., Clear, T., Taherkhani, A., Busjahn, T., Paterson, J.H.: An introduction to program comprehension for computer science educators. In: Proceedings of the 2010 ITiCSE Working Group Reports, ITiCSE-WGR 2010, pp. 65–86. ACM, New York (2010). http://doi.acm.org/10.1145/1971681.1971687
  14. 14.
    Busjahn, T., Schulte, C.: The use of code reading in teaching programming. In: Proceedings of the 13th Koli Calling International Conference on Computing Education Research, Koli Calling 2013, pp. 3–11. ACM, New York (2013). http://doi.acm.org/10.1145/2526968.2526969
  15. 15.
    Marter, T., Babucke, P., Lembken, P., Hanenberg, S.: Lightweight programming experiments without programmers and programs: an example study on the effect of similarity and number of object identifiers on the readability of source code using natural texts. In: Proceedings of the 2016 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward! 2016, pp. 1–14. ACM, New York (2016). http://doi.acm.org/10.1145/2986012.2986020
  16. 16.
    Hansen, M.: GitHub - synesthesiam/eyecode-tools: a collection of tools for analyzing data from my eyeCode experiment. https://github.com/synesthesiam/eyecode-tools
  17. 17.
    Cristino, F., Mathôt, S., Theeuwes, J., Gilchrist, I.D.: ScanMatch: a novel method for comparing fixation sequences. Behav. Res. Methods 42(3), 692–700 (2010). http://www.springerlink.com/index/10.3758/BRM.42.3.692 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Patrick Peachock
    • 1
  • Nicholas Iovino
    • 1
  • Bonita Sharif
    • 1
  1. 1.Youngstown State UniversityYoungstownUSA

Personalised recommendations