Skip to main content
SpringerLink
Log in

Supporting comprehension of unfamiliar programs by modeling cues

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Developers need to comprehend a program before modifying it. In such cases, developers use cues to establish the relevance of a piece of information with a task. Being familiar with different kinds of cues will help the developers to comprehend a program faster. But unlike the experienced developers, novice developers fail to recognize the relevant cues, because (a) there are many cues and (b) they might be unfamiliar with the artifacts. However, not much is known about the developers’ choice of cue. Hence, we conducted two independent studies to understand the kind of cues used by the developers and how a tool influences their cue selection. First, from our user study on two common comprehension tasks, we found that developers actively seek the cues and their cue source choices are similar but task dependent. In our second exploratory study, we investigated whether an integrated development environment (IDE) influences a developer’s cue choices. By observing their interaction history while fixing bugs in Eclipse IDE, we found that the IDE’s influence on their cue choices was not statistically significant. Finally, as a case in point, we propose a novel task-specific program summarization approach to aid novice developers in comprehending an unfamiliar program. Our approach used developers cue choices to synthesize the summaries. A comparison of the synthesized summaries with the summaries recorded by the developers shows both had similar content. This promising result encourages us to explore task-specific cue models, which can aid novice developers to accomplish complex comprehension tasks faster.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The concept of information scent can be related to beacons, which Brooks (1983) described as a stimulus among the cues used by developers to take quick decision whether to pursue a hypothesis.

  2. https://www.eclipse.org/.

  3. https://www.jetbrains.com/idea/.

  4. https://www.visualstudio.com/.

  5. During a program comprehension task, cues only facilitate the inferential process or concept assignment and by themselves do not lend to comprehension.

  6. The editor in an IDE provides many features apart from syntax highlight, such as code folding, type definitions/documentation on hover, enhanced search and content assists.

  7. https://www.textpad.com/.

  8. Contributors to Eclipse projects are required to earn commit rights, which requires mentoring from one of the existing contributors and follow rigorous review process.

  9. Though our study included 10 graduate students, we are reporting the results based on the input of 5 graduate students as the remaining did not identify the artifacts and complete both the tasks.

  10. http://jedit.org.

  11. Though our results and discussions are based on Eclipse IDE, we believe our findings can be extended to other IDEs with similar characteristics.

  12. http://jedit.org/index.php?page=devel.

  13. https://bugs.eclipse.org/bugs/.

  14. http://eclipse.org/mylyn/.

  15. Detailed description on interaction events can be found at—https://wiki.eclipse.org/Mylyn/Integrator_Reference#Interaction_events.

  16. We also assume that Eclipse developers report all the bugs identified before they are fixed.

  17. http://conexp.sourceforge.net/.

  18. Detailed description on annotating summaries with SCUs is discussed at—http://www1.cs.columbia.edu/~ani/DUC2005/AnnotationGuide.htm.

  19. The DUCView tool to annotate SCUs can be downloaded from—http://www1.cs.columbia.edu/~ani/DUC2005/Tool.html.

References

  • Ali, N., Sharafl, Z., Gueheneuc, Y., & Antoniol, G. (2012). An empirical study on requirements traceability using eye-tracking. In: The 28th IEEE international conference on software maintenance, ICSM’12, pp. 191–200.

  • Bednarik, R., & Tukiainen, M. (2006). An eye-tracking methodology for characterizing program comprehension processes. In: Proceedings of the symposium on eye tracking research and applications, ETRA ’06, pp. 125–132.

  • Biggers, L. R., Eddy, B. P., Kraft, N. A., & Etzkorn, L. H. (2011). Toward a metrics suite for source code lexicons. In: The IEEE international conference on software maintenance, ICSM’11, pp. 492–495.

  • Biggerstaff, T. J., Mitbander, B. G., Webster, D. (1993). The concept assignment problem in program understanding. In: The IEEE international conference on software engineering, ICSE’93, pp. 482–498.

  • Brooks, R. (1980). Studying programmer behavior experimentally: The problems of proper methodology. Communications of the ACM, 23(4), 207–213.

    Article  Google Scholar 

  • Brooks, R. (1983). Towards a theory of the comprehension of computer program. International Journal of Man-Machine Studies, 18(6), 543–554.

    Article  Google Scholar 

  • Corbi, T. A. (1989). Program understanding: Challenge for the 1990’s. IBM Systems Journal, 28(2), 294–306.

    Article  Google Scholar 

  • Crosby, M. E., Scholtz, J., & Wiedenbeck, S. (2002). The roles beacons play in comprehension for novice and expert programmers. In: Programmers, 14th workshop of the psychology of programming interest group, Brunel University, PPIG’02, pp. 58–73.

  • Deissenboeck, F., & Pizka, M. (2006). Concise and consistent naming. Software Quality Journal, 14(3), 261–282.

    Article  Google Scholar 

  • Fritz, T., Murphy, G. C., & Hill, E. (2007). Does a programmer’s activity indicate knowledge of code? In: Proceedings of the 6th joint meeting of the european software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering, ESEC-FSE ’07, pp. 341–350.

  • Geer, D. (2005). Eclipse becomes the dominant java IDE. Computer, 38(7), 16–18.

    Article  Google Scholar 

  • Gellenbeck, E. M., & Cook, C. R. (1991). An investigation of procedure and variable names as beacons during program comprehension. Technical report, Corvallis, OR, USA.

  • Gu, Z., Schleck, D., Barr, E. T., & Su, Z. (2014). Capturing and exploiting IDE interactions. In: Proceedings of the 2014 ACM international symposium on new ideas, new paradigms, and reflections on programming & software, onward! ’14, pp. 83–94.

  • Haiduc, S., Aponte, J., & Marcus, A. (2010). Supporting program comprehension with source code summarization. In: The ACM/IEEE international conference on software engineering, ICSE’10, pp. 223–226.

  • Hamou-Lhadj, A., Lethbridge, T. (2006). Summarizing the content of large traces to facilitate the understanding of the behaviour of a software system. In: The IEEE international conference on program comprehension, ICPC’06, pp. 181–190.

  • Hesse, W., & Tilley, T. (2005). Formal concept analysis used for software analysis and modelling. In: Formal concept analysis (FCA), lecture notes in computer science, vol. 3626, pp. 288–303.

  • Karahasanovic, A., Levine, A. K., & Thomas, R. C. (2007). Comprehension strategies and difficulties in maintaining object-oriented systems: An explorative study. The Journal of Systems and Software, 80(9), 1541–1559.

    Article  Google Scholar 

  • Katragadda, R., Pingali, P., & Varma, V. (2009). Sentence position revisited: A robust light-weight update summarization baseline algorithm. In: Proceedings of the third international workshop on cross lingual information access: Addressing the information need of multilingual societies, CLIAWS3 ’09, pp. 46–52.

  • Kersten, M., & Murphy, G. C. (2006). Using task context to improve programmer productivity. In: Proceedings of the 14th ACM SIGSOFT international symposium on foundations of software engineering, SIGSOFT ’06/FSE-14, pp. 1–11.

  • Ko, A. J., Myers, B. A., Coblenz, M. J., & Aung, H. H. (2006). An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks. The IEEE Transactions on Software Engineering, 32(12), 971–987.

    Article  Google Scholar 

  • LaToza, T. D., Venolia, G., & DeLine, R. (2006). Maintaining mental models: A study of developer work habits. In: The ACM/IEEE international conference on software engineering, ICSE’06, pp. 492–501.

  • LaToza, T. D., Garlan, D., Herbsleb, J. D., & Myers, B. A. (2007). Program comprehension as fact finding. In: Proceedings of the the 6th joint meeting of the European Software Engineering conference and the ACM SIGSOFT symposium on The Foundations of Software Engineering, ESEC-FSE’07, pp. 361–370.

  • Lawrance, J., Bellamy, R., Burnett, M., & Rector, K. (2008). Using information scent to model the dynamic foraging behavior of programmers in maintenance tasks. In: The ACM conference on human factors in computing systems (CHI), CHI’08, pp. 1323–1332.

  • Letovsky, S. (1986). Cognitive processes in program comprehension. In: The first workshop on empirical studies of programmers on empirical studies of programmers, pp. 58–79.

  • von Mayrhauser, A., Vans, A. M., & Howe, A. E. (1997). Program understanding behaviour during enhancement of large-scale software. Journal of Software Maintenance, 9(5), 299–327.

    Article  Google Scholar 

  • Moreno, L., Aponte, J., Sridhara, G., Marcus, A., Pollock, L. L., & Vijay-Shanker, K. (2013). Automatic generation of natural language summaries for java classes. In: The IEEE international conference on program comprehension, ICPC’13, pp. 23–32.

  • Murphy, G. C., Kersten, M., & Findlater, L. (2006). How are java software developers using the Eclipse IDE? IEEE Software, 23(4), 76–83.

    Article  Google Scholar 

  • Nenkova, A., & Passonneau, R. J. (2004). Evaluating content selection in summarization: The pyramid method. In: The North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HTL’04, pp. 145–152.

  • Pennington, N. (1987). Stimulus structures and mental representations in expert comprehension of computer programs. Cognitive psychology, 19(3), 295–341.

    Article  Google Scholar 

  • Pirolli, P. (1997). Computational models of information scent following in a very large browsable text collection. In: The ACM SIGCHI conference on human factors in computing systems, CHI’97, pp. 3–10.

  • Pirolli, P., & Card, S. (1999). Information foraging. Psychological Review, 106(4), 643–675.

    Article  Google Scholar 

  • Rastkar, S., Murphy, G. C., & Bradley, A. W. J. (2011). Generating natural language summaries for crosscutting source code concerns. In: The IEEE international conference on software maintenance, ICSM’11, pp. 103–112.

  • Robillard, M., Coelho, W., & Murphy, G. (2004). How effective developers investigate source code: An exploratory study. The IEEE Transactions on Software Engineering, 30(12), 889–903.

    Article  Google Scholar 

  • Roehm, T., Tiarks, R., Koschke, R., & Maalej, W. (2012). How do professional developers comprehend software? In: Proceedings of the 34th international conference on software engineering, ICSE ’12, pp. 255–265.

  • Sharif, B., & Maletic, J. I. (2010) An eye tracking study on camelcase and under-score identifier styles. In: Proceedings of the IEEE 18th international conference on program comprehension, ICPC ’10, pp. 196–205.

  • Singer, J., Lethbridge, T., Vinson, N., & Anquetil, N. (1997). An examination of software engineering work practices. In: Proceedings of the 1997 conference of the center for advanced studies on collaborative research, CASCON ’97, pp. 209–223.

  • Soloway, E., & Ehrlich, K. (1984). Empirical studies of programming knowledge. IEEE Transactions on Software Engineering, 10(5), 595–609.

    Article  Google Scholar 

  • Sridhara, G., Pollock, L.L., & Vijay-Shanker, K. (2011) Automatically detecting and describing high level actions within methods. In: The ACM/IEEE international conference on software engineering (ICSE), pp. 101–110.

  • Stefik, A., & Siebert, S. (2013). An empirical investigation into programming language syntax. Transactions on Computing Education, 13(4), 19:1–19:40.

    Article  Google Scholar 

  • Storey, M. A. (2006). Theories, tools and research methods in program comprehension: Past, present and future. Software Quality Control, 14(3), 187–208.

    Article  Google Scholar 

  • Storey, M. A. D., Cheng, L. T., Singer, J., Muller, M. J., Myers, D., & Ryall, J. (2007). How programmers can turn comments into waypoints for code navigation. In: The IEEE international conference on software maintenance, ICSM ’07, pp. 265–274.

  • Vlasenko, J. (2011). Exploring developer’s tool path. In: Proceedings of the ACM international conference companion on object oriented programming systems languages and applications companion, OOPSLA ’11, pp. 219–220.

  • Von Mayrhauser, A., & Vans, A. M. (1995). Program comprehension during software maintenance and evolution. Computer, 28(8), 44–55.

    Article  Google Scholar 

  • Wiedenbeck, S., & Scholtz, J. (1989). Beacons: A knowledge structure in program comprehension. In: Proceedings of the third international conference on human-computer interaction on designing and using human-computer interfaces and knowledge based systems (2nd ed.), pp. 82–87.

  • Ying, A. T. T., & Robillard, M. P. (2014). Selection and presentation practices for code example summarization. In: Proceedings of the 22Nd ACM SIGSOFT international symposium on foundations of software engineering, FSE’14, pp. 460–471.

Download references

Acknowledgments

We like to thank all the developers from Infosys Ltd., Hyderabad, India, and IIIT-H graduate students for participating in our study. We also like to thank all the anonymous reviews for their valuable feedback.

Author information

Authors and Affiliations

  1. Software Engineering Research Center, International Institute of Information Technology (IIIT-H), Hyderabad, 500 032, India

    Naveen Kulkarni & Vasudeva Varma

  2. Infosys Ltd., Bengaluru, India

    Naveen Kulkarni

Authors
  1. Naveen Kulkarni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vasudeva Varma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Naveen Kulkarni.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kulkarni, N., Varma, V. Supporting comprehension of unfamiliar programs by modeling cues. Software Qual J 25, 307–340 (2017). https://doi.org/10.1007/s11219-015-9303-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-015-9303-5

Keywords

Search

Navigation