Studying the advancement in debugging practice of professional software developers

Abstract

In 1997, Henry Lieberman stated that debugging is the dirty little secret of computer science. Since then, several promising debugging technologies have been developed such as back-in-time debuggers and automatic fault localization methods. However, the last study about the state-of-the-art in debugging is still more than 15 years old and so it is not clear whether these new approaches have been applied in practice or not. For that reason, we investigate the current state of debugging in a comprehensive study. First, we review the available literature and learn about current approaches and study results. Second, we observe several professional developers while debugging and interview them about their experiences. Third, we create a questionnaire that serves as the basis for a larger online debugging survey. Based on these results, we present new insights into debugging practice that help to suggest new directions for future research.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Notes

  1. 1.

    The Pearson coefficient can be used to find linear relations between numeric variables. Values between \(-1\) and 1 indicate how well the actual relation can be approximated by a linear function (Pearson 1895).

  2. 2.

    As early results of the survey have shown a large number of parallel behavioral bugs, we added it as a new choice that was not represented in Eisenstadt’s original version.

  3. 3.

    A χ 2 test can be used for any kind of discretely categorized data with a large number of samples in each category. A large value indicates dependence and a low independence (Pearson 1900).

References

  1. Agans, D. J. (2002). Debugging: The 9 indispensable rules for finding even the most elusive software and hardware problems. AMACOM Div American Mgmt Assn.

  2. Agrawal, H., Horgan, J., London, S., & Wong, W. (1995). Fault localization using execution slices and dataflow tests. In conference on software reliability engineering (pp. 143–151).

  3. Ahmadzadeh, M., Elliman, D., & Higgins, C. (2005). An analysis of patterns of debugging among novice computer science students. ACM SIGCSE Bulletin, 37(3), 84–88.

    Article  Google Scholar 

  4. Artzi, S., Dolby, J., Tip, F., & Pistoia, M. (2010). Practical fault localization for dynamic web applications. In international conference on software engineering (pp. 265–274). ACM.

  5. Arumuga Nainar, P., & Liblit, B. (2010). Adaptive bug isolation. In international conference on software engineering (pp. 255–264). ACM.

  6. Baah, G. K., Podgurski, A., & Harrold, M. J. (2010). Causal inference for statistical fault localization. In international symposium on software testing and analysis (pp. 73–84). ACM.

  7. Ballou, M. C. (2008). Improving software quality to drive business agility. IDC Survey and White Paper.

  8. Chmiel, R., & Loui, M. C. (2004). Debugging: From novice to expert. ACM SIGCSE Bulletin, 36(1), 17–21.

    Article  Google Scholar 

  9. Cleve, H., & Zeller, A. (2005). Locating causes of program failures. In international conference on software engineering (pp. 342–351). ACM.

  10. Dallmeier, V., Lindig, C., & Zeller, A. (2005). Lightweight defect localization for java. In European conference on object-oriented programming (pp. 528–550). Springer.

  11. Eisenstadt, M. (1997). My hairiest bug war stories. Communications of the ACM, 40(4), 30–37.

    Article  Google Scholar 

  12. Gould, J. D. (1975). Some psychological evidence on how people debug computer programs. International Journal of Man-Machine Studies, 7(2), 151–182.

    Article  MATH  Google Scholar 

  13. Gould, J. D., & Drongowski, P. (1974). An exploratory study of computer program debugging. The Journal of the Human Factors and Ergonomics Society, 16(3), 258–277.

    Article  Google Scholar 

  14. Grötker, T., Holtmann, U., Keding, H., & Wloka, M. (2012). The developer’s guide to debugging (2nd ed.). NewYork: Self-publishing company.

    Google Scholar 

  15. Gupta, N., He, H., Zhang, X., & Gupta, R. (2005). Locating faulty code using failure-inducing chops. In international conference on automated software engineering (pp. 263–272). ACM.

  16. Hailpern, B., & Santhanam, P. (2002). Software debugging, testing, and verification. IBM Systems Journal, 41(1), 4–12.

    Article  Google Scholar 

  17. Hanks, B., & Brandt, M. (2009). Successful and unsuccessful problem solving approaches of novice programmers. ACM SIGCSE Bulletin, 41(1), 24–28.

    Article  Google Scholar 

  18. James, S., Bidgoli, M., & Hansen, J. (2008). Why sally and joey can’t debug: Next generation tools and the perils they pose. Journal of Computing Sciences in Colleges, 24(1), 27–35.

    Google Scholar 

  19. Janssen, T., Abreu, R., & van Gemund, A. J. (2009). Zoltar: A toolset for automatic fault localization. In international conference on automated software engineering (pp. 662–664). IEEE Computer Society.

  20. Jeffrey, D., Gupta, N., & Gupta, R. (2008). Fault localization using value replacement. In international symposium on software testing and analysis (pp. 167–178). ACM.

  21. Jiang, L., & Su, Z. (2007). Context-aware statistical debugging: From bug predictors to faulty control flow paths. In international conference on automated software engineering (pp. 184–193). ACM.

  22. Jones, J. A., Bowring, J. F., & Harrold, M. J. (2007). Debugging in parallel. In international symposium on software testing and analysis (pp. 16–26). ACM.

  23. Kernighan, B. W., & Plauger, P. J. (1978). The elements of programming style (Vol. 1). NewYork: McGraw-Hill.

    Google Scholar 

  24. LaToza, T. D., & Myers, B. A. (2010). Developers ask reachability questions. In international conference on software engineering, vol. 1, (pp. 185–194). IEEE.

  25. Lencevicius, R. (2000). On-the-fly query-based debugging with examples. arXiv.

  26. Lewis, B. (2003). Debugging backwards in time. In proceedings of the international workshop on automated debugging, AADEBUG (pp. 225–235). Arxiv.

  27. Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005). Scalable statistical bug isolation. ACM SIGPLAN Notices, 40(6), 15–26.

    Article  Google Scholar 

  28. Lieberman, H. (1997). The debugging scandal and what to do about it (introduction to the special section). Community ACM, 40(4), 26–29.

    Article  Google Scholar 

  29. Liu, C., Yan, X., Fei, L., Han, J., & Midkiff, S. P. (2005). Sober: Statistical model-based bug localization. ACM SIGSOFT Software Engineering Notes, 30(5), 286–295.

    Article  Google Scholar 

  30. Metzger, R. C. (2004). Debugging by thinking: A multidisciplinary approach. NewYork: Elsevier Digital Press.

    Google Scholar 

  31. Murphy, L., Lewandowski, G., McCauley, R., Simon, B., Thomas, L., & Zander, C. (2008). Debugging: The good, the bad, and the quirky-a qualitative analysis of novices’ strategies. ACM SIGCSE Bulletin, 40(1), 163–167.

    Article  Google Scholar 

  32. Park, S., Vuduc, R. W., & Harrold, M. J. (2010). Falcon: Fault localization in concurrent programs. In international conference on software engineering (pp. 245–254). ACM.

  33. Pearson, K. (1895). Notes on regression and inheritance in the case of two parents. In proceedings of the royal society of London (pp. 240–242). The royal society.

  34. Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. In philosophical magazine (pp. 157–175).

  35. Perscheid, M. (2013). Test-driven fault navigation for debugging reproducible failures. Ph.D. thesis, Hasso Plattner Institute, University of Potsdam.

  36. Renieres, M., & Reiss, S. P. (2003). Fault localization with nearest neighbor queries. In international conference on automated software engineering (pp. 30–39). IEEE.

  37. Vessey, I. (1985). Expertise in debugging computer programs: A process analysis. International Journal of Man-Machine Studies, 23(5), 459–494.

    Article  Google Scholar 

  38. Weiser, M. (1982). Programmers use slices when debugging. Communications of the ACM, 25(7), 446–452.

    Article  Google Scholar 

  39. Wong, W. E., & Debroy, V. (2009). A survey of software fault localization. Department of Computer Science, University of Texas at Dallas, Technical Report UTDCS-45-09.

  40. Yilmaz, C., Paradkar, A., & Williams, C. (2008). Time will tell: Fault localization using time spectra. In international conference on software engineering (pp. 81–90). ACM.

  41. Zeller, A. (2002). Isolating cause-effect chains from computer programs. In symposium on foundations of software engineering (pp. 1–10). ACM.

  42. Zeller, A. (2009). Why programs fail: A guide to systematic debugging. Burlington: Morgan Kaufmann.

    Google Scholar 

  43. Zhang, X., Gupta, N., & Gupta, R. (2006). Locating faults through automated predicate switching. In international conference on software engineering (pp. 272–281). ACM.

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Michael Perscheid.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Perscheid, M., Siegmund, B., Taeumel, M. et al. Studying the advancement in debugging practice of professional software developers. Software Qual J 25, 83–110 (2017). https://doi.org/10.1007/s11219-015-9294-2

Download citation

Keywords

  • Debugging
  • Literature review
  • Field study
  • Online survey