Software Quality Journal

, Volume 25, Issue 1, pp 83–110 | Cite as

Studying the advancement in debugging practice of professional software developers

  • Michael PerscheidEmail author
  • Benjamin Siegmund
  • Marcel Taeumel
  • Robert Hirschfeld


In 1997, Henry Lieberman stated that debugging is the dirty little secret of computer science. Since then, several promising debugging technologies have been developed such as back-in-time debuggers and automatic fault localization methods. However, the last study about the state-of-the-art in debugging is still more than 15 years old and so it is not clear whether these new approaches have been applied in practice or not. For that reason, we investigate the current state of debugging in a comprehensive study. First, we review the available literature and learn about current approaches and study results. Second, we observe several professional developers while debugging and interview them about their experiences. Third, we create a questionnaire that serves as the basis for a larger online debugging survey. Based on these results, we present new insights into debugging practice that help to suggest new directions for future research.


Debugging Literature review Field study Online survey 


  1. Agans, D. J. (2002). Debugging: The 9 indispensable rules for finding even the most elusive software and hardware problems. AMACOM Div American Mgmt Assn.Google Scholar
  2. Agrawal, H., Horgan, J., London, S., & Wong, W. (1995). Fault localization using execution slices and dataflow tests. In conference on software reliability engineering (pp. 143–151).Google Scholar
  3. Ahmadzadeh, M., Elliman, D., & Higgins, C. (2005). An analysis of patterns of debugging among novice computer science students. ACM SIGCSE Bulletin, 37(3), 84–88.CrossRefGoogle Scholar
  4. Artzi, S., Dolby, J., Tip, F., & Pistoia, M. (2010). Practical fault localization for dynamic web applications. In international conference on software engineering (pp. 265–274). ACM.Google Scholar
  5. Arumuga Nainar, P., & Liblit, B. (2010). Adaptive bug isolation. In international conference on software engineering (pp. 255–264). ACM.Google Scholar
  6. Baah, G. K., Podgurski, A., & Harrold, M. J. (2010). Causal inference for statistical fault localization. In international symposium on software testing and analysis (pp. 73–84). ACM.Google Scholar
  7. Ballou, M. C. (2008). Improving software quality to drive business agility. IDC Survey and White Paper.Google Scholar
  8. Chmiel, R., & Loui, M. C. (2004). Debugging: From novice to expert. ACM SIGCSE Bulletin, 36(1), 17–21.CrossRefGoogle Scholar
  9. Cleve, H., & Zeller, A. (2005). Locating causes of program failures. In international conference on software engineering (pp. 342–351). ACM.Google Scholar
  10. Dallmeier, V., Lindig, C., & Zeller, A. (2005). Lightweight defect localization for java. In European conference on object-oriented programming (pp. 528–550). Springer.Google Scholar
  11. Eisenstadt, M. (1997). My hairiest bug war stories. Communications of the ACM, 40(4), 30–37.CrossRefGoogle Scholar
  12. Gould, J. D. (1975). Some psychological evidence on how people debug computer programs. International Journal of Man-Machine Studies, 7(2), 151–182.CrossRefzbMATHGoogle Scholar
  13. Gould, J. D., & Drongowski, P. (1974). An exploratory study of computer program debugging. The Journal of the Human Factors and Ergonomics Society, 16(3), 258–277.CrossRefGoogle Scholar
  14. Grötker, T., Holtmann, U., Keding, H., & Wloka, M. (2012). The developer’s guide to debugging (2nd ed.). NewYork: Self-publishing company.Google Scholar
  15. Gupta, N., He, H., Zhang, X., & Gupta, R. (2005). Locating faulty code using failure-inducing chops. In international conference on automated software engineering (pp. 263–272). ACM.Google Scholar
  16. Hailpern, B., & Santhanam, P. (2002). Software debugging, testing, and verification. IBM Systems Journal, 41(1), 4–12.CrossRefGoogle Scholar
  17. Hanks, B., & Brandt, M. (2009). Successful and unsuccessful problem solving approaches of novice programmers. ACM SIGCSE Bulletin, 41(1), 24–28.CrossRefGoogle Scholar
  18. James, S., Bidgoli, M., & Hansen, J. (2008). Why sally and joey can’t debug: Next generation tools and the perils they pose. Journal of Computing Sciences in Colleges, 24(1), 27–35.Google Scholar
  19. Janssen, T., Abreu, R., & van Gemund, A. J. (2009). Zoltar: A toolset for automatic fault localization. In international conference on automated software engineering (pp. 662–664). IEEE Computer Society.Google Scholar
  20. Jeffrey, D., Gupta, N., & Gupta, R. (2008). Fault localization using value replacement. In international symposium on software testing and analysis (pp. 167–178). ACM.Google Scholar
  21. Jiang, L., & Su, Z. (2007). Context-aware statistical debugging: From bug predictors to faulty control flow paths. In international conference on automated software engineering (pp. 184–193). ACM.Google Scholar
  22. Jones, J. A., Bowring, J. F., & Harrold, M. J. (2007). Debugging in parallel. In international symposium on software testing and analysis (pp. 16–26). ACM.Google Scholar
  23. Kernighan, B. W., & Plauger, P. J. (1978). The elements of programming style (Vol. 1). NewYork: McGraw-Hill.zbMATHGoogle Scholar
  24. LaToza, T. D., & Myers, B. A. (2010). Developers ask reachability questions. In international conference on software engineering, vol. 1, (pp. 185–194). IEEE.Google Scholar
  25. Lencevicius, R. (2000). On-the-fly query-based debugging with examples. arXiv.Google Scholar
  26. Lewis, B. (2003). Debugging backwards in time. In proceedings of the international workshop on automated debugging, AADEBUG (pp. 225–235). Arxiv.Google Scholar
  27. Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005). Scalable statistical bug isolation. ACM SIGPLAN Notices, 40(6), 15–26.CrossRefGoogle Scholar
  28. Lieberman, H. (1997). The debugging scandal and what to do about it (introduction to the special section). Community ACM, 40(4), 26–29.CrossRefGoogle Scholar
  29. Liu, C., Yan, X., Fei, L., Han, J., & Midkiff, S. P. (2005). Sober: Statistical model-based bug localization. ACM SIGSOFT Software Engineering Notes, 30(5), 286–295.CrossRefGoogle Scholar
  30. Metzger, R. C. (2004). Debugging by thinking: A multidisciplinary approach. NewYork: Elsevier Digital Press.Google Scholar
  31. Murphy, L., Lewandowski, G., McCauley, R., Simon, B., Thomas, L., & Zander, C. (2008). Debugging: The good, the bad, and the quirky-a qualitative analysis of novices’ strategies. ACM SIGCSE Bulletin, 40(1), 163–167.CrossRefGoogle Scholar
  32. Park, S., Vuduc, R. W., & Harrold, M. J. (2010). Falcon: Fault localization in concurrent programs. In international conference on software engineering (pp. 245–254). ACM.Google Scholar
  33. Pearson, K. (1895). Notes on regression and inheritance in the case of two parents. In proceedings of the royal society of London (pp. 240–242). The royal society.Google Scholar
  34. Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. In philosophical magazine (pp. 157–175).Google Scholar
  35. Perscheid, M. (2013). Test-driven fault navigation for debugging reproducible failures. Ph.D. thesis, Hasso Plattner Institute, University of Potsdam.Google Scholar
  36. Renieres, M., & Reiss, S. P. (2003). Fault localization with nearest neighbor queries. In international conference on automated software engineering (pp. 30–39). IEEE.Google Scholar
  37. Vessey, I. (1985). Expertise in debugging computer programs: A process analysis. International Journal of Man-Machine Studies, 23(5), 459–494.CrossRefGoogle Scholar
  38. Weiser, M. (1982). Programmers use slices when debugging. Communications of the ACM, 25(7), 446–452.CrossRefGoogle Scholar
  39. Wong, W. E., & Debroy, V. (2009). A survey of software fault localization. Department of Computer Science, University of Texas at Dallas, Technical Report UTDCS-45-09.Google Scholar
  40. Yilmaz, C., Paradkar, A., & Williams, C. (2008). Time will tell: Fault localization using time spectra. In international conference on software engineering (pp. 81–90). ACM.Google Scholar
  41. Zeller, A. (2002). Isolating cause-effect chains from computer programs. In symposium on foundations of software engineering (pp. 1–10). ACM.Google Scholar
  42. Zeller, A. (2009). Why programs fail: A guide to systematic debugging. Burlington: Morgan Kaufmann.Google Scholar
  43. Zhang, X., Gupta, N., & Gupta, R. (2006). Locating faults through automated predicate switching. In international conference on software engineering (pp. 272–281). ACM.Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Michael Perscheid
    • 1
    Email author
  • Benjamin Siegmund
    • 2
  • Marcel Taeumel
    • 2
  • Robert Hirschfeld
    • 2
  1. 1.SAP Innovation CenterPotsdamGermany
  2. 2.Hasso Plattner InstituteUniversity of PotsdamPotsdamGermany

Personalised recommendations