Advertisement

An Approach to Generate Effective Fault Localization Methods for Programs

  • Babak Bagheri
  • Mohammad Rezaalipour
  • Mojtaba Vahidi-AslEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11761)

Abstract

Software Debugging is a tedious and costly task in software development life-cycle. Thus, various automated fault localization approaches have been proposed to address this problem, among which, spectrum-based fault localization has attracted a lot of attention. Using various formulas, known as ranking metrics, spectrum-based fault localization techniques assign scores to the entities of programs (e.g., statements) based on their suspiciousness of being the root cause of failures. Despite the obvious advantages of spectrum-based fault localization techniques, such as being lightweight, they cannot effectively locate faults in every program owing to the fact that they do not consider the characteristics of the programs. We believe that program characteristics can be helpful at finding the right ranking metrics for programs, and they can assist at combining several existing ones to produce a customized ranking metric specific to a given program.

In this paper, we have proposed an approach which combines 40 different ranking metrics to generate a new ranking metric specific to a given program. Employing mutation testing operators, the proposed approach retrieves information from the program and then, using different preferential voting systems, it combines various ranking metrics based on the collected information. We have evaluated our approach on 154 faulty versions from eight different programs of Space and Siemens test suite and compare it with nine state-of-the-art ranking metrics. The experimental results indicate that the ranking metrics generated by our approach is superior with respect to evaluation metrics such as the Exam score and TOP-N.

Keywords

Software fault localization Spectrum-based fault localization Mutation testing Ranking metric Preferential voting system 

References

  1. 1.
    Abreu, R., Zoeteweij, P., Van Gemund, A.J.: An evaluation of similarity coefficients for software fault localization. In: Proceedings of the 12th Pacific Rim International Symposium on Dependable Computing, pp. 39–46 (2006).  https://doi.org/10.1109/PRDC.2006.18
  2. 2.
    Amy, D.J.: Behind the Ballot Box: A Citizen’s Guide to Voting Systems. Greenwood Publishing Group (2000)Google Scholar
  3. 3.
    Andrews, J.H., Briand, L.C., Labiche, Y.: Is mutation an appropriate tool for testing experiments? (software testing). In: Proceedings of the 27th International Conference on Software Engineering, pp. 402–411 (2005).  https://doi.org/10.1109/ICSE.2005.1553583
  4. 4.
    Cary, D.: Estimating the margin of victory for instant-runoff voting. In: Website Proceedings of the Conference on Electronic Voting Technology/Workshop on Trustworthy Elections (2011)Google Scholar
  5. 5.
    Cranor, L.F.: Declared-strategy voting: an instrument for group decision-making. Ph.D. thesis, Washington University (1996)Google Scholar
  6. 6.
    Dallmeier, V., Lindig, C., Zeller, A.: Lightweight bug localization with ample. In: Proceedings of the 6th International Symposium on Automated Analysis-driven Debugging, pp. 99–104 (2005).  https://doi.org/10.1145/1085130.1085143
  7. 7.
    Do, H., Elbaum, S., Rothermel, G.: Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empir. Softw. Eng. 10(4), 405–435 (2005).  https://doi.org/10.1007/s10664-005-3861-2CrossRefGoogle Scholar
  8. 8.
    Gough, B., Stallman, R.M.: An Introduction to GCC for the GNU Compilers gcc and g++. Network Theory Ltd. (2004)Google Scholar
  9. 9.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Softw. Eng. 37(5), 649–678 (2011).  https://doi.org/10.1109/TSE.2010.62CrossRefGoogle Scholar
  10. 10.
    Jones, J.A., Harrold, M.J., Stasko, J.: Visualization of test information to assist fault localization. In: Proceedings of the 24th International Conference on Software Engineering, pp. 467–477 (2002).  https://doi.org/10.1145/581396.581397
  11. 11.
    Kemeny, J.G.: Mathematics without numbers. Daedalus 88(4), 577–591 (1959). https://www.mitpressjournals.org/loi/daedGoogle Scholar
  12. 12.
    Mao, X., Lei, Y., Dai, Z., Qi, Y., Wang, C.: Slice-based statistical fault localization. J. Syst. Softw. 89, 51–62 (2014).  https://doi.org/10.1016/j.jss.2013.08.031CrossRefGoogle Scholar
  13. 13.
    Monperrus, M.: Automatic software repair: a bibliography. ACM Comput. Surv. 51(1), 17:1–17:24 (2018).  https://doi.org/10.1145/3105906CrossRefGoogle Scholar
  14. 14.
    Nath, A., Domingos, P.: Learning tractable probabilistic models for fault localization. In: Proceedings of the 13th AAAI Conference on Artificial Intelligence, pp. 1294–1301 (2016)Google Scholar
  15. 15.
    Parnin, C., Orso, A.: Are automated debugging techniques actually helping programmers? In: Proceedings of the International Symposium on Software Testing and Analysis, pp. 199–209 (2011).  https://doi.org/10.1145/2001420.2001445
  16. 16.
    Pearson, S., et al.: Evaluating and improving fault localization. In: Proceedings of the 39th International Conference on Software Engineering, pp. 609–620 (2017).  https://doi.org/10.1109/ICSE.2017.62
  17. 17.
    Saha, R.K., Lyu, Y., Yoshida, H., Prasad, M.R.: Elixir: effective object-oriented program repair. In: Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering, pp. 648–659 (2017).  https://doi.org/10.1109/ASE.2017.8115675
  18. 18.
    Schulze, M.: A new monotonic, clone-independent, reversal symmetric, and condorcet-consistent single-winner election method. Soc. Choice Welf. 36(2), 267–303 (2011).  https://doi.org/10.1007/s00355-010-0475-4MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    de Souza, H.A., Chaim, M.L., Kon, F.: Spectrum-based software fault localization: a survey of techniques, advances, and challenges (2016). http://arxiv.org/abs/1607.04347
  20. 20.
    Tassey, G.: The economic impacts of inadequate infrastructure for software testing. Technical report 7007.011, National Institute of Standards and Technology (2002). https://www.nist.gov/document/report02-3pdf
  21. 21.
    Thung, F., Wang, S., Lo, D., Jiang, L.: An empirical study of bugs in machine learning systems. In: Proceedings of the 23rd International Symposium on Software Reliability Engineering, pp. 271–280 (2012).  https://doi.org/10.1109/ISSRE.2012.22
  22. 22.
    Wang, Y., Patil, H., Pereira, C., Lueck, G., Gupta, R., Neamtiu, I.: Drdebug: Deterministic replay based cyclic debugging with dynamic slicing. In: Proceedings of the Annual IEEE/ACM International Symposium on Code Generation and Optimization, pp. 98:98–98:108 (2014).  https://doi.org/10.1145/2544137.2544152
  23. 23.
    Wong, W.E., Gao, R., Li, Y., Abreu, R., Wotawa, F.: A survey on software fault localization. IEEE Trans. Softw. Eng. 42(8), 707–740 (2016).  https://doi.org/10.1109/TSE.2016.2521368CrossRefGoogle Scholar
  24. 24.
    Xie, X., Chen, T.Y., Kuo, F.C., Xu, B.: A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization. ACM Trans. Softw. Eng. Methodol. 22(4), 31:1–31:40 (2013).  https://doi.org/10.1145/2522920.2522924CrossRefGoogle Scholar
  25. 25.
    Xie, X., Kuo, F.-C., Chen, T.Y., Yoo, S., Harman, M.: Provably optimal and human-competitive results in SBSE for spectrum based fault localisation. In: Ruhe, G., Zhang, Y. (eds.) SSBSE 2013. LNCS, vol. 8084, pp. 224–238. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-39742-4_17CrossRefGoogle Scholar
  26. 26.
    Xuan, J., Monperrus, M.: Learning to combine multiple ranking metrics for fault localization. In: Proceedings of the IEEE International Conference on Software Maintenance and Evolution, pp. 191–200 (2014).  https://doi.org/10.1109/ICSME.2014.41
  27. 27.
    Xuan, J., Monperrus, M.: Test case purification for improving fault localization. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 52–63 (2014).  https://doi.org/10.1145/2635868.2635906
  28. 28.
    Yoo, S., Xie, X., Kuo, F.C., Chen, T.Y., Harman, M.: No pot of gold at the end of program spectrum rainbow: greatest risk evaluation formula does not exist. Technical report Research Note RN/14/14, University College London (2014). http://www.cs.ucl.ac.uk/fileadmin/UCL-CS/research/Research_Notes/rn-14-14_03.pdf
  29. 29.
    Zhang, M., Li, X., Zhang, L., Khurshid, S.: Boosting spectrum-based fault localization using PageRank. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 261–272 (2017).  https://doi.org/10.1145/3092703.3092731
  30. 30.
    Zhang, S., Zhang, C.: Software bug localization with Markov logic. In: Companion Proceedings of the 36th International Conference on Software Engineering, pp. 424–427 (2014).  https://doi.org/10.1145/2591062.2591099

Copyright information

© IFIP International Federation for Information Processing 2019

Authors and Affiliations

  • Babak Bagheri
    • 1
  • Mohammad Rezaalipour
    • 1
  • Mojtaba Vahidi-Asl
    • 1
    Email author
  1. 1.Faculty of Computer Science and EngineeringShahid Beheshti University G. C.TehranIran

Personalised recommendations