Advertisement

Measuring the Odds of Statements Being Faulty

  • Xiaozhen Xue
  • Akbar Siami Namin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7896)

Abstract

The statistics captured during testing a faulty program are the primary source of information for effective fault localization. A typical ranking metric estimates suspiciousness of executable statements and ranks them according to the estimated scores. The coverage-based ranking schemes, such as the metric used in Tarantula and Ochiai score, utilize the execution profile of each test case, including code coverage and the statistics associated with the number of failing and passing test cases. Although the coverage-based fault localization metrics could be extended to hypothesis testing and in particular to the chi-square test associated with crosstab or known as contingency tables, not all contingency table association metrics are explored and studied.

We introduce the odds ratio metric and its application to the fault localization problem. The odds-ratio metric has been used extensively in categorical data analysis and in measuring the association of dependency between dichotomous variables. However, its application to fault localization metric is new. Furthermore, we investigate the effectiveness of conditional odds ratio metric for fault localization when there are multiple faults in the programs. Our experimental results show that the odds ratio metric performs better than the other ranking metrics studied for single faults, whereas, the conditional odds ratio ranking scheme is competitive when there are multiple faults in the software under test.

Keywords

testing fault localization 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Jones, J.A.: Fault localization using visualization of test information. In: ICSE, pp. 54–56 (2004)Google Scholar
  2. 2.
    Abreu, R., Zoeteweij, P., van Gemund, J.: On the accuracy of spectrum-based fault localization. In: Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques, pp. 89–98 (2007)Google Scholar
  3. 3.
    Jones, J.A., Harrold, M.J.: Empirical evaluation of the tarantula automatic fault-localization technique. In: ASE, pp. 273–282 (2005)Google Scholar
  4. 4.
    Naish, L., Lee, H.J., Ramamohanarao, K.: A model for spectra-based software diagnosis. ACM Transactions on Software Engineering and Methodology 20 (August 2011)Google Scholar
  5. 5.
    Wong, W.E., Wei, T., Qi, Y., Zhao, L.: A crosstab-based statistical method for effective fault localization. In: ICST, pp. 42–51 (2008)Google Scholar
  6. 6.
    Jin, X., Xu, A., Bie, R., Guo, P.: Machine learning techniques and chi-square feature selection for cancer classification using SAGE gene expression profiles. In: Li, J., Yang, Q., Tan, A.-H. (eds.) BioDM 2006. LNCS (LNBI), vol. 3916, pp. 106–115. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Liu, H., Motoda, H.: Computational Methods of Feature Selection. Chapman and Hall/CRC (2008)Google Scholar
  8. 8.
    Santelices, R.A., Jones, J.A., Yu, Y., Harrold, M.J.: Lightweight fault-localization using multiple coverage types. In: ICSE, pp. 56–66 (2009)Google Scholar
  9. 9.
    Do, H., Elbaum, S.G., Rothermel, G.: Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering: An International Journal 10(4), 405–435 (2005)CrossRefGoogle Scholar
  10. 10.
    Ma, Y.-S., Offutt, J., Kwon, Y.R.: MuJava: a mutation system for Java. In: ICSE, pp. 827–830 (2006)Google Scholar
  11. 11.
    Agresti, A.: An Introduction to Categorical Data Analyis, 2nd edn. Wiley, NY (2007)CrossRefGoogle Scholar
  12. 12.
    Anderson, C.J.: Applied categorical data analysis - a graduate course (February 2013), http://courses.education.illinois.edu/EdPsy589/#general
  13. 13.
    Shadish, W., Cook, T., Campbell, D.: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company, New York (2002)Google Scholar
  14. 14.
    Andrews, J.H., Briand, L.C., Labiche, Y.: Is mutation an appropriate tool for testing experiments? In: ICSE, pp. 402–411 (2005)Google Scholar
  15. 15.
    Zeller, A., Hildebrandt, R.: Simplifying and isolating failure-inducing input. IEEE Transactions on Software Engineering 28(2), 183–200 (2002)CrossRefGoogle Scholar
  16. 16.
    Cleve, H., Zeller, A.: Locating causes of program failures. In: ICSE, pp. 342–351 (2005)Google Scholar
  17. 17.
    Buse, R.P.L., Weimer, W.: The road not taken: Estimating path execution frequency statically. In: ICSE, pp. 144–154 (2009)Google Scholar
  18. 18.
    Zhang, X., Gupta, N., Gupta, R.: A study of effectiveness of dynamic slicing in locating real faults. Empirical Software Engineering 12(2), 143–160 (2007)CrossRefGoogle Scholar
  19. 19.
    Agrawal, H., Horgan, J.R., London, S., Wong, W.E.: Fault localization using execution slices and dataflow tests. In: Proceedings of IEEE Software Reliability Engineering, pp. 143–151 (1995)Google Scholar
  20. 20.
    Hao, D., Zhang, L., Xie, T., Mei, H., Sun, J.: Interactive fault localization using test information. Journal of Computer Science Technology 24(5), 962–974 (2009)CrossRefGoogle Scholar
  21. 21.
    Briand, L.C.: Novel applications of machine learning in software testing. In: QSIC, pp. 3–10 (2008)Google Scholar
  22. 22.
    Liblit, B., Naik, M., Zheng, A.X., Aiken, A., Jordan, M.I.: Scalable statistical bug isolation. In: PLDI, pp. 15–26 (2005)Google Scholar
  23. 23.
    Baah, G.K., Podgurski, A., Harrold, M.J.: Causal inference for statistical fault localization. In: ISSTA, Trento, Italy (July 2010)Google Scholar
  24. 24.
    Feng, M., Gupta, R.: Learning universal probabilistic models for fault localization. In: ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, Toronto, Canada (June 2010)Google Scholar
  25. 25.
    Briand, L.C., Labiche, Y., Liu, X.: Using machine learning to support debugging with tarantula. In: International Symposium on Software Reliability Engineering, pp. 137–146 (2007)Google Scholar
  26. 26.
    Zimmermann, T., Nagappan, N.: Predicting defects using network analysis on dependency graphs. In: ICSE, pp. 531–540 (2008)Google Scholar
  27. 27.
    DiGiuseppe, N., Jones, J.A.: On the influence of multiple faults on coverage-based fault localization. In: ISSTA, pp. 210–220 (2011)Google Scholar
  28. 28.
    Lucia, Lo, D., Jiang, L., Budi, A.: Comprehensive evaluation of association measures for fault localization. In: ICSM, pp. 1–10 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Xiaozhen Xue
    • 1
  • Akbar Siami Namin
    • 1
  1. 1.Department of Computer ScienceTexas Tech UniversityLubbockUSA

Personalised recommendations