Advertisement

Regression Test Case Prioritisation for Guava

  • Yi Bian
  • Serkan Kirbas
  • Mark Harman
  • Yue Jia
  • Zheng Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9275)

Abstract

We present a three objective formulation of regression test prioritisation. Our formulation involves the well-known, and widely-used objectives of Average Percentage of Statement Coverage (APSC) and Effective Execution Time (EET). However, we additionally include the Average Percentage of Change Coverage (APCC), which has not previously been used in search-based regression test optimisation. We apply our approach to prioritise the base and the collection package of the Guava project, which contains over 26,815 test cases. Our results demonstrate the value of search-based test case prioritisation: the sequences we find require only 0.2 % of the 26,815 test cases and only 0.45 % of their effective execution time. However, we find solutions that achieve more than 99.9 % of both regression testing objectives; covering both changed code and existing code. We also investigate the tension between these two objectives for Guava.

Keywords

Regression testing Test case prioritisation NSGA-II 

References

  1. 1.
    Yoo, S., Harman, M.: Regression testing minimization, selection and prioritisation: a survey. Softw. Test. Verification Reliab. 22(2), 67–120 (2012)CrossRefGoogle Scholar
  2. 2.
    Rothermel, G., Untch, R.H., Chu, C., Harrold, M.J.: Prioritizing test cases for regression testing. IEEE Trans. Softw. Eng. 27(10), 929–948 (2001)CrossRefGoogle Scholar
  3. 3.
    Huang, P., Ma, X., Shen, D., Zhou, Y.: Performance regression testing target prioritisation via performance risk analysis. In: Proceedings of the 36th International Conference on Software Engineering, pp. 60–71. ACM, May 2014Google Scholar
  4. 4.
    Guava Project Web Site. https://github.com/google/guava
  5. 5.
  6. 6.
    Jiang, B., Chan, W.K.: On the integration of test adequacy, test case prioritisation, and statistical fault localization. In: 2010 10th International Conference on Quality Software (QSIC), pp. 377–384. IEEE (2010)Google Scholar
  7. 7.
    Elbaum, S., Malishevsky, A.G., Rothermel, G.: Test case prioritisation: a family of empirical studies. IEEE Trans. Softw. Eng. 28(2), 159–182 (2002)CrossRefGoogle Scholar
  8. 8.
    Li, Z., Harman, M., Hierons, R.M.: Search algorithms for regression test case prioritisation. IEEE Trans. Softw. Eng. 33(4), 225–237 (2007)CrossRefGoogle Scholar
  9. 9.
    Yoo, S., Harman, M.: Pareto efficient multi-objective test case selection. In: Proceedings of the 2007 International Symposium on Software Testing and Analysis, pp. 140–150. ACM (2007)Google Scholar
  10. 10.
    Deb, K., Pratap, A., Agarwal, S., et al.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)CrossRefGoogle Scholar
  11. 11.
    Harman, M.: Making the case for MORTO: multi objective regression test optimization. In: ICST Workshops, pp. 111–114 (2011)Google Scholar
  12. 12.
    Sun, W., Gao, Z., Yang, W., et al.: Multi-objective test case prioritization for GUI applications. In: Proceedings of the 28th Annual ACM Symposium on Applied Computing. pp. 1074–1079. ACM (2013)Google Scholar
  13. 13.
    Snchez, A.B., Segura, S., Ruiz-Corts, A.A.: Comparison of test case prioritization criteria for software product lines. In: 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation (ICST), pp. 41–50. IEEE (2014)Google Scholar
  14. 14.
    Li, Z., Bian, Y., Zhao, R., Cheng, J.: A fine-grained parallel multi-objective test case prioritization on GPU. In: Ruhe, G., Zhang, Y. (eds.) SSBSE 2013. LNCS, vol. 8084, pp. 111–125. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  15. 15.
    Epitropakis, M.G., Yoo, S., Harman, M., Burke, E.K.: Empirical evaluation of pareto efficient multi-objective regression test case prioritisation. In: Proceedings of the 2015 International Symposium on Software Testing and Analysis, pp. 234–245. ACM (2015)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Yi Bian
    • 2
  • Serkan Kirbas
    • 3
    • 4
  • Mark Harman
    • 1
  • Yue Jia
    • 1
  • Zheng Li
    • 2
  1. 1.CREST, Department of Computer ScienceUniversity College LondonLondonUK
  2. 2.Department of Computer ScienceBeijing University of Chemical TechnologyBeijingPeople’s Republic of China
  3. 3.Department of Computer ScienceBrunel University LondonLondonUK
  4. 4.Computer Engineering DepartmentBogazici UniversityBebek, IstanbulTurkey

Personalised recommendations