Skip to main content
SpringerLink
Log in

Hypervolume-Based Search for Test Case Prioritization

  • Conference paper
  • First Online:
Search-Based Software Engineering (SSBSE 2015)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 9275))

Included in the following conference series:

Abstract

Test case prioritization (TCP) is aimed at finding an ideal ordering for executing the available test cases to reveal faults earlier. To solve this problem greedy algorithms and meta-heuristics have been widely investigated, but in most cases there is no statistically significant difference between them in terms of effectiveness. The fitness function used to guide meta-heuristics condenses the cumulative coverage scores achieved by a test case ordering using the Area Under Curve (AUC) metric. In this paper we notice that the AUC metric represents a simplified version of the hypervolume metric used in many objective optimization and we propose HGA, a Hypervolume-based Genetic Algorithm, to solve the TCP problem when using multiple test criteria. The results shows that HGA is more cost-effective than the additional greedy algorithm on large systems and on average requires 36 % of the execution time required by the additional greedy algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Since the additional greedy is a deterministic algorithm, the variance over 20 independent runs is zero. Conversely, because of the random inheritance of GAs, HGA does not reach a zero variance.

References

  1. Arcuri, A., Briand, L.C.: A practical guide for using statistical tests to assess randomized algorithms in software engineering. In: Proceedings of International Conference on Software Engineering (ICSE), pp. 1–10. ACM (2011)

    Google Scholar 

  2. Auger, A., Bader, J., Brockhoff, D., Zitzler, E.: Theory of the hypervolume indicator: optimal \(\mu \)-distributions and the choice of the reference point. In: Proceedings of SIGEVO workshop on Foundations of Genetic Algorithms (FOGA), pp. 87–102. ACM (2009)

    Google Scholar 

  3. Bryce, R.C., Colbourn, C.J., Cohen, M.B.: A framework of greedy methods for constructing interaction test suites. In: Proceedings International Conference on Software Engineering (ICSE), pp. 146–155 (2005)

    Google Scholar 

  4. Chen, T.Y., Lau, M.F.: Dividing strategies for the optimization of a test suite. Inf. Process. Lett. 60(3), 135–141 (1996)

    Article  MathSciNet  Google Scholar 

  5. Cohen, M., Dwyer, M., Shi, J.: Constructing interaction test suites for highly-configurable systems in the presence of constraints: A greedy approach. IEEE Trans. Softw. Eng. 34, 633–650 (2008)

    Article  Google Scholar 

  6. Conover, W.J.: Practical Nonparametric Statistics, 3rd edn. Wiley, New York (1998)

    Google Scholar 

  7. De Lucia, A., Di Penta, M., Oliveto, R., Panichella, A.: On the role of diversity measures for multi-objective test case selection. In: Proceedings of International Workshop on Automation of Software Test (AST), pp. 145–151 (2012)

    Google Scholar 

  8. Do, H., Rothermel, G., Kinneer, A.: Empirical studies of test case prioritization in a junit testing environment. In: 15th International Symposium on Software Reliability Engineering, pp. 113–124. IEEE Computer Society (2004)

    Google Scholar 

  9. Durillo, J.J., Nebro, A.J.: jMetal: a java framework for multi-objective optimization. Adv. Eng. Softw. 42, 760–771 (2011)

    Article  Google Scholar 

  10. Elbaum, S., Malishevsky, A., Rothermel, G.: Incorporating varying test costs and fault severities into test case prioritization. In: Proceedings of International Conference on Software Engineering (ICSE), pp. 329–338. IEEE (2001)

    Google Scholar 

  11. Elbaum, S., Malishevsky, A.G., Rothermel, G.: Prioritizing test cases for regression testing. In: Proceedings of International Symposium on Software Testing and Analysis (ISSTA), pp. 102–112. ACM (2000)

    Google Scholar 

  12. Elbaum, S., Malishevsky, A.G., Rothermel, G.: Prioritizing test cases for regression testing. Softw. Eng. Notes 25, 102–112 (2000)

    Article  Google Scholar 

  13. Elbaum, S., Malishevsky, A.G., Rothermel, G.: Test case prioritization: a family of empirical studies. IEEE Trans. Softw. Eng. 28(2), 159–182 (2002)

    Article  Google Scholar 

  14. Elbaum, S., Rothermel, G., Kanduri, S., Malishevsky, A.: Selecting a cost-effective test case prioritization technique. Softw. Qual. J. 12(3), 185–210 (2004)

    Article  Google Scholar 

  15. Hao, D., Zhang, L., Zhang, L., Rothermel, G., Mei, H.: A unified test case prioritization approach. ACM Trans. Softw. Eng. Methodol. 24(2), 10:1–10:31 (2014)

    Article  Google Scholar 

  16. Harrold, M.J., Gupta, R., Soffa, M.L.: A methodology for controlling the size of a test suite. ACM Trans. Softw. Eng. Methodol. 2, 270–285 (1993)

    Article  Google Scholar 

  17. Huang, Y.C., Huang, C.Y., Chang, J.R., Chen, T.Y.: Design and analysis of cost-cognizant test case prioritization using genetic algorithm with test history. In: Proceedings of Annual Computer Software and Applications Conference (COMPSAC), pp. 413–418. IEEE (2010)

    Google Scholar 

  18. Hughes, E.: Evolutionary many-objective optimisation: many once or one many? IEEE Congr. Evol. Comput. 1, 222–227 (2005)

    Google Scholar 

  19. Hyunsook Do, S.G.E., Rothermel, G.: Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empirical Softw. Eng.: Int. J. 10, 405–435 (2005)

    Article  Google Scholar 

  20. Islam, M., Marchetto, A., Susi, A., Scanniello, G.: A multi-objective technique to prioritize test cases based on latent semantic indexing. In: Proceedings of European Conf. on Software Maintenance and Reengineering (CSMR), pp. 21–30. IEEE (2012)

    Google Scholar 

  21. Li, Z., Bian, Y., Zhao, R., Cheng, J.: A fine-grained parallel multi-objective test case prioritization on GPU. In: Ruhe, G., Zhang, Y. (eds.) SSBSE 2013. LNCS, vol. 8084, pp. 111–125. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  22. Li, Z., Harman, M., Hierons, R.M.: Search algorithms for regression test case prioritization. IEEE Trans. Softw. Eng. 33(4), 225–237 (2007)

    Article  Google Scholar 

  23. Malishevsky, A.G., Ruthruff, J.R., Rothermel, G., Elbaum, S.: Cost-cognizant test case prioritization. Technical report, Department of Computer Science and Engineering (2006)

    Google Scholar 

  24. Panichella, A., Oliveto, R., Di Penta, M., De Lucia, A.: Improving multi-objective test case selection by injecting diversity in genetic algorithms. IEEE Trans. Softw. Eng. 41(4), 358–383 (2015)

    Article  Google Scholar 

  25. Rothermel, G., Untch, R., Chu, C., Harrold, M.: Prioritizing test cases for regression testing. IEEE Trans. Softw. Eng. 27(10), 929–948 (2001)

    Article  Google Scholar 

  26. Rothermel, G., Harrold, M.J., von Ronne, J., Hong, C.: Empirical studies of test-suite reduction. Softw. Test. Verif. Reliab. 12, 219–249 (2002)

    Article  Google Scholar 

  27. Srikanth, H., Williams, L., Osborne, J.: System test case prioritization of new and regression test cases. In: International Symposium on Empirical Software Engineering (2005)

    Google Scholar 

  28. Vargha, A., Delaney, H.D.: A critique and improvement of the cl common language effect size statistics of mcgraw and wong. J. Educ. Behav. Stat. 25(2), 101–132 (2000)

    Google Scholar 

  29. Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Softw. Test. Verif. Reliab. 22(2), 67–120 (2012)

    Article  Google Scholar 

  30. Yoo, S., Harman, M.: Pareto efficient multi-objective test case selection. In: Proceedings of International Symposium on Software Testing and Analysis (ISSTA), pp. 140–150. ACM (2007)

    Google Scholar 

  31. Yoo, S., Harman, M.: Using hybrid algorithm for Pareto efficient multi-objective test suite minimisation. J. Syst. Softw. 83(4), 689–701 (2010)

    Article  Google Scholar 

  32. Zhang, L., Hao, D., Zhang, L., Rothermel, G., Mei, H.: Bridging the gap between the total and additional test-case prioritization strategies. In: Proceedings of International Conference on Software Engineering (ICSE), pp. 192–201. IEEE (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Salerno, Salerno, Italy

    Dario Di Nucci & Andrea De Lucia

  2. Delft University of Technology, Delft, The Netherlands

    Annibale Panichella & Andy Zaidman

Authors
  1. Dario Di Nucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Annibale Panichella
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andy Zaidman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrea De Lucia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dario Di Nucci .

Editor information

Editors and Affiliations

  1. Federal University of Rio de Janeiro State, Rio de Janeiro, Brazil

    Márcio Barros

  2. Carleton University, Ottawa, Ontario, Canada

    Yvan Labiche

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Di Nucci, D., Panichella, A., Zaidman, A., De Lucia, A. (2015). Hypervolume-Based Search for Test Case Prioritization. In: Barros, M., Labiche, Y. (eds) Search-Based Software Engineering. SSBSE 2015. Lecture Notes in Computer Science(), vol 9275. Springer, Cham. https://doi.org/10.1007/978-3-319-22183-0_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-22183-0_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-22182-3

  • Online ISBN: 978-3-319-22183-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation