Abstract
Testing is a crucial part of the development of a new product. For software validation a transformation from manual to automated tests can be observed which enables companies to implement large numbers of test cases. However, during testing situations may occur where it is not feasible to run all tests due to time constraints. Hence a set of critical test cases must be compiled which usually fulfills several criteria. Within this work we focus on criteria that are feasible for black box testing such as system tests. We adapt an existing artificial immune system for our use case and evaluate our method in a series of experiments using industrial datasets. We compare our approach with several other test selection methods where our algorithm shows superior performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Available here: https://github.com/LagLukas/moa_testing.
- 2.
It is worth mentioning that for this statistical test no preconditions must be checked. Furthermore, the one-sided variant checks if the median of a random variable X is higher than the median of a random variable Y.
References
Arrieta, A., Wang, S., Arruabarrena, A., Markiegi, U., Sagardui, G., Etxeberria, L.: Multi-objective black-box test case selection for cost-effectively testing simulation models. In: Proceedings of the Genetic and Evolutionary Computation Conference, New York, NY, USA, pp. 1411–1418. GECCO 2018. Association for Computing Machinery (2018)
Azuaje, F.: Review of “Artificial Immune Systems: A New Computational Intelligence Approach” by L.N. de Castro and J. Timmis (Eds) Springer, London, 2002. Neural Net. 16(8), 1229 (2003)
Bath, G., McKay, J.: The Software Test Engineer’s Handbook: A Study Guide for the ISTQB Test Analyst and Technical Test Analyst Advanced Level Certificates 2012. Rocky Nook Computing, Rocky Nook (2014)
Dinur, I., Steurer, D.: Analytical approach to parallel repetition. In: Proceedings of the Forty-sixth Annual ACM Symposium on Theory of Computing, New York, NY, USA, pp. 624–633. STOC 2014. ACM (2014)
Duran, J.W., Ntafos, S.C.: an evaluation of random testing. IEEE Trans. Softw. Eng.SE 10(4), 438–444 (1984)
Dustin, E., Rashka, J., Paul, J.: Automated Software Testing: Introduction, Management, and Performance. Addison-Wesley Longman Publishing Co., Inc., New York (1999)
Gotlieb, A., Marijan, D.: FLOWER: optimal test suite reduction as a network maximum flow. In: Proceedings of the 2014 International Symposium on Software Testing and Analysis, New York, NY, USA, pp. 171–180. ISSTA 2014. ACM (2014)
Haga, H., Suehiro, A.: Automatic test case generation based on genetic algorithm and mutation analysis. In: 2012 IEEE International Conference on Control System, Computing and Engineering, pp. 119–123 (2012)
Jeffrey, D., Gupta, N.: Improving fault detection capability by selectively retaining test cases during test suite reduction. IEEE Trans. Softw. Eng. 33(2), 108–123 (2007)
Jia, Y., Harman, M.: Constructing subtle faults using higher order mutation testing. In: 2008 Eighth IEEE International Working Conference on Source Code Analysis and Manipulation, pp. 249–258 (2008)
Joshi, A.: The Germinal Centre Artificial Immune System. Ph.D. thesis, University of Birmingham (2017)
Joshi, A., Rowe, J.E., Zarges, C.: An immune-inspired algorithm for the set cover problem. In: Bartz-Beielstein, T., Branke, J., Filipič, B., Smith, J. (eds.) PPSN 2014. LNCS, vol. 8672, pp. 243–251. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10762-2_24
Joshi, A., Rowe, J.E., Zarges, C.: Improving the performance of the germinal center artificial immune system using epsilon-dominance: a multi-objective knapsack problem case study. In: Ochoa, G., Chicano, F. (eds.) EvoCOP 2015. LNCS, vol. 9026, pp. 114–125. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16468-7_10
Joshi, A., Rowe, J., Zarges, C.: On the effects of incorporating memory in GC-AIS for the set cover problem. In: MIC 2015: The XI Metaheuristics International Conference (2015)
Lachmann, R., Felderer, M., Nieke, M., Schulze, S., Seidl, C., Schaefer, I.: Multi-objective black-box test case selection for system testing. In: Proceedings of the Genetic and Evolutionary Computation Conference, New York, NY, USA, pp. 1311–1318. GECCO 2017. Association for Computing Machinery (2017)
Luo, W., Liu, R., Jiang, H., Zhao, D., Wu, L.: Three branches of negative representation of information: a survey. IEEE Trans. Emerg. Top. Comput. Intell. 2(6), 411–425 (2018)
Marijan, D., Gotlieb, A., Sen, S.: Test case prioritization for continuous regression testing: an industrial case study. In: 2013 IEEE International Conference on Software Maintenance, pp. 540–543 (2013)
McMaster, S., Memon, A.M.: Call stack coverage for test suite reduction. In: 21st IEEE International Conference on Software Maintenance (ICSM 2005), pp. 539–548 (2005)
Mondal, D., Hemmati, H., Durocher, S.: Exploring test suite diversification and code coverage in multi-objective test case selection. In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pp. 1–10 (2015)
Neumann, F., Witt, C.: Bioinspired Computation in Combinatorial Optimization: Algorithms and Their Computational Complexity. Natural Computing Series. Springer-Verlag, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16544-3. ISBN 978-3-642-16543-6
Nguyen, A., Le, B., Nguyen, V.: Prioritizing automated user interface tests using reinforcement learning. In: Proceedings of the Fifteenth International Conference on Predictive Models and Data Analytics in Software Engineering, New York, NY, USA, pp. 56–65. PROMISE 2019. Association for Computing Machinery (2019)
Note Narciso, E., Delamaro, M., Nunes, F.: Test case selection: a systematic literature review. Int. J. Softw. Eng. Knowl. Eng. 24, 653–676 (2014)
Rodrigues, D.S., Delamaro, M.E., Corrêa, C.G., Nunes, F.L.S.: Using genetic algorithms in test data generation: a critical systematic mapping. ACM Comput. Surv. 51(2) (2018)
Rosenbauer, L., Stein, A., Hähner, J.: An artificial immune system for adaptive test selection. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI) (2020)
Rosenbauer, L., Stein, A., Hähner, J.: A Germinal Centre Artificial Immune System for Software Test Suite Reduction. Artificial Life (2020)
de Souza, L.S., de Miranda, P.B.C., Prudencio, R.B.C., Barros, F.D.A.: A multi-objective particle swarm optimization for test case selection based on functional requirements coverage and execution effort. In: 2011 IEEE 23rd International Conference on Tools with Artificial Intelligence, pp. 245–252 (2011)
Spieker, H., Gotlieb, A., Marijan, D., Mossige, M.: Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration. CoRR abs/1811.04122 (2018)
Vaysburg, B., Tahat, L.H., Korel, B.: Dependence Analysis in Reduction of Requirement Based Test Suites. In: Proceedings of the 2002 ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 107–111. ISSTA 2002, Association for Computing Machinery, New York, NY, USA (2002)
Whalen, M.W., Rajan, A., Heimdahl, M.P., Miller, S.P.: Coverage metrics for requirements-based testing. In: Proceedings of the 2006 International Symposium on Software Testing and Analysis, New York, NY, USA, pp. 25–36. ISSTA 2006. Association for Computing Machinery (2006)
Xiao, L., Miao, H., Shi, T., Hong, Y.: LSTM-based deep learning for spatial-temporal software testing. Distrib. Parallel Databases 38(3), 687–712 (2020)
Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Softw. Test. Verif. Reliab. 22(2), 67–120 (2012)
Yu, Y., Yao, X., Zhou, Z.H.: On the approximation ability of evolutionary optimization with application to minimum set cover. Artif. Intell. S180–181 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Rosenbauer, L., Stein, A., Hähner, J. (2021). An Artificial Immune System for Black Box Test Case Selection. In: Zarges, C., Verel, S. (eds) Evolutionary Computation in Combinatorial Optimization. EvoCOP 2021. Lecture Notes in Computer Science(), vol 12692. Springer, Cham. https://doi.org/10.1007/978-3-030-72904-2_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-72904-2_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-72903-5
Online ISBN: 978-3-030-72904-2
eBook Packages: Computer ScienceComputer Science (R0)