Abstract
Automation of test data generation is of prime importance in software testing because of the high cost and time incurred in manual testing. This paper proposes an Improved Crow Search Algorithm (ICSA) to automate the generation of test suites using the concept of mutation testing by simulating the intelligent behaviour of crows and Cauchy distribution. The Crow Search Algorithm suffers from the problem of search solutions getting trapped into the local search. The ICSA attempts to enhance the exploration capabilities of the metaheuristic algorithm by utilizing the concept of Cauchy random number. The concept of Mutation Sensitivity Testing has been used for defining the fitness function for the search based approach. The fitness function used, aids in finding optimal test suite which can achieve high detection score for the Program Under Test. The empirical evaluation of the proposed approach with other popular meta-heuristics, prove the effectiveness of ICSA for test suite generation using the concepts of mutation testing.
Similar content being viewed by others
References
Jamil MA, Arif M, Abubakar NSA, Ahmad A (2016) Software testing techniques: a literature review. In: 2016 6th international conference on information and communication technology for the Muslim world (ICT4M), pp 177–182
Kasurinen J (2010) Elaborating software test processes and strategies. In: 2010 third international conference on software testing, verification and validation, IEEE, pp 355–358
Jovanović I (2006) Software testing methods and techniques. In: The IPSI BgD transactions on internet research, p 30
Chen TY, Lau MF (1998) A new heuristic for test suite reduction. Inf Softw Technol 40(5–6):347–354
Jia Y, Harman M (2010) An analysis and survey of the development of mutation testing. IEEE Trans Softw Eng 37(5):649–678
Rani S, Dhawan H, Nagpal G, Suri B (2019) Implementing time-bounded automatic test data generation approach based on search-based mutation testing. In: Progress in advanced computing and intelligent engineering. Springer, Singapore, pp 113–122
Papadakis M, Kintis M, Zhang J, Jia Y, Le Traon Y, Harman M (2019) Mutation testing advances: an analysis and survey. Adv Comput Elsevier 112:275–378
Fraser G, Arcuri A (2015) Achieving scalable mutation-based generation of whole test suites. Empir Softw Eng 20(3):783–812
Offutt AJ, Lee A, Rothermel G, Untch RH, Zapf C (1996) An experimental determination of sufficient mutant operators. ACM Trans Softw Eng Methodol TOSEM 5(2):99–118
King KN, Offutt AJ (1991) A fortran language system for mutation-based software testing. Softw Pract Exp 21(7):685–718
Kintis M, Papadakis M, Papadopoulos A, Valvis E, Malevris N (2016) Analysing and comparing the effectiveness of mutation testing tools: a manual study. In: 2016 IEEE 16th international working conference on source code analysis and manipulation (SCAM), IEEE, pp 147–156
Rani S, Suri B (2019) Adopting social group optimization algorithm using mutation testing for test suite generation: SGO-MT. In: International conference on computational science and its applications. Springer, Cham, pp 520–528
Bashir MB, Nadeem A (2018) An experimental tool for search-based mutation testing. In: 2018 International conference on frontiers of information technology (FIT), IEEE, pp 30–34
Klammer C, Ramler R (2017) Journey from manual testing to automated test generation in an industry project. In: 2017 IEEE international conference on software quality, reliability and security companion (QRS-C), IEEE, pp 591–592
Ali S, Briand LC, Hemmati H, Panesar-Walawege RK (2009) A systematic review of the application and empirical investigation of search-based test case generation. IEEE Trans Softw Eng 36(6):742–762
Harman M, Jia Y, Zhang Y (2015) Achievements, open problems and challenges for search based software testing. In: 2015 IEEE 8th international conference on software testing, verification and validation (ICST), IEEE, pp 1–12
McMinn P (2011) Search-based software testing: past, present and future. In: 2011 IEEE fourth international conference on software testing, verification and validation workshops, IEEE, pp 153–163
Jatana N, Suri B, and Rani S (2017) Systematic literature review on search based mutation testing. e Inf Softw Eng J 11(1)
Hook D (2009) Using code mutation to study code faults in scientific software (Doctoral dissertation)
Wang GG, Deb S, Gandomi AH, Alavi AH (2016) Opposition-based krill herd algorithm with Cauchy mutation and position clamping. Neurocomputing 177:147–157
Baudry B, Fleurey F, Jézéquel JM, Le Traon Y (2002) Automatic test case optimization using a bacteriological adaptation model: application to.net components. In: Proceedings 17th IEEE international conference on automated software engineering, IEEE, pp 253–256
Srivatsava PR, Mallikarjun B, Yang XS (2013) Optimal test sequence generation using firefly algorithm. Swarm Evolut Comput 8:44–53
Lam SSB, Raju MHP, Ch S, Srivastav PR (2012) Automated generation of independent paths and test suite optimization using artificial bee colony. Proc Eng 30:191–200
Ayari K, Bouktif S, Antoniol G (2007) Automatic mutation test input data generation via ant colony. In: Proceedings of the 9th annual conference on genetic and evolutionary computation, pp 1074–1081
Srivastava PR, Baby K (2010) Automated software testing using metahurestic technique based on an ant colony optimization. In 2010 international symposium on electronic system design, IEEE, pp 235–240
Fraser G, Zeller A (2011) Mutation-driven generation of unit tests and oracles. IEEE Trans Softw Eng 38(2):278–292
Bueno PM, Jino M, Wong WE (2014) Diversity oriented test data generation using metaheuristic search techniques. Inf Sci 259:490–509
Binh NT, Tung KT (2015) A novel test data generation approach based upon mutation testing by using artificial immune system for Simulink models. In: Knowledge and systems engineering. Springer, Cham, pp 169–181
Bottaci L (2001) A genetic algorithm fitness function for mutation testing. In: Proceedings of the SEMINALL-workshop at the 23rd international conference on software engineering, Toronto, Canada
Shulga TE, Ivanov EA, Slastihina MD, Vagarina NS (2016) Developing a software system for automata-based code generation. Program Comput Softw 42(3):167–173
Zhang J, Zhang L, Harman M, Hao D, Jia Y, Zhang L (2018) Predictive mutation testing. IEEE Trans Softw Eng 45:898–918
Panichella A, Kifetew FM, Tonella P (2017) Automated test case generation as a many-objective optimisation problem with dynamic selection of the targets. IEEE Trans Softw Eng 44(2):122–158
Kelly D, Gray R, Shao Y (2011) Examining random and designed tests to detect code mistakes in scientific software. J Comput Sci 2(1):47–56
Durelli VH, Durelli RS, Borges SS, Endo AT, Eler MM, Dias DR, Guimarães MP (2019) Machine learning applied to software testing: a systematic mapping study. IEEE Trans Reliab 68:1189–1212
Askarzadeh A (2016) A novel metaheuristic method for solving constrained engineering optimization problems: crow search algorithm. Comput Struct 169:1–12
Jain M, Rani A, Singh V (2017) An improved Crow search algorithm for high-dimensional problems. J Intell Fuzzy Syst 33:3597–3614
Sayed A, Ahmad (2019) Feature selection via a novel chaotic crow search algorithm. Neural Comput Appl 31:171–188
Antono A, Santosa B, Siswanto N (2018) A meta-heuristic method for solving scheduling problem: crow search algorithm. In: IOP conference series: materials science and engineering, vol 337. no. 1. IOP Publishing
Allahverdipoor A, Gharehchopogh FS (2018) An improved K-nearest neighbor with crow search algorithm for feature selection in text documents classification. J Adv Comput Res 9(2):37–48
Hassanien AE, Rizk-Allah RM, Elhoseny M (2018) A hybrid crow search algorithm based on a rough searching scheme for solving engineering optimization problems. J Ambient Intell Humaniz Comput 1–25
Matlab programs. https://in.mathworks.com/matlabcentral/fileexchange/. Accessed 6 Sept 2016
Mishra DB, Mishra R, Acharya AA, Das KN (2019) Test data generation for mutation testing using genetic algorithm. Springer, Singapore, pp 857–867
Trelea IC (2003) The particle swarm optimization algorithm: convergence analysis and parameter selection. Inf Process Lett 85(6):317–325
Jatana N, Suri B (2019) Particle swarm and genetic algorithm applied to mutation testing for test data generation: a comparative evaluation. J King Saud Univ Comput Inf Sci
Dorigo M, Stützle T (2003) The ant colony optimization metaheuristic: algorithms, applications, and advances. In: Handbook of metaheuristics. Springer, Boston, pp 250–285
Loubière P, Jourdan A, Siarry P, Chelouah R (2016) A sensitivity analysis method for driving the Artificial Bee Colony algorithm’s search process. Appl Soft Comput 41:515–531
Woolson RF (2007) Wilcoxon signed-rank test. Wiley Encycl Clin Trials 9:1–3
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Function: Area of a triangle; Let the Original Program = bh/2; and Mutated Programme = (b + h)/2
Step 1: Initialize the test cases and memory.
Let the initial test cases be T = \( \left[ \begin{aligned} 2\,\,\,\,3 \hfill \\ 4\,\,\,\,6 \hfill \\ 8\,\,\,\,\,9 \hfill \\ \end{aligned} \right] \); let the memory be randomly allocated as M = \( \left[ \begin{aligned} 9\,\,\,6 \hfill \\ 2\,\,\,\,8 \hfill \\ 4\,\,\,\,\,3 \hfill \\ \end{aligned} \right] \)
Step 2: Evaluate the fitness function
Step 3: Based on the principle of CSA the new position of test case is generated from memory and original position according to the following formula.
New position of each test case = original position +0.5*flight length*|(memory-original position)|.
Now flight length is adjusted such that fl is less than 1. Let fl = 0.5
Step 4: The fitness function of new position is evaluated
It is found that there is no much improvement in the relative error when (3) is compared to (1). This shows that when flight length is less than one, search solutions are restricted to local area. When the flight length is greater than one, search will be a purely global one.
Step 5: When the Cauchy operator is applied to the initial position the following result occurs. The Cauchy random number for the each of the test cases is obtained from the inverse of the Cauchy distribution equation: equation \( F(x) = 1/2 + 1/\pi \arctan (x/t) \), t > 0
Step 6: Replace the value of flight length with the Cauchy random number and calculate the new position and fitness function.
The new position we obtained after applying Cauchy random number is given as
The fitness function is calculated for the new positions (5) is given as
The test cases obtained after using the Cauchy random number shows that the search yields better results. The fitness function of the new position is found to be better than the memorized position. The matrix shown by Eq. (6) is found to be better than that shown in (3). The test cases which generated the better fitness function is considered to be optimal one. The process is carried out till the highest relative error is obtained or till the termination criteria is met.
Rights and permissions
About this article
Cite this article
Jatana, N., Suri, B. An Improved Crow Search Algorithm for Test Data Generation Using Search-Based Mutation Testing. Neural Process Lett 52, 767–784 (2020). https://doi.org/10.1007/s11063-020-10288-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-020-10288-7