An Improved Crow Search Algorithm for Test Data Generation Using Search-Based Mutation Testing

Abstract

Automation of test data generation is of prime importance in software testing because of the high cost and time incurred in manual testing. This paper proposes an Improved Crow Search Algorithm (ICSA) to automate the generation of test suites using the concept of mutation testing by simulating the intelligent behaviour of crows and Cauchy distribution. The Crow Search Algorithm suffers from the problem of search solutions getting trapped into the local search. The ICSA attempts to enhance the exploration capabilities of the metaheuristic algorithm by utilizing the concept of Cauchy random number. The concept of Mutation Sensitivity Testing has been used for defining the fitness function for the search based approach. The fitness function used, aids in finding optimal test suite which can achieve high detection score for the Program Under Test. The empirical evaluation of the proposed approach with other popular meta-heuristics, prove the effectiveness of ICSA for test suite generation using the concepts of mutation testing.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

References

  1. 1.

    Jamil MA, Arif M, Abubakar NSA, Ahmad A (2016) Software testing techniques: a literature review. In: 2016 6th international conference on information and communication technology for the Muslim world (ICT4M), pp 177–182

  2. 2.

    Kasurinen J (2010) Elaborating software test processes and strategies. In: 2010 third international conference on software testing, verification and validation, IEEE, pp 355–358

  3. 3.

    Jovanović I (2006) Software testing methods and techniques. In: The IPSI BgD transactions on internet research, p 30

  4. 4.

    Chen TY, Lau MF (1998) A new heuristic for test suite reduction. Inf Softw Technol 40(5–6):347–354

    Article  Google Scholar 

  5. 5.

    Jia Y, Harman M (2010) An analysis and survey of the development of mutation testing. IEEE Trans Softw Eng 37(5):649–678

    Article  Google Scholar 

  6. 6.

    Rani S, Dhawan H, Nagpal G, Suri B (2019) Implementing time-bounded automatic test data generation approach based on search-based mutation testing. In: Progress in advanced computing and intelligent engineering. Springer, Singapore, pp 113–122

  7. 7.

    Papadakis M, Kintis M, Zhang J, Jia Y, Le Traon Y, Harman M (2019) Mutation testing advances: an analysis and survey. Adv Comput Elsevier 112:275–378

    Article  Google Scholar 

  8. 8.

    Fraser G, Arcuri A (2015) Achieving scalable mutation-based generation of whole test suites. Empir Softw Eng 20(3):783–812

    Article  Google Scholar 

  9. 9.

    Offutt AJ, Lee A, Rothermel G, Untch RH, Zapf C (1996) An experimental determination of sufficient mutant operators. ACM Trans Softw Eng Methodol TOSEM 5(2):99–118

    Article  Google Scholar 

  10. 10.

    King KN, Offutt AJ (1991) A fortran language system for mutation-based software testing. Softw Pract Exp 21(7):685–718

    Article  Google Scholar 

  11. 11.

    Kintis M, Papadakis M, Papadopoulos A, Valvis E, Malevris N (2016) Analysing and comparing the effectiveness of mutation testing tools: a manual study. In: 2016 IEEE 16th international working conference on source code analysis and manipulation (SCAM), IEEE, pp 147–156

  12. 12.

    Rani S, Suri B (2019) Adopting social group optimization algorithm using mutation testing for test suite generation: SGO-MT. In: International conference on computational science and its applications. Springer, Cham, pp 520–528

  13. 13.

    Bashir MB, Nadeem A (2018) An experimental tool for search-based mutation testing. In: 2018 International conference on frontiers of information technology (FIT), IEEE, pp 30–34

  14. 14.

    Klammer C, Ramler R (2017) Journey from manual testing to automated test generation in an industry project. In: 2017 IEEE international conference on software quality, reliability and security companion (QRS-C), IEEE, pp 591–592

  15. 15.

    Ali S, Briand LC, Hemmati H, Panesar-Walawege RK (2009) A systematic review of the application and empirical investigation of search-based test case generation. IEEE Trans Softw Eng 36(6):742–762

    Article  Google Scholar 

  16. 16.

    Harman M, Jia Y, Zhang Y (2015) Achievements, open problems and challenges for search based software testing. In: 2015 IEEE 8th international conference on software testing, verification and validation (ICST), IEEE, pp 1–12

  17. 17.

    McMinn P (2011) Search-based software testing: past, present and future. In: 2011 IEEE fourth international conference on software testing, verification and validation workshops, IEEE, pp 153–163

  18. 18.

    Jatana N, Suri B, and Rani S (2017) Systematic literature review on search based mutation testing. e Inf Softw Eng J 11(1)

  19. 19.

    Hook D (2009) Using code mutation to study code faults in scientific software (Doctoral dissertation)

  20. 20.

    Wang GG, Deb S, Gandomi AH, Alavi AH (2016) Opposition-based krill herd algorithm with Cauchy mutation and position clamping. Neurocomputing 177:147–157

    Article  Google Scholar 

  21. 21.

    Baudry B, Fleurey F, Jézéquel JM, Le Traon Y (2002) Automatic test case optimization using a bacteriological adaptation model: application to.net components. In: Proceedings 17th IEEE international conference on automated software engineering, IEEE, pp 253–256

  22. 22.

    Srivatsava PR, Mallikarjun B, Yang XS (2013) Optimal test sequence generation using firefly algorithm. Swarm Evolut Comput 8:44–53

    Article  Google Scholar 

  23. 23.

    Lam SSB, Raju MHP, Ch S, Srivastav PR (2012) Automated generation of independent paths and test suite optimization using artificial bee colony. Proc Eng 30:191–200

    Article  Google Scholar 

  24. 24.

    Ayari K, Bouktif S, Antoniol G (2007) Automatic mutation test input data generation via ant colony. In: Proceedings of the 9th annual conference on genetic and evolutionary computation, pp 1074–1081

  25. 25.

    Srivastava PR, Baby K (2010) Automated software testing using metahurestic technique based on an ant colony optimization. In 2010 international symposium on electronic system design, IEEE, pp 235–240

  26. 26.

    Fraser G, Zeller A (2011) Mutation-driven generation of unit tests and oracles. IEEE Trans Softw Eng 38(2):278–292

    Article  Google Scholar 

  27. 27.

    Bueno PM, Jino M, Wong WE (2014) Diversity oriented test data generation using metaheuristic search techniques. Inf Sci 259:490–509

    Article  Google Scholar 

  28. 28.

    Binh NT, Tung KT (2015) A novel test data generation approach based upon mutation testing by using artificial immune system for Simulink models. In: Knowledge and systems engineering. Springer, Cham, pp 169–181

  29. 29.

    Bottaci L (2001) A genetic algorithm fitness function for mutation testing. In: Proceedings of the SEMINALL-workshop at the 23rd international conference on software engineering, Toronto, Canada

  30. 30.

    Shulga TE, Ivanov EA, Slastihina MD, Vagarina NS (2016) Developing a software system for automata-based code generation. Program Comput Softw 42(3):167–173

    Article  Google Scholar 

  31. 31.

    Zhang J, Zhang L, Harman M, Hao D, Jia Y, Zhang L (2018) Predictive mutation testing. IEEE Trans Softw Eng 45:898–918

    Article  Google Scholar 

  32. 32.

    Panichella A, Kifetew FM, Tonella P (2017) Automated test case generation as a many-objective optimisation problem with dynamic selection of the targets. IEEE Trans Softw Eng 44(2):122–158

    Article  Google Scholar 

  33. 33.

    Kelly D, Gray R, Shao Y (2011) Examining random and designed tests to detect code mistakes in scientific software. J Comput Sci 2(1):47–56

    Article  Google Scholar 

  34. 34.

    Durelli VH, Durelli RS, Borges SS, Endo AT, Eler MM, Dias DR, Guimarães MP (2019) Machine learning applied to software testing: a systematic mapping study. IEEE Trans Reliab 68:1189–1212

    Article  Google Scholar 

  35. 35.

    Askarzadeh A (2016) A novel metaheuristic method for solving constrained engineering optimization problems: crow search algorithm. Comput Struct 169:1–12

    Article  Google Scholar 

  36. 36.

    Jain M, Rani A, Singh V (2017) An improved Crow search algorithm for high-dimensional problems. J Intell Fuzzy Syst 33:3597–3614

    Article  Google Scholar 

  37. 37.

    Sayed A, Ahmad (2019) Feature selection via a novel chaotic crow search algorithm. Neural Comput Appl 31:171–188

    Article  Google Scholar 

  38. 38.

    Antono A, Santosa B, Siswanto N (2018) A meta-heuristic method for solving scheduling problem: crow search algorithm. In: IOP conference series: materials science and engineering, vol 337. no. 1. IOP Publishing

  39. 39.

    Allahverdipoor A, Gharehchopogh FS (2018) An improved K-nearest neighbor with crow search algorithm for feature selection in text documents classification. J Adv Comput Res 9(2):37–48

    Google Scholar 

  40. 40.

    Hassanien AE, Rizk-Allah RM, Elhoseny M (2018) A hybrid crow search algorithm based on a rough searching scheme for solving engineering optimization problems. J Ambient Intell Humaniz Comput 1–25

  41. 41.

    Matlab programs. https://in.mathworks.com/matlabcentral/fileexchange/. Accessed 6 Sept 2016

  42. 42.

    Mishra DB, Mishra R, Acharya AA, Das KN (2019) Test data generation for mutation testing using genetic algorithm. Springer, Singapore, pp 857–867

    Google Scholar 

  43. 43.

    Trelea IC (2003) The particle swarm optimization algorithm: convergence analysis and parameter selection. Inf Process Lett 85(6):317–325

    MathSciNet  Article  Google Scholar 

  44. 44.

    Jatana N, Suri B (2019) Particle swarm and genetic algorithm applied to mutation testing for test data generation: a comparative evaluation. J King Saud Univ Comput Inf Sci

  45. 45.

    Dorigo M, Stützle T (2003) The ant colony optimization metaheuristic: algorithms, applications, and advances. In: Handbook of metaheuristics. Springer, Boston, pp 250–285

  46. 46.

    Loubière P, Jourdan A, Siarry P, Chelouah R (2016) A sensitivity analysis method for driving the Artificial Bee Colony algorithm’s search process. Appl Soft Comput 41:515–531

    Article  Google Scholar 

  47. 47.

    Woolson RF (2007) Wilcoxon signed-rank test. Wiley Encycl Clin Trials 9:1–3

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Nishtha Jatana.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Function: Area of a triangle; Let the Original Program = bh/2; and Mutated Programme = (b + h)/2

Step 1: Initialize the test cases and memory.

Let the initial test cases be T = \( \left[ \begin{aligned} 2\,\,\,\,3 \hfill \\ 4\,\,\,\,6 \hfill \\ 8\,\,\,\,\,9 \hfill \\ \end{aligned} \right] \); let the memory be randomly allocated as M = \( \left[ \begin{aligned} 9\,\,\,6 \hfill \\ 2\,\,\,\,8 \hfill \\ 4\,\,\,\,\,3 \hfill \\ \end{aligned} \right] \)

Step 2: Evaluate the fitness function

$$ {\text{Relative}}\;{\text{error}} = \left[ \begin{aligned} 0. 7 2 2\hfill \\ 0. 3 7 5\, \hfill \\ 0. 4 1 6\hfill \\ \end{aligned} \right] $$
(11)

Step 3: Based on the principle of CSA the new position of test case is generated from memory and original position according to the following formula.

New position of each test case = original position +0.5*flight length*|(memory-original position)|.

Now flight length is adjusted such that fl is less than 1. Let fl = 0.5

$$ {\text{The}}\;{\text{new}}\;{\text{position}}\;{\text{is}}\;{\text{given}}\;{\text{as}} = \left[ \begin{aligned} 3\,\,\,\,\,3 \hfill \\ 4\,\,\,\,6 \hfill \\ 4\,\,\,\,2 \hfill \\ \end{aligned} \right] $$
(12)

Step 4: The fitness function of new position is evaluated

$$ {\text{Relative}}\;{\text{error}} = \left[ \begin{aligned} 0. 3 2 5\hfill \\ 0. 1 6 6\hfill \\ 0. 2 5 5\hfill \\ \end{aligned} \right] $$
(13)

It is found that there is no much improvement in the relative error when (3) is compared to (1). This shows that when flight length is less than one, search solutions are restricted to local area. When the flight length is greater than one, search will be a purely global one.

Step 5: When the Cauchy operator is applied to the initial position the following result occurs. The Cauchy random number for the each of the test cases is obtained from the inverse of the Cauchy distribution equation: equation \( F(x) = 1/2 + 1/\pi \arctan (x/t) \), t > 0

$$ {\text{The}}\;{\text{Cauchy}}\;{\text{random}}\;{\text{number}}\;{\text{for}}\;{\text{each}}\;{\text{of}}\;{\text{the}}\;{\text{test}}\;{\text{case}}\;{\text{is}}\;{\text{C}} = \left[ \begin{aligned} 12.82\,\,\,\,8.75 \hfill \\ 6.77\,\,\,\,\,\,4.88 \hfill \\ 4.00\,\,\,\,\,\,\,3.72 \hfill \\ \end{aligned} \right] $$
(14)

Step 6: Replace the value of flight length with the Cauchy random number and calculate the new position and fitness function.

The new position we obtained after applying Cauchy random number is given as

$$ \left[ \begin{aligned} 44\,\,\,\,15 \hfill \\ 2\,\,\,\,\,\,\,10 \hfill \\ 16\,\,\,\,\,\,3 \hfill \\ \end{aligned} \right] $$
(15)

The fitness function is calculated for the new positions (5) is given as

$$ {\text{Relative}}\;{\text{error}} = \left[ \begin{aligned} 0. 9 1 1\hfill \\ 0. 4\hfill \\ 0. 6 0 4\hfill \\ \end{aligned} \right] $$
(16)

The test cases obtained after using the Cauchy random number shows that the search yields better results. The fitness function of the new position is found to be better than the memorized position. The matrix shown by Eq. (6) is found to be better than that shown in (3). The test cases which generated the better fitness function is considered to be optimal one. The process is carried out till the highest relative error is obtained or till the termination criteria is met.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jatana, N., Suri, B. An Improved Crow Search Algorithm for Test Data Generation Using Search-Based Mutation Testing. Neural Process Lett (2020). https://doi.org/10.1007/s11063-020-10288-7

Download citation

Keywords

  • Improved Crow Search Algorithm
  • Cauchy random number
  • Mutation sensitivity testing
  • Mothra mutation operators