Implementing Time-Bounded Automatic Test Data Generation Approach Based on Search-Based Mutation Testing

  • Shweta RaniEmail author
  • Hrithik Dhawan
  • Gagandeep Nagpal
  • Bharti Suri
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 714)


Automatic test generation is a backbreaking task in software testing, and it is the need of the research community as well as for industry. Search-based mutation testing has been effectively applied for solving the testing problems. In this paper, an idea following the behavior of genetic algorithm with the benefits of mutation testing is proposed and implemented to generate the test cases automatically. For the sake of minimizing the cost incurred due to mutation testing, selective mutation technique is encouraged to generate the lesser number of mutants using delete mutation operators instead of all the traditional mutation operators. The process stops when it reaches the predefined time limit. In each iteration, it tries to optimize the size of the test suite by searching and eliminating the redundant less fit test inputs with the aim of mutation coverage. Results suggest that the generated test cases successfully detect more than 90% mutants.


Search-based mutation testing Mutation testing Genetic algorithm Automatic test data generation 



The authors would like to acknowledge Ministry of Electronics and Information Technology, Govt. of India for supporting this research under Visvesvaraya Ph.D. Scheme for Electronics and IT.


  1. 1.
    D. C. Ince, “The automatic generation of test data,” The Computer Journal, vol. 30, no. 1, p. 63, 1987.Google Scholar
  2. 2.
    M. Harman and P. McMinn, “A theoretical & empirical analysis of evolutionary testing and hill climbing for structural test data generation,” in Proceedings of the 2007 International Symposium on Software Testing and Analysis, ser. ISSTA ’07. ACM, 2007, pp. 73–83.Google Scholar
  3. 3.
    J. Wegener, A. Baresel, and H. Sthamer, “Evolutionary test environment for automatic structural testing,” Information and Software Technology, vol. 43, no. 14, pp. 841–854, 2001.Google Scholar
  4. 4.
    J. Clarke, J. J. Dolado, M. Harman, R. Hierons, B. Jones, M. Lumkin, B. Mitchell, S. Mancoridis, K. Rees, M. Roper, and M. Shepperd, “Reformulating software engineering as a search problem,” IEEE Proceedings - Software, vol. 150, no. 3, pp. 161–175, 2003.Google Scholar
  5. 5.
    M. Dave and R. Agrawal, “Search based techniques and mutation analysis in automatic test case generation: A survey,” in 2015 IEEE International Advance Computing Conference (IACC), 2015, pp. 795–799.Google Scholar
  6. 6.
    P. McMinn, “Search-based software test data generation: A survey: Research articles,” Software Testing, Verification and Reliability, vol. 14, no. 2, pp. 105–156, 2004.Google Scholar
  7. 7.
    P. McMinn, “Search-based software testing: Past, present and future,” in Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, ser. ICSTW ’11. IEEE Computer Society, 2011, pp. 153–163.Google Scholar
  8. 8.
    O. Sahin and B. Akay, “Comparisons of metaheuristic algorithms and fitness functions on software test data generation,” Applied Soft Computing, vol. 49, pp. 1202–1214, 2016.Google Scholar
  9. 9.
    W. Miller and D. L. Spooner, “Automatic generation of floating-point test data,” IEEE Transactions on Software Engineering, vol. SE-2, no. 3, pp. 223–226, 1976.Google Scholar
  10. 10.
    Z. Li, M. Harman, and R. M. Hierons, “Search algorithms for regression test case prioritization,” IEEE Transaction on Software Engineering, vol. 33, no. 4, pp. 225–237, 2007.Google Scholar
  11. 11.
    J. H. Holland, Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. Cambridge, MA, USA: MIT Press, 1992.Google Scholar
  12. 12.
    S. N. Sivanandan and S. N. Deepa, Intorduction to Genetic Algorithms. Springer-Verleg, 2008.Google Scholar
  13. 13.
    S. Luke, Essentials of Metaheuristics. Lulu, 2009, available for free at
  14. 14.
    M. Delamaro, M. L. Chaim, A. M. R. Vincenzi, M. Jino, and J. C. Maldonado, “Twenty-five years of research in structural and mutation testing,” in 2011 25th Brazilian Symposium on Software Engineering, Sept 2011, pp. 40–49.Google Scholar
  15. 15.
    Y. Jia and M. Harman, “An analysis and survey of the development of mutation testing,” IEEE Transaction on Software Engineering, vol. 37, no. 5, pp. 649–678, 2011.Google Scholar
  16. 16.
    R. A. DeMillo, R. J. Lipton, and F. G. Sayward, “Hints on test data selection: Help for the practicing programmer,” Computer, vol. 11, no. 4, pp. 34–41, 1978.Google Scholar
  17. 17.
    R. G. Hamlet, “Testing programs with the aid of a compiler,” IEEE Transactions on Software Engineering, vol. SE-3, pp. 279–290, 1977.Google Scholar
  18. 18.
    R. H. Untch, “On reduced neighborhood mutation analysis using a single mutagenic operator,” in Proceedings of the 47th Annual Southeast Regional Conference. ACM, 2009, pp. 71:1–71:4.Google Scholar
  19. 19.
    J. H. Andrews, L. C. Briand, and Y. Labiche, “Is mutation an appropriate tool for testing experiments?” in Proceedings of the 27th International Conference on Software Engineering, ser. ICSE ’05. ACM, 2005, pp. 402–411.Google Scholar
  20. 20.
    S. Ali, L. C. Briand, H. Hemmati, and R. K. Panesar-Walawege, “A systematic review of the application and empirical investigation of search-based test case generation,” IEEE Transactions on Software Engineering, vol. 36, no. 6, pp. 742–762, 2010.Google Scholar
  21. 21.
    R. A. Silva, S. d. R. S. d. Souza, and P. S. L. d. Souza, “A systematic review on search based mutation testing,” Information and Software Technology, vol. 81, pp. 19 – 35, 2017.Google Scholar
  22. 22.
    N. Jatana, B. Suri, and S. Rani, “Systematic literature review on search based mutation testing,” e-Informatica Software Engineering Journal, vol. 11, no. 1, pp. 61–78, 2017.Google Scholar
  23. 23.
    N. Jatana, S. Rani, and B. Suri, “State of art in the field of search-based mutation testing,” in 2015 4th International Conference on Reliability, Infocom Technologies and Optimization (ICRITO) (Trends and Future Directions), 2015, pp. 1–6.Google Scholar
  24. 24.
    Y.-S. Ma, J. Offutt, and Y. R. Kwon, “Mujava: An automated class mutation system,” Software Testing, Verification and Reliability, vol. 15, no. 2, pp. 97–133, 2005.Google Scholar
  25. 25.
    B. F. Jones, D. E. Eyres, and H.-H. Sthamer, “A strategy for using genetic algorithms to automate branch and fault-based testing,” The Computer Journal, vol. 41, no. 2, p. 98, 1998.Google Scholar
  26. 26.
    L. Bottaci, “A genetic algorithm fitness function for mutation testing,” in Proceedings of the SEMINALL-workshop at the 23rd international conference on software engineering, Toronto, Canada, 2001.Google Scholar
  27. 27.
    M. Masud, A. Nayak, M. Zaman, and N. Bansal, “Strategy for mutation testing using genetic algorithms,” in Canadian Conference on Electrical and Computer Engineering, 2005. IEEE, 2005, pp. 1049–1052.Google Scholar
  28. 28.
    B. Baudry, V. L. Hanh, and Y. L. Traon, “Testing-for-trust: The genetic selection model applied to component qualification,” in Proceedings of the Technology of Object-Oriented Languages and Systems (TOOLS 33), ser. TOOLS ’00. IEEE Computer Society, 2000, pp. 108.Google Scholar
  29. 29.
    B. Baudry, V. Le Hanh, J.-M. Jézéquel, and Y. Le Traon, “Trustable components: Yet another mutation-based approach,” in Mutation testing for the new century. Springer, 2001, pp. 47–54.Google Scholar
  30. 30.
    B. Baudry, F. Fleurey, J.-M. Jézéquel, and Y. Le Traon, “From genetic to bacteriological algorithms for mutation-based testing,” Software Testing, Verification and Reliability, vol. 15, no. 2, pp. 73–96, 2005.Google Scholar
  31. 31.
    Y. M. Ben Ali and F. Benmaiza, “Generating test case for object-oriented software using genetic algorithm and mutation testing method,” International Journal of Applied Metaheuristic Computing, vol. 3, no. 1, pp. 15–23, 2012.Google Scholar
  32. 32.
    H. Haga and A. Suehiro, “Automatic test case generation based on genetic algorithm and mutation analysis,” in 2012 IEEE International Conference on Control System, Computing and Engineering. IEEE, 2012, pp. 119–123.Google Scholar
  33. 33.
    L. Louzada, C. G. Camilo-Junior, A. Vincenzi, and C. Rodrigues, “An elitist evolutionary algorithm for automatically generating test data,” in 2012 IEEE Congress on Evolutionary Computation. IEEE, 2012, pp. 1–8.Google Scholar
  34. 34.
    S. Subramanian and N. Natarajan, “A tool for generation and minimization of test suite by mutant gene algorithm,” Journal of Computer Science, vol. 7, no. 10, pp. 1581–1589, 2011.Google Scholar
  35. 35.
    G. Fraser and A. Zeller, “Mutation-driven generation of unit tests and oracles,” IEEE Transactions on Software Engineering, vol. 38, no. 2, pp. 278–292, 2012.Google Scholar
  36. 36.
    M. B. Bashir and A. Nadeem, “A fitness function for evolutionary mutation testing of object-oriented programs,” in 2013 IEEE 9th International Conference on Emerging Technologies (ICET), 2013, pp. 1–6.Google Scholar
  37. 37.
    K. K. Mishra, S. Tiwari, A. Kumar, and A. K. Misra, “An approach for mutation testing using elitist genetic algorithm,” in 2010 3rd International Conference on Computer Science and Information Technology, vol. 5, 2010, pp. 426–429.Google Scholar
  38. 38.
    C. Molinero, M. Nunez, and C. Andres, “Combining genetic algorithms and mutation testing to generate test sequences,” in Proceedings of the 10th In-ternational Work-Conference on Artificial Neural Networks: Part I: Bio- Inspired Systems: Computational and Ambient Intelligence. Springer, 2009, pp. 343–350.Google Scholar
  39. 39.
    R. Nilsson, J. Offutt, and J. Mellin, “Test case generation for mutation-based testing of timeliness,” Electronic Notes in Theoretical Computer Science, vol. 164, no. 4, pp. 97–114, 2006.Google Scholar
  40. 40.
    C. P. Rao and P. Govindarajulu, “Genetic algorithm for automatic generation of representative test suite for mutation testing,” International Journal of Computer Science and Network Security, vol. 15, no. 2, pp. 11–17, 2015.Google Scholar
  41. 41.
    L. Deng, J. Offutt, and N. Li, “Empirical evaluation of the statement deletion mutation operator,” in Proceedings of the 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation. IEEE Computer Society, 2013, pp. 84–93.Google Scholar
  42. 42.
    M. E. Delamaro, J. Offutt, and P. Ammann, “Designing deletion mutation operators,” in Proceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation. IEEE Computer Society, 2014, pp. 11–20.Google Scholar
  43. 43.
    Java Code Coverage for Eclipse. Available for free at

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Shweta Rani
    • 1
    Email author
  • Hrithik Dhawan
    • 1
  • Gagandeep Nagpal
    • 1
  • Bharti Suri
    • 1
  1. 1.USICTGGS Indraprastha UniversityNew DelhiIndia

Personalised recommendations