Search-Based Software Engineering to Construct Binary Test-Suites

  • Jose Torres-JimenezEmail author
  • Himer Avila-GeorgeEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 405)


Search-based software engineering is the application of optimization techniques in solving software engineering problems. One challenge to testing software systems is the effort involved in creating test suites that will systematically test the system and reveal faults in an effective manner. Given the importance of the software testing phase, a specific subarea called search-based software testing has become increasingly important. This paper presents a search-based software testing tool (SBSTT), for constructing test suites. Through the use of SBSTT we were able to find 370 new upper bounds for binary test suites.


Search-based software testing Combinatorial testing Binary test-suites Covering arrays 



The authors acknowledge GENERAL COORDINATION OF INFORMATION AND COMMUNICATIONS TECHNOLOGIES (CGSTIC) at CINVESTAV for providing HPC resources on the Hybrid Cluster Supercomputer “Xiuhcoatl”, that have contributed to the research results reported. The following projects have funded the research reported in this paper: 51623 - Fondo Mixto CONACyT y Gobierno del Estado de Tamaulipas; 238469 - CONACyT Métodos Exactos para Construir Covering Arrays Óptimos; 232987 - CONACyT Conjuntos de Prueba Óptimos para Métodos Combinatorios (Optimal Test Sets for Combinatorial Methods); 2143 - Cátedras CONACyT.


  1. 1.
    Harman, M., Jones, B.F.: Search-based software engineering. Inf. Softw. Technol. 43(14), 833–839 (2001)CrossRefGoogle Scholar
  2. 2.
    Ali, S., Briand, L.C., Hemmati, H., Panesar-Walawege, R.K.: A systematic review of the application and empirical investigation of search-based test case generation. IEEE Trans. Softw. Eng. 36(6), 742–762 (2010)CrossRefGoogle Scholar
  3. 3.
    Kuhn, D.R., Lei, Y., Kacker, R.N.: Practical combinatorial testing: beyond pairwise. IT Prof. 10(3), 19–23 (2008)CrossRefGoogle Scholar
  4. 4.
    Cawse, J.N.: Experimental Design for Combinatorial and High Throughput Materials Development. Wiley, New York (2003)Google Scholar
  5. 5.
    Hedayat, A.S., Sloane, N.J.A., Stufken, J.: Orthogonal Arrays: Theory and Applications. Springer Science & Business Media, Berlin (1999)Google Scholar
  6. 6.
    Shasha, D.E., Kouranov, A.Y., Lejay, L.V., Chou, M.F., Coruzzi, G.M.: Using combinatorial design to study regulation by multiple input signals: a tool for parsimony in the post-genomics era. Plant Physiol. 127(4), 1590–1594 (2001)CrossRefGoogle Scholar
  7. 7.
    Vadde, K.K., Syrotiuk, V.R.: Factor interaction on service delivery in mobile ad hoc networks. IEEE J. Sel. Areas Commun. 22(7), 1335–1346 (2004)CrossRefGoogle Scholar
  8. 8.
    Avila-George, H., Torres-Jimenez, J., Gonzalez-Hernandez, L., Hernández, V.: Metaheuristic approach for constructing functional test-suites. IET Softw. 7(2), 104–117 (2013)CrossRefGoogle Scholar
  9. 9.
    Avila-George, H., Torres-Jimenez, J., Hernández, V.: Constructing real test-suites using an enhanced simulated annealing. In: Pavón, J., Duque-Méndez, N.D., Fuentes-Fernández, R. (eds.) Advances in Artificial Intelligence – IBERAMIA 2012, pp. 611–620. Springer, Berlin (2012)CrossRefGoogle Scholar
  10. 10.
    Colbourn, C.J.: Covering array tables for t = 2,3,4,5,6. Accessed 1 July 2015
  11. 11.
    Lawrence, J., Kacker, R.N., Lei, Y., Kuhn, D.R., Forbes, M.: A survey of binary covering arrays. Electron J. Comb. 18(1), 1–30 (2011)MathSciNetGoogle Scholar
  12. 12.
    Afzal, W., Torkar, R., Feldt, R.: A systematic review of search-based testing for non-functional system properties. Inf. Softw. Technol. 51(6), 957–976 (2009)CrossRefGoogle Scholar
  13. 13.
    Stardom, J.: Metaheuristics and the Search for Covering and Packing Arrays. Simon Fraser University, Burnaby (2001)Google Scholar
  14. 14.
    Cohen, M.B., Gibbons, P.B., Mugridge, W.B., Colbourn, C.J.: Constructing test suites for interaction testing. In: Proceedings of the 25th International Conference on Software Engineering, 2003, pp. 38–48 (2003)Google Scholar
  15. 15.
    Tung, Y.-W., Aldiwan, W.S.: Automating test case generation for the new generation mission software system. In: 2000 IEEE Aerospace Conference Proceedings, vol. 1, pp. 431–437 (2000)Google Scholar
  16. 16.
    Cohen, D.M., Dalal, S.R., Fredman, M.L., Patton, G.C.: The AETG system: an approach to testing based on combinatorial design. IEEE Trans. Softw. Eng. 23(7), 437–444 (1997)CrossRefGoogle Scholar
  17. 17.
    Shiba, T., Tsuchiya, T., Kikuno, T.: Using artificial life techniques to generate test cases for combinatorial testing. In: Proceedings of the 28th Annual International Computer Software and Applications Conference, pp. 72–77 (2004)Google Scholar
  18. 18.
    Lei, Y., Tai, K.-C.: In-parameter-order: a test generation strategy for pairwise testing. In: Proceedings of the Third IEEE International High-Assurance Systems Engineering Symposium, 1998, pp. 254–261 (1998)Google Scholar
  19. 19.
    Nurmela, K.J.: Upper bounds for covering arrays by tabu search. Discret. Appl. Math. 138(1–2), 143–152 (2004)CrossRefMathSciNetzbMATHGoogle Scholar
  20. 20.
    Bryce, R.C., Colbourn, C.J.: The density algorithm for pairwise interaction testing. Softw. Test. Verif. Reliab. 17(3), 159–182 (2007)CrossRefGoogle Scholar
  21. 21.
    Cohen, M.B., Colbourn, C.J., Ling, A.C.H.: Constructing strength three covering arrays with augmented annealing. Discret. Math. 308(13), 2709–2722 (2008)CrossRefMathSciNetzbMATHGoogle Scholar
  22. 22.
    Walker II, R.A., Colbourn, C.J.: Tabu search for covering arrays using permutation vectors. J. Stat. Planning Infer. 139(1), 69–80 (2009)CrossRefMathSciNetzbMATHGoogle Scholar
  23. 23.
    Martinez-Pena, J., Torres-Jimenez, J., Rangel-Valdez, N., Avila-George, H.: A heuristic approach for constructing ternary covering arrays using trinomial coefficients. In: Kuri-Morales, A., Simari, G. (eds.) Advances in Artificial Intelligence—IBERAMIA 2010, vol. 6433, pp. 572–581. Springer, Berlin (2010)CrossRefGoogle Scholar
  24. 24.
    Torres-Jimenez, J., Rodriguez-Tello, E.: New bounds for binary covering arrays using simulated annealing. Inf. Sci. 185(1), 137–152 (2012)CrossRefGoogle Scholar
  25. 25.
    Avila-George, H., Torres-Jimenez, J., Hernández, V.: New bounds for ternary covering arrays using a parallel simulated annealing. Math. Probl. Eng. 2012(Article ID 897027), 19 (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Information Technology LaboratoryCINVESTAV-TamaulipasCiudad VictoriaMexico
  2. 2.Haramara TIC-LABCICESE-UTTepicMexico
  3. 3.Cátedras CONACyTCiudad deMexico

Personalised recommendations