An Efficient Algorithm for Combining Verification and Validation Methods

  • Isela MendozaEmail author
  • Uéverton Souza
  • Marcos Kalinowski
  • Ruben Interian
  • Leonado Gresta Paulino Murta
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11376)


An adequate combination of verification and validation (V&V) methods is important to improve software quality control throughout the development process and to reduce costs. However, to find an appropriate set of V&V methods that properly addresses the desired quality characteristics of a given project is a NP-hard problem. In this paper, we present a novel approach that combines V&V methods efficiently in order to properly cover a set of quality characteristics. We modelled the problem using a bipartite graph to represent the relationships between V&V methods and quality characteristics. Then we interpreted our problem as the Set Cover problem. Although Set Cover is considered hard to be solved, through the theoretical framework of Parameterized Complexity we propose an FPT-Algorithm (fixed-parameter tractable algorithm) that effectively solves the problem, considering the number of quality characteristics to be covered as a fixed parameter. We conclude that the proposed algorithm enables combining V&V methods in a scalable and efficient way, representing a valuable contribution to the community.


Combination Verification Validation Software quality FPT Set cover Parameterized Complexity 



The authors would like to thank CNPq and FAPERJ for the financial support (Project No. E-26/010.001578/2016, Title: “Resolution of Critical Problems of the Software Industry through Graph Theory and its algorithms”). Thanks also to the survey respondents, which provided us the initial configurations to test our approach.


  1. 1.
    Meyers, G.J., Badgett, T., Thomas, T., Csandler, C.: The Art of Software Testing, 3rd edn. Wiley, Hoboken (2011). ISBN 978-1118031964Google Scholar
  2. 2.
    Feldt, R., Torkar, R., Ahmad, E., Raza, B.: Challenges with software verification and validation activities in the space industry. In: Third International Conference on Software Testing, Verification and Validation (ICST) (2010)Google Scholar
  3. 3.
    Boehm, B., Basili, V.: Software defect reduction top 10 list. IEEE Softw. 34(1), 135–137 (2001)Google Scholar
  4. 4.
    Feldt, R., Marculescu, B., Schulte, J., Torkar, R., Preissing, P., Hult, E.: Optimizing verification and validation activities for software in the space industry. In: Data Systems in Aerospace (DASIA), Budapest (2010)Google Scholar
  5. 5.
    Bourque, P., Fairley, R.E.: SWEBOK guide V3.0, guide to the software engineering body of knowledge. IEEE Computer Society (2004)Google Scholar
  6. 6.
    Endres, A., Rombach, D.: A Handbook of Software and Systems Engineering. Addison Wesley, Reading (2003)Google Scholar
  7. 7.
    Myers, G.J.: A controlled experiment in program testing and code walkthroughs/inspections. Commun. ACM 21(9), 760–768 (1978)CrossRefGoogle Scholar
  8. 8.
    Wood, M., Roper, M., Brooks, A., Miller, J.: Comparing and combining software defect detection techniques: a replicated empirical study. In: Jazayeri, M., Schauer, H. (eds.) ESEC/SIGSOFT FSE-1997. LNCS, vol. 1301, pp. 262–277. Springer, Heidelberg (1997). Scholar
  9. 9.
    Elberzhager, F., Münch, J., Nha, V.T.N.: A systematic mapping study on the combination of static and dynamic quality assurance techniques. Inf. Softw. Technol. 54(1), 1–15 (2012)CrossRefGoogle Scholar
  10. 10.
    ISO25000 Software Product Quality, ISO/IEC 25010, Official site (2011).
  11. 11.
    Karp, R.M.: Reducibility among combinatorial problems. In: Miller, R.E., Thatcher, J.W. (eds.) Complexity of Computer Computations, pp. 85–103. Plenum, New York (1972)CrossRefGoogle Scholar
  12. 12.
    dos Santos, V.F., dos Santos Souza, U.: Uma Introdução à Complexidade Parametrizada. In: Anais da 34º Jornada de Atualização em Informática, CSBC, pp. 232–273 (2015)Google Scholar
  13. 13.
    Downey, R.G., Fellows, M.R.: Parameterized Complexity. Monographs in Computer Science. Springer, New York (1999). Scholar
  14. 14.
    Flum, J., Grohe, M.: Parameterized Complexity Theory. Springer, Heidelberg (2006). Scholar
  15. 15.
    Niedermeier, R.: Invitation to Fixed-Parameter Algorithms. Oxford Lecture Series in Mathematics and Its Applications. Oxford University Press, Oxford (2006)CrossRefGoogle Scholar
  16. 16.
    Bondi, A.B.: Characteristics of scalability and their impact on performance. In: Proceedings Second International Workshop on Software and Performance WOSP, pp. 195–203 (2000)Google Scholar
  17. 17.
    Laudon, K.C., Traver, C.G.: E-commerce: Business, Technology, Society. Stanford University, Stanford (2008)Google Scholar
  18. 18.
    Lubell, D.: A short proof of Sperner’s lemma. J. Comb. Theory 1(2), 299 (1996)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Wagner, S.: Software Product Quality Control. Springer, Heidelberg (2013). Scholar
  20. 20.
    Wiegers, K.E.: Peer Reviews in Software: A Practical Guide, 1st edn. Addison-Wesley Longman Publishing Co., Inc., Boston (2002)Google Scholar
  21. 21.
    Mendoza, I., Kalinowski, M., Souza, U., Felderer, M.: Relating verification and validation methods to software product quality characteristics: results of an expert survey. In: 11th Software Quality Days (SWQD). Lecture Notes on Business Information Processing, Vienna, Austria. Springer (2019, to appear)Google Scholar
  22. 22.
    Basili, V.R.: Comparing the effectiveness of software testing strategies. IEEE Trans. Softw. Eng. 13(12), 1278–1296 (1987)CrossRefGoogle Scholar
  23. 23.
    Kamsties, E., Lott, C.M.: An empirical evaluation of three defect-detection techniques. In: Schäfer, W., Botella, P. (eds.) ESEC 1995. LNCS, vol. 989, pp. 362–383. Springer, Heidelberg (1995). Scholar
  24. 24.
    Wagner, S., Jürjens, J., Koller, C., Trischberger, P.: Comparing bug finding tools with reviews and tests. In: Khendek, F., Dssouli, R. (eds.) TestCom 2005. LNCS, vol. 3502, pp. 40–55. Springer, Heidelberg (2005). Scholar
  25. 25.
    Dwyer, M.B., Elbaum, S.: Unifying verification and validation techniques: relating behavior and properties through partial evidence. In: FSE/SDP Workshop on Future of Software Engineering Research (FOSE), Santa Fe, New Mexico, USA, pp. 93–98 (2010)Google Scholar
  26. 26.
    Cygan, M., et al.: Parameterized Algorithms. Springer, Cham (2015). Scholar
  27. 27.
    Runeson, P., Stefik, A., Andrews, A., Grönblom, S., Porres, I., Siebert, S.: A comparative analysis of three replicated experiments comparing inspection and unit testing. In: Proceedings 2nd International Workshop on Replication in Empirical Software Engineering Research (RESER), Banff, AB, Canada, Article No. 6148335, pp. 35–42 (2012)Google Scholar
  28. 28.
    Olorisade, B.K., Vegas, S., Juristo, N.: Determining the effectiveness of three software evaluation techniques through informal aggregation. Inf. Softw. Technol. 55(9), 1590–1601 (2013)CrossRefGoogle Scholar
  29. 29.
    Cotroneo, D., Pietrantuono, R., Russo, S.: A learning-based method for combining testing techniques. In: Proceedings 35th International Conference on Software Engineering (ICSE), San Francisco, CA, USA, Article No. 6606560, pp. 142–151 (2013)Google Scholar
  30. 30.
    Bishop, P., Bloomfield, R., Cyra, L.: Combining testing and proof to gain high assurance in software: a case study. In: IEEE 24th International Symposium on Software Reliability Engineering (ISSRE), Pasadena, CA, USA, Article No. 6698924, pp. 248–257 (2013)Google Scholar
  31. 31.
    Solari, M., Matalonga, S.: A controlled experiment to explore potentially undetectable defects for testing techniques. In: Proceedings of the 26th International Conference on Software Engineering and Knowledge Engineering (SEKE), Canada, pp. 106–109 (2014)Google Scholar
  32. 32.
    Gleirscher, M., Golubitskiy, D., Irlbeck, M., Wagner, S.: Introduction of static quality analysis in small- and medium-sized software enterprises: experiences from technology transfer. Softw. Qual. J. 22(3), 499–542 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Isela Mendoza
    • 1
    Email author
  • Uéverton Souza
    • 1
  • Marcos Kalinowski
    • 2
  • Ruben Interian
    • 1
  • Leonado Gresta Paulino Murta
    • 1
  1. 1.Computer InstituteFluminense Federal UniversityNiteróiBrazil
  2. 2.Informatics DepartmentPontifical Catholic University of Rio de JaneiroRio de JaneiroBrazil

Personalised recommendations