Skip to main content
SpringerLink
Log in

Life Sciences-Inspired Test Case Similarity Measures for Search-Based, FSM-Based Software Testing

  • Conference paper
  • First Online:
Modelling Foundations and Applications (ECMFA 2018)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10890))

Included in the following conference series:

Abstract

Researchers and practitioners alike have the intuition that test cases diversity is positively correlated to fault detection. Empirical results already show that some measurement of diversity within a pre-existing state-based test suite (i.e., a test suite not necessarily created to have diverse tests in the first place) indeed relates to fault detection. In this paper we show how our procedure, based on a genetic algorithm, to construct an entire (all-transition) adequate test suite with as diverse tests as possible fares in terms of fault detection. We experimentally compare on a case study nine different ways of computing test suite diversity, including measures already used by others in software testing as well as measures inspired by the notion of diversity in the life sciences. Although our results confirm a positive correlation between diversity and fault detection, we believe our results raise more questions than they answer about the notion and measurement of test suite diversity, which leads us to argue that more work needs to be dedicated to this topic.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    An allele is a form of a gene (one member of a pair) that is located at a specific position on a specific chromosome. Locus (plural: loci) is the location of a gene on a chromosome.

References

  1. Andrews, J.H., Briand, L.C., Labiche, Y.: Is mutation an appropriate tool for testing experiments? In: Proceedings of the IEEE ICSE, pp. 402–411 (2005)

    Google Scholar 

  2. Andrews, J.H., Briand, L.C., Labiche, Y., Namin, A.S.: Using mutation analysis for assessing and comparing testing coverage criteria. IEEE TSE 32(8), 608–624 (2006)

    Google Scholar 

  3. Asoudeh, N.: Test generation from an extended finite state machine as a multiobjective optimization problem, thesis, Carleton University (2016)

    Google Scholar 

  4. Asoudeh, N., Labiche, Y.: A multi-objective genetic algorithm for generating test suites from extended finite state machines. In: Ruhe, G., Zhang, Y. (eds.) SSBSE 2013. LNCS, vol. 8084, pp. 288–293. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39742-4_26

    Chapter  Google Scholar 

  5. Asoudeh, N., Labiche, Y.: Multi-objective construction of an entire adequate test suite for an EFSM. In: Proceedings of the IEEE ISSRE (2014)

    Google Scholar 

  6. Asoudeh, N., Labiche, Y.: On the effect of counters in guard conditions when state-based multi-objective testing. In: Proceedings of the IEEE Software Quality, Reliability and Security-Companion (2015)

    Google Scholar 

  7. Beizer, B.: Software Testing Techniques. International Thomson Computer Press, New York (1990)

    MATH  Google Scholar 

  8. Bellman, R., Cooke, K.L., Lockett, J.A.: Algorithms. Academic Press, New York (1970)

    Google Scholar 

  9. Binder, R.V.: Testing Object-Oriented Systems: Models, Patterns, and Tools`, Object Technology. Addison-Wesley, Boston (1999)

    Google Scholar 

  10. Briand, L.C., Di Penta, M., Labiche, Y.: Assessing and improving state-based class testing: a series of experiments. IEEE TSE 30(11), 770–793 (2004)

    Google Scholar 

  11. Briand, L.C., Labiche, Y., Wang, Y.: Using simulation to empirically investigate test coverage criteria. In: Proceedings of the IEEE/ACM ICSE, pp. 86–95 (2004)

    Google Scholar 

  12. Bruegge, B., Dutoit, A.H.: Object-Oriented Software Engineering Using UML, Patterns, and Java. Prentice Hall, Upper Saddle River (2004)

    Google Scholar 

  13. Chevalley, P., Thévenod-Fosse, P.: Automated generation of statistical test cases from UML state diagrams. In: Proceedings of the COMPSAC (2001)

    Google Scholar 

  14. Cohen, J.: Statistical Power Analysis for the Behavioral Sciences. Routledge, Abingdon (1988)

    MATH  Google Scholar 

  15. Devore, J.L.: Probability and Statistics for Engineering and the Sciences, 5th edn. Duxbury Press, Scituate (1999)

    Google Scholar 

  16. Feldt, R., Poulding, S., Clark, D., Yoo, S.: Test set diameter: quantifying the diversity of sets of test cases. In: Proceedings of the IEEE ICST (2016)

    Google Scholar 

  17. Gomaa, H.: Designing Concurrent, Distributed, and Real-Time Applications with UML. Addison Wesley, Boston (2000)

    Google Scholar 

  18. Gusfield, D.: Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology. Cambridge University Press, Cambridge (1997)

    Book  Google Scholar 

  19. Harman, M., Lakhotia, K., Singer, J., White, D.R., Yoo, S.: Cloud engineering is search based software engineering too. JSS 86(9), 2225–2241 (2013)

    Google Scholar 

  20. Hemmati, H., Arcuri, A., Briand, L.: Achieving scalable model-based testing through test case diversity. ACM TOSEM 22(1), 6:1–6:42 (2013)

    Article  Google Scholar 

  21. Holt, N.E., Anda, B.C.D., Asskildt, K., Briand, L., Endresen, J., Frøystein, S.: Experiences with precise state modeling in an industrial safety critical system. In: Proceedings of the Models Workshop on Critical Systems Development Using Modeling Languages (2006)

    Google Scholar 

  22. Holt, N.E., Torkar, R., Briand, L., Hansen, K.: State-based testing: industrial evaluation of the cost-effectiveness of round-trip path and sneak-path strategies. In: Proceedings of the IEEE ISSRE, pp. 321–330 (2012)

    Google Scholar 

  23. Inverardi, P., Autili, M., Di Ruscio, D., Pelliccione, P., Tivoli, M.: Producing software by integration: challenges and research directions. In: Proceedings of the FSE (2013)

    Google Scholar 

  24. Iversen, G.R., Norpoth, H.: Analysis of Variance. Sage publications, Thousand Oaks (1987)

    Book  Google Scholar 

  25. Jost, L.: Partitioning diversity into independent alpha and beta components. Ecology 88, 2427–2439 (2007)

    Article  Google Scholar 

  26. Jost, L.: GST and its relatives do not measure differentiation. Mol. Ecol. 17, 4015–4026 (2008)

    Article  Google Scholar 

  27. Just, R., Jalali, D., Inozemtseva, L., Ernst, M.D., Holmes, R., Fraser, G.: Are mutants a valid substitute for real faults in software testing. In: Proceedings of the FSE (2014)

    Google Scholar 

  28. Just, R., Schweiggert, F., Kapfhammer, G.M.: MAJOR: an efficient and extensible tool for mutation analysis in a Java compiler. In: Proceedings of the ASE (2011)

    Google Scholar 

  29. Kalaji, A.S., Hierons, R.M., Swift, S.: An integrated search-based approach for automatic testing from extended finite state machine (EFSM) models. IST 53(12), 1297–1318 (2011)

    Google Scholar 

  30. Khalil, M., Labiche, Y.: On the round trip path testing strategy. In: Proceedings of the IEEE ISSRE, pp. 388–397 (2010)

    Google Scholar 

  31. Kosman, E.: Measuring diversity: from individuals to populations. Eur. J. Plant Pathol. 138, 467–486 (2014)

    Article  Google Scholar 

  32. Kosman, E., Leonard, K.J.: Conceptual analysis of methods applied to assessment of diversity within and distance between populations with asexual or mixed mode of reproduction. New Phytol. 174, 683–696 (2007)

    Article  Google Scholar 

  33. Lee, D., Yannakakis, M.: Principles and methods of testing finite state machines - a survey. Proc. IEEE 84(8), 1090–1123 (1996)

    Article  Google Scholar 

  34. Legeard, B.: Model-based testing: next generation functional software testing. In: Dagstuhl Seminar Proceedings of Practical Software Testing: Tool Automation and Human Factors (2010)

    Google Scholar 

  35. Mondal, D., Hemmati, H., Durocher, S.: Exploring test suite diversification and code coverage in multi-objective test case selection. In: Proceedings of the IEEE ICST (2015)

    Google Scholar 

  36. Mouchawrab, S., Briand, L.C., Labiche, Y., Di Penta, M.: Assessing, comparing, and combining state machine-based testing and structural testing: a series of experiments. IEEE TSE 37(2), 161–187 (2011)

    Google Scholar 

  37. Nei, M.: Analysis of gene diversity in subdivided population. Nat. Acad. Sci. U.S.A. 70, 3321–3323 (1973)

    Article  Google Scholar 

  38. Neto, A., Travassos, G.H.: Surveying model based testing approaches characterization attributes. In: Proceedings of the ACM/IEEE ESEM, pp. 324–326 (2008)

    Google Scholar 

  39. Patrick, M., Jiab, Y.: KD-ART: should we intensify or diversify tests to kill mutants? IST 81, 36–51 (2017)

    Google Scholar 

  40. Rao, C.R.: Diversity and dissimilarity coefficients - a unified approach. Theor. Popul. Biol. 21, 24–43 (1982)

    Article  MathSciNet  Google Scholar 

  41. Ricotta, C.: Bridging the gap between ecological diversity indices and measures of biodiversity with Shannon’s entropy. Ecol. Model. 152, 1–3 (2002)

    Article  Google Scholar 

  42. Ricotta, C., Szeidl, L.: Towards a unifying approach to diversity measures: bridging the gap between Shannon’s entropy and Rao’s quadratic index. Theor. Popul. Biol. 70, 237–243 (2006)

    Article  Google Scholar 

  43. Saifan, A., Dingel, J.: A survey of using model-based testing to improve quality attributes in distributed systems. In: Elleithy, K. (ed.) Advanced Techniques in Computing Sciences and Software Engineering, pp. 283–288. Springer, Dordrecht (2010). https://doi.org/10.1007/978-90-481-3660-5_48

    Chapter  Google Scholar 

  44. Shafique, M., Labiche, Y.: A systematic review of state-based test tools. STTT 17(1), 59–76 (2015)

    Article  Google Scholar 

  45. Shi, Q., Chen, Z., Fang, C., Feng, Y., Xu, B.: Measuring the diversity of a test set with distance entropy. IEEE Trans. Reliab. 65(1), 19–27 (2016)

    Article  Google Scholar 

  46. Tuomisto, H.: An updated consumer’s guide to evenness and related indices. Oikos 121, 1203–1218 (2012)

    Article  Google Scholar 

  47. Utting, M., Legeard, B.: Practical Model-based Testing. Morgan Kaufmann, Los Altos (2006)

    Google Scholar 

  48. Wang, R., Jiang, S., Chen, D., Zhang, Y.: Empirical study of the effects of different similarity measures on test case prioritization. Math. Probl. Eng. 2016, 19 p. (2016). Article ID 8343910

    Google Scholar 

Download references

Acknowledgements

This work was performed under the umbrella of a Collaborative Research and Development Grant with the Natural Sciences and Engineering Research Council of Canada (NSERC), with support from NSERC, CRIAQ (Consortium for Research and Innovation in Aerospace in Québec), CAE, CMC Electronics, and Mannarino Systems & Software. We would like to thank Professor Root Gorelick from the department of Biology at Carleton University for fruitful discussions on diversity.

Author information

Authors and Affiliations

  1. Carleton University, Ottawa, K1S5B6, ON, Canada

    Nesa Asoudeh & Yvan Labiche

Authors
  1. Nesa Asoudeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yvan Labiche
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yvan Labiche .

Editor information

Editors and Affiliations

  1. University of L'Aquila, L'Aquila, Italy

    Alfonso Pierantonio

  2. Ikerlan, Arrasate-Mondragón, Spain

    Salvador Trujillo

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Asoudeh, N., Labiche, Y. (2018). Life Sciences-Inspired Test Case Similarity Measures for Search-Based, FSM-Based Software Testing. In: Pierantonio, A., Trujillo, S. (eds) Modelling Foundations and Applications. ECMFA 2018. Lecture Notes in Computer Science(), vol 10890. Springer, Cham. https://doi.org/10.1007/978-3-319-92997-2_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-92997-2_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-92996-5

  • Online ISBN: 978-3-319-92997-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation