Abstract
In this paper we give an overview of our work on combining model-based testing and mutation testing. Model-based testing is a black-box testing technique that avoids the labour of manually writing hundreds of test cases, but instead advocates the capturing of the expected behaviour in a model of the system-under-test. The test cases are automatically generated from this model. The technique is receiving growing interest in the embedded-systems domain, where models are the rule rather than the exception.
Mutation testing is a technique for assessing and improving a test suite. A number of faulty versions of a program-under-test are produced by injecting bugs into its source code. These faulty programs are called mutants. A tester analyses if his test suite can ”kill” all mutants. We say that a test kills a mutant if it is able to distinguish it from the original. The tester improves his test suite until all faulty mutants get killed.
In model-based mutation testing, we combine the central ideas of model-based testing and mutation testing: we inject bugs in a model and generate a test suite that will kill these bugs. In this paper, we discuss its scientific foundations and tools. The foundations include semantics and conformance relations; the supporting tools involve model checkers, constraint solvers and SMT solvers.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aichernig, B.K.: Test-case calculation through abstraction. In: Oliveira, J.N., Zave, P. (eds.) FME 2001. LNCS, vol. 2021, pp. 571–589. Springer, Heidelberg (2001)
Aichernig, B.K.: Mutation Testing in the Refinement Calculus. Formal Aspects of Computing 15(2-3), 280–295 (2003)
Aichernig, B.K., Brandl, H., Jöbstl, E., Krenn, W.: Efficient mutation killers in action. In: IEEE Fourth International Conference on Software Testing, Verification and Validation, ICST 2011, Berlin, Germany, March 21-25, pp. 120–129. IEEE Computer Society (2011)
Aichernig, B.K., Brandl, H., Wotawa, F.: Conformance testing of hybrid systems with qualitative reasoning models. In: Finkbeiner, B., Gurevich, Y., Petrenko, A.K. (eds.) Proceedings of Fifth Workshop on Model Based Testing, MBT 2009, York, England, March 22. Electronic Notes in Theoretical Computer Science, vol. 253(2), pp. 53–69. Elsevier (October 2009)
Aichernig, B.K., Delgado, C.C.: From faults via test purposes to test cases: on the fault-based testing of concurrent systems. In: Baresi, L., Heckel, R. (eds.) FASE 2006. LNCS, vol. 3922, pp. 324–338. Springer, Heidelberg (2006)
Aichernig, B.K., He, J.: Mutation testing in UTP. Formal Aspects of Computing 21(1-2), 33–64 (2009)
Aichernig, B.K., Jöbstl, E.: Towards symbolic model-based mutation testing: Combining reachability and refinement checking. In: 7th Workshop on Model-Based Testing, MBT 2012. EPTCS, vol. 80, pp. 88–102 (2012)
Aichernig, B.K., Jöbstl, E.: Towards symbolic model-based mutation testing: Pitfalls in expressing semantics as constraints. In: Workshops Proc. of the 5th Int. Conf. on Software Testing, Verification and Validation, ICST 2012, pp. 752–757. IEEE Computer Society (2012)
Aichernig, B.K., Jöbstl, E., Kegele, M.: Incremental refinement checking for test case generation. In: Veanes, M., Viganò, L. (eds.) TAP 2013. LNCS, vol. 7942, pp. 1–19. Springer, Heidelberg (2013)
Aichernig, B.K., Lorber, F., Ničković, D.: Time for mutants: Model-based mutation testing with timed automata. In: Veanes, M., Viganò, L. (eds.) TAP 2013. LNCS, vol. 7942, pp. 20–38. Springer, Heidelberg (2013)
Aichernig, B.K., Salas, P.A.P.: Test case generation by OCL mutation and constraint solving. In: Cai, K.-Y., Ohnishi, A. (eds.) Fifth International Conference on Quality Software, QSIC 2005, Melbourne, Australia, September 19-21, pp. 64–71. IEEE Computer Society (2005)
Back, R.-J., Kurki-Suonio, R.: Decentralization of process nets with centralized control. In: Proceedings of the 2nd ACM SIGACT-SIGOPS Symp. on Principles of Distributed Computing, Montreal, Quebec, Canada, pp. 131–142. ACM (1983)
Bloch, J.: Extra, extra - read all about it: Nearly all binary searches and mergesorts are broken. Google Research Blog (June 2006), http://googleresearch.blogspot.com/2006/06/extra-extra-read-all-about-it-nearly.html (last visit May 17, 2013)
Brandl, H., Weiglhofer, M., Aichernig, B.K.: Automated conformance verification of hybrid systems. In: Wang, J., Chan, W.K., Kuo, F.-C. (eds.) Proceedings of the 10th International Conference on Quality Software, QSIC 2010, Zhangjiajie, China, July 14-15, pp. 3–12. IEEE Computer Society (2010)
Dijkstra, E.W.: The humble programmer. Communications of the ACM 15(10), 859–866 (1972)
Hoare, C.A.R., He, J.: Unifying Theories of Programming. Prentice-Hall International (1998)
Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering 37(5), 649–678 (2011)
Krenn, W., Aichernig, B.K.: Test case generation by contract mutation in Spec#. In: Finkbeiner, B., Gurevich, Y., Petrenko, A.K. (eds.) Proceedings of Fifth Workshop on Model Based Testing, MBT 2009, York, England, March 22. Electronic Notes in Theoretical Computer Science, vol. 253(2), pp. 71–86. Elsevier (October 2009)
Meng, S., Arbab, F., Aichernig, B.K., Astefanoaei, L., de Boer, F.S., Rutten, J.: Connectors as designs: Modeling, refinement and test case generation. Science of Computer Programming (2011) (in press, corrected proof)
Popper, K.: Logik der Forschung, 10th edn. Mohr Siebeck (2005)
Tiran, S.: On the effects of UML modeling styles in model-based mutation testing. Master’s thesis, Institute for Software Technology, Graz University of Technology (2013)
Tretmans, J.: Test generation with inputs, outputs and repetitive quiescence. Software - Concepts and Tools 17(3), 103–120 (1996)
Weiglhofer, M., Aichernig, B., Wotawa, F.: Fault-based conformance testing in practice. International Journal of Software and Informatics 3(2-3), 375–411 (2009); Special double issue on Formal Methods of Program Development edited by Dines Bjoerner
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Aichernig, B.K. (2013). Model-Based Mutation Testing of Reactive Systems. In: Liu, Z., Woodcock, J., Zhu, H. (eds) Theories of Programming and Formal Methods. Lecture Notes in Computer Science, vol 8051. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39698-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-39698-4_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39697-7
Online ISBN: 978-3-642-39698-4
eBook Packages: Computer ScienceComputer Science (R0)