Abstract
In this paper, a novel complete model-based equivalence class testing strategy is experimentally evaluated. This black-box strategy applies to deterministic systems with infinite input domains and finite internal state and output domains. It is complete with respect to a given fault model. This means that conforming behaviours will never be rejected, and all nonconforming behaviours inside a given fault domain will be uncovered. We investigate the question how this strategy performs for systems under test whose behaviours lie outside the fault domain. Furthermore, a strategy extension is presented, that is based on randomised data selection from input equivalence classes. While this extension is still complete with respect to the given fault domain, it also promises a higher test strength when applied against members outside this domain. This is confirmed by an experimental evaluation that compares mutation coverage achieved by the original and the extended strategy with the coverage obtained by random testing.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Anand, S., Burke, E.K., Chen, T.Y., Clark, J.A., Cohen, M.B., Grieskamp, W., Harman, M., Harrold, M.J., McMinn, P.: An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software 86(8), 1978–2001 (2013)
Arcuri, A., Iqbal, M.Z., Briand, L.: Black-Box system testing of real-time embedded systems using random and search-based testing. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 95–110. Springer, Heidelberg (2010)
Baier, C., Katoen, J.: Principles of model checking. MIT Press (2008)
Biere, A., Heljanko, K., Junttila, T., Latvala, T., Schuppan, V.: Linear encodings of bounded LTL model checking. Logical Methods in Computer Science 2(5) (November 2006), arXiv: 0611029, arXiv: cs/0611029
Braunstein, C., Haxthausen, A.E., Huang, W., Hübner, F., Peleska, J., Schulze, U., Vu Hong, L.: Complete model-based equivalence class testing for the ETCS ceiling speed monitor. In: Merz, S., Pang, J. (eds.) ICFEM 2014. LNCS, vol. 8829, pp. 380–395. Springer, Heidelberg (2014)
Braunstein, C., Huang, W.l., Peleska, J., Schulze, U., Hübner, F., Haxthausen, A.E., Hong, L.V.: A SysML test model and test suite for the ETCS ceiling speed monitor. Tech. rep., Embedded Systems Testing Benchmarks Site (April 30, 2014). http://www.mbt-benchmarks.org
Chen, T.Y., Kuo, F.C., Merkel, R.G., Tse, T.H.: Adaptive random testing: the art of test case diversity. Journal of Systems and Software 83(1), 60–66 (2010)
Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Transactions on Software Engineering SE 4(3), 178–186 (1978)
Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. The MIT Press, Cambridge (1999)
Fujiwara, S., von Bochmann, G., Khendek, F., Amalou, M., Ghedamsi, A.: Test selection based on finite state models. IEEE Transactions on Software Engineering 17(6), 591–603 (1991)
Gaudel, M.-C.: Testing can be formal, too. In: Mosses, P.D., Nielsen, M. (eds.) CAAP 1995, FASE 1995, and TAPSOFT 1995. LNCS, vol. 915, pp. 82–96. Springer, Heidelberg (1995)
Gill, A.: Introduction to the theory of finite-state machines. McGraw-Hill, New York (1962)
Huang, W.l., Peleska, J.: Complete model-based equivalence class testing. International Journal on Software Tools for Technology Transfer, 1–19 (2014). http://dx.doi.org/10.1007/s10009-014-0356-8
Jaulin, L., Kieffer, M., Didrit, O., Walter, É.: Applied Interval Analysis. Springer, London (2001)
Just, R.: The major mutation framework: efficient and scalable mutation analysis for java. In: Proceedings of the International Symposium on Software Testing and Analysis, ISSTA, July 23–25, pp. 433–436, San Jose, CA, USA (2014)
Ma, Y.S., Offutt, J., Kwon, Y.R.: MuJava: An Automated Class Mutation System: Research Articles. Softw. Test. Verif. Reliab. 15(2), 97–133 (2005). http://dx.doi.org/10.1002/stvr.v15:2
Object Management Group: OMG Unified Modeling Language (OMG UML), superstructure, version 2.4.1. Tech. rep., OMG (2011)
Object Management Group: OMG Systems Modeling Language (OMG SysML\(^{TM}\)), Version 1.3. Tech. rep., Object Management Group (2012). http://www.omg.org/spec/SysML/1.3
Peleska, J., Siegel, M.: Test automation of safety-critical reactive systems. South African Computer Journal 19, 53–77 (1997)
Peleska, J.: Industrial-strength model-based testing - state of the art and current challenges. In: Petrenko, A.K., Schlingloff, H. (eds.) Proceedings Eighth Workshop on Model-Based Testing. Electronic Proceedings in Theoretical Computer Science, vol. 111, pp. 3–28. Open Publishing Association, Rome (2013)
Petrenko, A., Simao, A., Maldonado, J.C.: Model-based testing of software and systems: Recent advances and challenges. Int. J. Softw. Tools Technol. Transf. 14(4), 383–386 (2012). http://dx.doi.org/10.1007/s10009-012-0240-3
Springintveld, J., Vaandrager, F., D’Argenio, P.: Testing timed automata. Theoretical Computer Science 254(1–2), 225–257 (2001)
Tretmans, J.: Model based testing with labelled transition systems. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) FORTEST. LNCS, vol. 4949, pp. 1–38. Springer, Heidelberg (2008)
UNISIG: ERTMS/ETCS System Requirements Specification, Chapter 3, Principles, vol. Subset-026-3, chap. 3, issue 3.3.0 (February 2012)
Vasilevskii, M.P.: Failure diagnosis of automata. Kibernetika (Transl.) 4, 98–108 (1973)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Hübner, F., Huang, Wl., Peleska, J. (2015). Experimental Evaluation of a Novel Equivalence Class Partition Testing Strategy. In: Blanchette, J., Kosmatov, N. (eds) Tests and Proofs. TAP 2015. Lecture Notes in Computer Science(), vol 9154. Springer, Cham. https://doi.org/10.1007/978-3-319-21215-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-21215-9_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-21214-2
Online ISBN: 978-3-319-21215-9
eBook Packages: Computer ScienceComputer Science (R0)