Abstract
In model-based testing, test cases are generated from a specification model. To avoid an exhaustive search for all possible test cases that can be obtained, usually an expensive and infeasible activity, test case generation may be guided by a test selection criterion. The objective of a test selection criterion is to produce a minimal test suite and yet effective to reveal faults. However, the choice of a criterion is not straightforward specially for real-time systems, because most criteria presented in the literature are general-purpose. Moreover, the relationship between general-purpose and specific criteria for real-time systems is not clear. In this paper, we investigate the criteria that can be applied for test case generation in the scope of model-based testing of real-time systems, specifically of Timed Input-Output Symbolic Transition Systems (TIOSTS) models. We formalize a family of 19 test selection criteria ordered by strict inclusion relation for TIOSTS models. The family combines general-purpose data-flow-oriented and transition-based criteria with specific reactive and real-time systems criteria. We also perform an empirical study to compare the effectiveness of selected criteria. Results of the empirical study indicate that failure detection capability of the generated test suite may vary, but differences are not significant for time failures. We conclude that more effective criteria for the model-based testing of real-time systems are still needed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alagar, V.S., Ormandjieva, O., Zheng, M.: Specification-based testing for real-time reactive systems. In: Proceedings of the 34th International Conference on Technology of Object-Oriented Languages and Systems, pp. 25–36 (2000)
Almeida, D.R.: Critérios de Geração de Casos de Teste de Sistemas de Tempo Real. Master’s thesis, Federal University of Campina Grande, Campina Grande, PB, Brazil (2012)
Alur, R., Dill, D.L.: A theory of timed automata. Theoretical Computer Science 126(2), 183–235 (1994)
Andrade, W.L., Machado, P.D.L.: Testing interruptions in reactive systems. Formal Aspects of Computing 24, 331–353 (2012)
Andrade, W.L., Machado, P.D.L.: Generating test cases for real-time systems based on symbolic models. IEEE Transactions on Software Engineering 39(9), 1216–1229 (2013)
Andrade, W.L., Machado, P.D.L., Jéron, T., Marchand, H.: Abstracting time and data for conformance testing of real-time systems. In: Proceedings of the 8th Workshop on Advances in Model Based Testing, pp. 9–17 (2011)
Arcuri, A., Iqbal, M.Z., Briand, L.: Black-box system testing of real-time embedded systems using random and search-based testing. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 95–110. Springer, Heidelberg (2010)
Bengtsson, J., Griffioen, W.O.D., Kristoffersen, K.J., Larsen, K.G., Larsson, F., Pettersson, P., Yi, W.: Verification of an audio protocol with bus collision using UPPAAL. In: Alur, R., Henzinger, T.A. (eds.) CAV 1996. LNCS, vol. 1102, pp. 244–256. Springer, Heidelberg (1996)
Clarke, D., Lee, I.: Automatic test generation for the analysis of a real-time system: case study. In: Proceedings of the 3rd IEEE Real-Time Technology and Applications Symposium, pp. 112–124 (1997)
El-Far, I.K., Whittaker, J.A.: Model-based software testing. In: Marciniak, J.J. (ed.) Encyclopedia of Software Engineering, vol. 1, pp. 825–837. John Wiley & Sons, Inc. (2002)
En-Nouaary, A., Khendek, F., Dssouli, R.: Fault coverage in testing real-time systems. In: Proceedings of the 6th Real-Time Computing Systems and Applications, pp. 150–157 (1999)
En-Nouaary, A.: Test selection criteria for real-time systems modeled as timed input-output automata. International Journal of Web Information Systems 3(4), 279–292 (2007)
En-Nouaary, A., Hamou-Lhadj, A.: A boundary checking technique for testing real-time systems modeled as timed input output automata. In: Proceedings of the 8th International Conference on Quality Software, pp. 209–215 (2008)
Hessel, A.: Model-Based Test Case Selection and Generation for Real-Time Systems. Ph.D. thesis, Uppsala University, Uppsala, Sweden (2007)
Jeannet, B., Jéron, T., Rusu, V., Zinovieva, E.: Symbolic test selection based on approximate analysis. In: Halbwachs, N., Zuck, L.D. (eds.) TACAS 2005. LNCS, vol. 3440, pp. 349–364. Springer, Heidelberg (2005)
Krichen, M., Tripakis, S.: Black-box conformance testing for real-time systems. In: Graf, S., Mounier, L. (eds.) SPIN 2004. LNCS, vol. 2989, pp. 109–126. Springer, Heidelberg (2004)
Krichen, M., Tripakis, S.: Conformance testing for real-time systems. Formal Methods in System Design 34(3), 238–304 (2009)
Laplante, P.A.: Real-Time System Design and Analysis. John Wiley & Sons (2004)
Locke, C.D., Vogel, D.R., Lucas, L., Goodenough, J.B.: Generic avionics software specification. Software Engineering Institute, Carnegie Mellon University, Tech. rep. (1990)
Nielsen, B., Skou, A.: Test generation for time critical systems: tool and case study. In: Proceedings of the 13th Euromicro Conference on Real-Time Systems, pp. 155–162 (2001)
Peleska, J.: Industrial-strength model-based testing - state of the art and current challenges. In: Proceedings of the 8th Workshop on Model-Based Testing, pp. 3–28 (2013)
Pretschner, A., Slotosch, O., Aiglstorfer, E., Kriebel, S.: Model-based testing for real. International Journal on Software Tools for Technology Transfer 5(2), 140–157 (2004)
Rapps, S., Weyuker, E.J.: Selecting software test data using data flow information. IEEE Transactions on Software Engineering 11(4), 367–375 (1985)
Rusu, V., du Bousquet, L., Jéron, T.: An approach to symbolic test generation. In: Grieskamp, W., Santen, T., Stoddart, B. (eds.) IFM 2000. LNCS, vol. 1945, pp. 338–557. Springer, Heidelberg (2000)
Sommerville, I.: Software Engineering. International Computer Science Series, 9th edn. Addison-Wesley, Boston (2010)
Trab, M.S.A., Alrouh, B., Counsell, S., Hierons, R.M., Ghinea, G.: A multi-criteria decision making framework for real time model-based testing. In: Bottaci, L., Fraser, G. (eds.) TAIC PART 2010. LNCS, vol. 6303, pp. 194–197. Springer, Heidelberg (2010)
Tretmans, J.: Model-based testing and some steps towards test-based modelling. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 297–326. Springer, Heidelberg (2011)
Tretmans, J.: Testing concurrent systems: a formal approach. In: Baeten, J.C.M., Mauw, S. (eds.) CONCUR 1999. LNCS, vol. 1664, pp. 46–65. Springer, Heidelberg (1999)
Utting, M., Legeard, B.: Practical Model Based Testing: A Tools Approach. Elsevier, San Francisco (2007)
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer, New York (2012)
Zheng, M., Alagar, V., Ormandjieva, O.: Automated generation of test suites from formal specifications of real-time reactive systems. Journal of Systems and Software 81(2), 286–304 (2008)
Zhu, H., Hall, P.A.V., May, J.H.R.: Software unit test coverage and adequacy. ACM Computing Surveys 29(4), 366–427 (1997)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Almeida, D.R., Moraes, A., Andrade, W.L., Machado, P.D.L. (2015). Towards a Family of Test Selection Criteria for Symbolic Models of Real-Time Systems. In: Braga, C., Martí-Oliet, N. (eds) Formal Methods: Foundations and Applications. SBMF 2014. Lecture Notes in Computer Science(), vol 8941. Springer, Cham. https://doi.org/10.1007/978-3-319-15075-8_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-15075-8_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15074-1
Online ISBN: 978-3-319-15075-8
eBook Packages: Computer ScienceComputer Science (R0)