Symbolic Test Selection Based on Approximate Analysis
This paper addresses the problem of generating symbolic test cases for testing the conformance of a black-box implementation with respect to a specification, in the context of reactive systems. The challenge we consider is the selection of test cases according to a test purpose, which is here a set of scenarios of interest that one wants to observe during test execution. Because of the interactions that occur between the test case and the implementation, test execution can be seen as a game involving two players, in which the test case attempts to satisfy the test purpose.
Efficient solutions to this problem have been proposed in the context of finite-state models, based on the use of fixpoint computations. We extend them in the context of infinite-state symbolic models, by showing how approximate fixpoint computations can be used in a conservative way. The second contribution we provide is the formalization of a quality criterium for test cases, and a result relating the quality of a generated test case to the approximations used in the selection algorithm.
KeywordsTest Generation Test Purpose Test Selection Label Transition System Approximate Analysis
- 1.ISO/IEC 9646. Conformance Testing Methodology and Framework (1992)Google Scholar
- 2.Belinfante, A., Feenstra, J., de Vries, R.G., Tretmans, J., Goga, N., Feijs, L., Mauw, S., Heerink, L.: Formal test automation: A simple experiment. In: 12th Int. Workshop on Testing of Communicating Systems. Kluwer Academic Publishers, Dordrecht (1999)Google Scholar
- 3.Benjamin, M., Geist, D., Hartman, A., Mas, G., Smeets, R., Wolfsthal, Y.: A feasibility study in formal coverage driven test generation. In: DAC 1999 (1999)Google Scholar
- 4.Cousot, P., Cousot, R.: Abstract intrepretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: 4th ACM Symposium on Principles of Programming Languages, Los Angeles, CA (1977)Google Scholar
- 6.Grabowski, J., Hogrefe, D., Nahm, R.: Test Case Generation with Test Purpose Specification by MSCs. In: 6th SDL Forum, Darmstadt (Germany). North-Holland, Amsterdam (1993)Google Scholar
- 7.Halbwachs, N., Proy, Y.E., Roumanoff, P.: Verification of real-time systems using linear relation analysis. Formal Methods in System Design 11(2) (1997)Google Scholar
- 9.Jard, C., Jéron, T.: TGV: theory, principles and algorithms. International Journal on Software Tools for Technology Transfer 6 (2004)Google Scholar
- 10.Jeannet, B.: Dynamic partitioning in linear relation analysis. application to the verification of reactive systems. Formal Methods in System Design 23(1) (2003)Google Scholar
- 11.Jeannet, B., Jéron, T., Rusu, V., Zinovieva, E.: Symbolic test selection using approximate analysis. Technical Report 1649, INRIA (2004), Available at http://www.irisa.fr/bibli/publi/pi/2004/1649/1649.html
- 13.Lee, D., Yannakakis, M.: Principles and Methods of Testing Finite State Machines - A Survey. Proceedings of the IEEE 84(8) (1996)Google Scholar
- 15.Rusu, V., Marchand, H., Jéron, T.: Verification and symbolic test generation for safety properties. Technical Report 5285, INRIA (2004), Available at http://www.inria.fr/rrrt/rr-5285.html
- 16.Tretmans, J.: Test generation with inputs, outputs and repetitive quiescence. Software—Concepts and Tools 17(3) (1996)Google Scholar