Using search-based techniques for testing executable software models specified through graph transformations

Abstract

Design by contract is a software development methodology that uses contracts for defining interfaces among interacting components of a software system. Graph transformation system is used to specify the behavioral aspects of software components by defining the pre- and post-conditions of methods as contracts. In this paper, we focus on testing executable software models specified by a graph transformation system. A set of model-specific coverage criteria and a cost-aware search-based test generation approach are introduced. To evaluate the effectiveness of the proposed coverage criteria and the test generation approach, a type of mutation analysis is presented at the model level. Furthermore, a couple of fault-detection methods are used to assess the quality of the generated tests in the model-level mutation analysis. The proposed approach is implemented in GROOVE, a toolset for model checking graph transformation systems. The empirical results based on some well-known case studies demonstrate the efficiency and scalability of each proposed coverage criterion and testing approach. The comparison of the proposed test generation approach with state-of-the-art techniques indicates a significant improvement in terms of fault-detection capability and testing costs.

Appendix

Algorithm 4 shows how coverage goals concerning the introduced data-dependencies in a given test case are detected in GROOVE toolset. It receives a test sequence as input and keeps track of data-flow along the execution path. Lines 1–3 of the algorithm initializes needed data structures including four empty sets to maintain detected coverage goals and put the SUC to the start state. Then each transformation step of the test sequence is applied to the system state and experienced data-flow, i.e. def-use, are recorded through the while loop (Lines 4–26). In each transformation step, the applied rule was registered for created objects (def step, Lines 5–8) to detect the source of the data object in the use step. Next, all data-dependencies experienced by the applied transformation step extracted as Create_Read (Lines 9–13), Create_Delete (Lines 14–18), Create_Update (Lines 19–23), and Delete_NAC (Lines 24 and 25). To specify the Delete_NAC relations, as shown in Algorithm 5, we should check for each transition t how the elements deleted in the previous transitions enable transition t through the provision of its NAC conditions.

Algorithm 6 shows how we calculate the number of covered test objectives by all test cases of a test suite as a fitness of the test suite. In this algorithm initially, we have four empty sets (CreatRead, CreateDelete, CreateUpdate, DeleteNAC) to save test goals covered by the test suite (Line 1). Then, data-dependencies covered by each test case are extracted by Algorithm 1 (Line 3) and added to the corresponding sets of the test suite (Lines 4–7). Finally, the fitness value is calculated as a sum of the size of all introduced sets which selected as coverage criteria (Line 8, in this algorithm all of the introduced data-flow coverage criteria are considered).

Algorithm 7 is designed for generating a random test path. This algorithm launches in the start state and initializes a new empty test path (Lines 1 and 2), then selects randomly an outgoing transition from the current state and goes to the next state through the selected transition (Lines 4–6). The number of the selected transition added to the encoded test path (Line 7).