A Coverage Relationship Model for Test Case Selection and Ranking for Multi-version Software

  • Wei-Tek Tsai
  • Xinyu Zhou
  • Raymond A. Paul
  • Yinong Chen
  • Xiaoying Bai


Testing a group of software artifacts that implement the same specification is time consuming, especially when the test case repository is large. In the meantime, some of test cases may cover the same aspects in the software under test, and thus it is not necessary to apply all the test cases. This paper proposes a Model-based Adaptive Test (MAT) case selection and ranking technique to eliminate redundant test cases, and rank the test cases according to their potency and coverage. This technique can be applied in various domains where multiple versions of an application are available for testing, such as web service group testing, n-version applications, regression testing, and specification-based application testing. MAT is a statistical model based on earlier testing results, and the model can accurately determine the next sets of test cases to minimize the testing effort. It can be applied to testing of multi-versioned web services, and the results shows that MAT can reduce testing effort while still maintain the effectiveness of testing.


Model Check Coverage Probability Coverage Relationship Test Case Selection Control Markov Chain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. Bar-Noy, F. Hwang, H. Kessler, and S. Kutten. A new competitive algorithm for group testing. Discrete Applied Mathematics, 52:29--38, July 1994.CrossRefMathSciNetGoogle Scholar
  2. [2]
    L. Baresi, and E. Di Nitto, Test and Analysis of Web Services, Springer, 1st edition, November, 2007Google Scholar
  3. [3]
    A. Bertolino, L. Frantzen, A. Polini, and J. Tretmans. Audition of web services for testing conformance to open specified protocols. In Architecting Systems with Trustworthy Components, No. 3938 in LNCS. Springer-Verlag, 2006.Google Scholar
  4. [4]
    P.A. Bonatti, P. Festa, “On Optimal Service Selection”, Proc. of the International World Wide Web Conference (WWW), 2005, pp.530-538.Google Scholar
  5. [5]
    Kai-Yuan Cai, Yong-Chao Li, Ke Liu, "Optimal and adaptive testing for software reliability assessment," Information & Software Technologies, volume 46, December 2004, pp. 9891000.Google Scholar
  6. [6]
    G. Canfora and M. Di Penta, “Testing Services and Service-centric Systems, Challenges and Opportunities,” IT Professional, vol. 8, no. 2, 2006, pp. 10-17.CrossRefGoogle Scholar
  7. [7]
    Liming Chen; A. Avizienis, N-Version Programming: a Fault-Tolerance Approach to Reliability of Software Operation, Twenty-Fifth International Symposium on Fault-Tolerant Computing, 1995, pp. 113-119.Google Scholar
  8. [8]
    B. De, “Web Services -Challenges and Solutions”, WIPRO white paper, 2003,
  9. [9]
    D. Z. Du and F. Hwang, Combinatorial Group Testing And Its Applications, World Scientific, 2nd edition, 2000Google Scholar
  10. [10]
    J. Garcia-Fanjul, J. Tuya, C. de la Riva. Generating test cases specifications for bpel compositions of web services using spin, International Workshop on Web Services Modeling and Testing (WSMaTe), 2006.Google Scholar
  11. [11]
    Heckel, R. and Mariani, L., Automatic conformance testing of Web Services, Proceedings of FASE 05: 34-48.Google Scholar
  12. [12]
    Andrew B. Kahng, Sherief Reda, Combinatorial group testing methods for the BIST diagnosis problem, Proceedings of the 2004 conference on Asia South Pacific design automation: electronic design and solution fair, pp. 113 – 116, Yokohama, Japan, 2004Google Scholar
  13. [13]
    J. Koehler, B. Srivastava, “Web Service Composition: Current Solutions and Open Problems”, ICAPS 2003 Workshop on Planning for Web Services, pp. 28-35.Google Scholar
  14. [14]
    N. Milanovic, M. Malek, “Current Solutions for Web Service Composition”, IEEE Internet Computing, Nov/Dec 2004, Volume: 8, Issue: 6. pp. 51-59.CrossRefGoogle Scholar
  15. [15]
    S. Narayanan and S. Mcllraith, “Simulation, verification and automated composition of web services”, In Proc. WWW, 2002.Google Scholar
  16. [16]
    Dmitri Nevedrov, Using JMeter to Performance Test Web Services,
  17. [17]
    OASIS: Business Process Execution Language for Web Services (BPEL4WS), 2003.
  18. [18]
  19. [19]
    William N. Robinson, "A requirements monitoring framework for enterprise systems," Requirements Engineering Journal, 11 (2006): 17-41.CrossRefGoogle Scholar
  20. [20]
    W.T Tsai, X. Bai, Y. Chen, X. Zhou, "Web Service Group Testing with Windowing Mechanisms," IEEE International Workshop on Service-Oriented System Engineering (SOSE), Beijing October 2005, 213-218.Google Scholar
  21. [21]
    W.T. Tsai, Y. Chen, R. Paul, H. Huang, X. Zhou, X. Wei, "Adaptive Testing, Oracle Generation, and Test Script Ranking for Web Services," 29th IEEE Annual International Computer Software and Applications Conference (COMPSAC), Edinburgh, July 2005, pp.101-106.Google Scholar
  22. [22]
    Web Services Policy 1.2 -Framework (WS-Policy) W3C Member Submission 25 April 2006, available at
  23. [23]
    J. A. Whittaker and M. G. Thomason. AMarkov chain model for statistical software testing. IEEE Trans. on Software Engineering,20(10):812–824, Oct. 1994.CrossRefGoogle Scholar
  24. [24]
    W3C, the World Wide Web Consortium,
  25. [25]
  26. [26]
    WSDL, available at
  27. [27]
    Pulei Xiong, Robert L. Probert, Bernard Stepien, An Efficient Formal Testing Approach for Web Service with TTCN-3 , In Proceedings of the 13th International Conference on Software, Telecommunications and Computer Networks (SoftCOM 2005), Split, Croatia, 2005.Google Scholar
  28. [28]
    Yongyan Zheng, Jiong Zhou, P. Krause, A model checking based test case generation framework for Web services, 4th International Conference on Information Technology New Generations, 2007.Google Scholar
  29. [29]
    Hong Zhu, A Framework for Service-Oriented Testing of Web Services, 30th Annual International Computer Software and Applications Conference, 2006.Google Scholar

Copyright information

© Springer-Verlag US 2009

Authors and Affiliations

  • Wei-Tek Tsai
    • 1
  • Xinyu Zhou
    • 1
  • Raymond A. Paul
    • 2
  • Yinong Chen
    • 1
  • Xiaoying Bai
    • 3
  1. 1.Computer Science & Engineering DepartmentArizona State UniversityUSA
  2. 2.Department of DefenseWashingtonUSA
  3. 3.Department of Computer Science and EngineeringTsinghua UniversityChina

Personalised recommendations