Abstract
Automated test case selection for a new product in a product line is challenging due to several reasons. First, the variability within the product line needs to be captured in a systematic way; second, the reusable test cases from the repository are required to be identified for testing a new product. The objective of such automated process is to reduce the overall effort for selection (e.g., selection time), while achieving an acceptable level of the coverage of testing functionalities. In this paper, we propose a systematic and automated methodology using a Feature Model for Testing (FM_T) to capture commonalities and variabilities of a product line and a Component Family Model for Testing (CFM_T) to capture the overall structure of test cases in the repository. With our methodology, a test engineer does not need to manually go through the repository to select a relevant set of test cases for a new product. Instead, a test engineer only needs to select a set of relevant features using FM_T at a higher level of abstraction for a product and a set of relevant test cases will be selected automatically. We applied our methodology to a product line of video conferencing systems called Saturn developed by Cisco and the results show that our methodology can reduce the selection effort significantly. Moreover, we conducted a questionnaire-based study to solicit the views of test engineers who were involved in developing FM_T and CFM_T. The results show that test engineers are positive about adapting our methodology and models (FM_T and CFM_T) in their current practice.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Benavides, D., Segura, S., Cortés, A.R.: Automated analysis of feature models 20 years later. A literature review. Information Systems (35), 615–636 (2010)
Czarnecki, K., Kim, C., Kalleberg, K.: Feature models are views on ontologies. In: Proceedings of the International Software Product Line Conference, pp. 41–51 (2006)
Ali, S., Yue, T., Briand, L.C., Walawege, S.: A product line modeling and configuration methodology to support model-based testing: an industrial case study. In: Proceedings of the ACM International Conference on Model Driven Engineering Languages and Systems (MODELS), pp. 726–742 (2012)
Wang, S., Ali, S., Tao, Y.: Product Line Modeling and Configuration Methodology using Feature Model for Supporting Model-Based Testing. Simula Research Laboratory. Technical Report 2012-24 (2013)
McGregor, J.: Testing a Software Product Line. Technical Report. CMU/SEI-2001-TR-022. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania (2001)
Engström, E.: Regression Test Selection and Product Line System Testing. In: Proceedings of Third International Conference on Software Testing, Verification and Validation (ICST), pp. 512–515 (2010)
Engström, E., Runeson, P., Skoglund, M.: A systematic review on regression test selection techniques. Information and Software Technology (IST) 52(1), 14–30 (2010)
Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Software: Testing, Verification and Reliability 22(2), 67–120 (2012)
Cisco Systems: Cisco telepresence codec c90, Data sheet (2010), http://www.cisco.com
Wang, S., Gotlieb, A., Liaaen, M., Briand, L.C.: Automatic selection of test execution plans from a Video Conference System Product Line. In: Proceedings of the ACM MODELS Workshop VARiability for You (VARY 2012), pp. 32–37 (2012)
Beuche, D., Papajewski, H., Schröder-Preikschat, W.: Variability management with feature models. Science of Computer Programming 53(3), 333–352 (2004)
Pure systems GmbH: Variant management with pure:variants. Technical white paper (2006), http://web.pure-systems.com
Pure systems GmbH: Pure:Variants User’s Guide (2011), http://web.pure-systems.com
Wang, S., Ali, S., Gotlieb, A.: Minimizing Test Suites in Software Product Lines Using Weighted-based Genetic Algorithms. Simula Research Laboratory. Technical Report 2012-25 (2013)
Wohlin, C., Runeson, P., Host, M., Ohlsson, M.C., Regnell, B., Wesslen, A.: Experimentation in Software Engineering. Springer (2012)
Muccini, H., Van Der Hoek, A.: Towards Testing Product Line Architectures. Electronic Notes in Theoretical Computer Science 82(6), 99–109 (2003)
Uzuncaova, E., Garcia, D., Khurshid, S., Batory, D.: Testing software product lines using incremental test generations. In: Proceedings of the IEEE International Symposium on Software Reliability Engineering (ISSRE), pp. 249–258 (2008)
Nebut, C., Le Traon, Y., Jézéquel, J.M.: System Testing of Product Lines: From Requirements to Test Cases. Software Product Lines. In: Research Issues in Engineering and Management, pp. 447–477. Springer (2006)
Chen, Y.F., Rosenblum, D.S., Vo, K.P.: Test tube: a system for selective regression testing. In: Proceedings of IEEE International Conference on Software Engineering (ICSE), Los Alamitos, CA, USA, pp. 211–220 (1994)
Hartmann, J., Robson, D.J.: Techniques for selective revalidation. IEEE Software 7(1), 31–36 (1990)
Harrold, M.J., Souffa, M.L.: An incremental approach to unit testing during maintenance. In: Proceedings of IEEE International Conference on Software Maintenance (ICSM), pp. 362–367 (1988)
Orso, A., Harrold, M.J., Rosenblum, D., Rothermel, G., Soffa, M.L., Do, H.: Using component metacontent to support the regression testing of component-based software. In: Proceedings of IEEE International Conference on Software Maintenance (ICSM), pp. 716–725 (2001)
Chen, Y., Probert, R.L., Sims, D.P.: Specification-based regression test selection with risk analysis. In: Proceedings of Conference of the Centre for Advanced Studies on Collaborative Research. IBM Press (2002)
Bible, J., Rothermel, G., Rosenblum, D.S.: A comparative study of coarse- and fine- grained safe regression test-selection techniques. ACM Transactions on Software Engineering and Methodology 10(2), 149–183 (2001)
Graves, T.L., Harrold, M.J., Kim, J.M., Porter, A., Rothermel, G.: An empirical study of regression test selection techniques. ACM Transactions on Software Engineering and Methodology 10(2), 184–208 (2001)
Mansour, N., Bahsoon, R., Baradhi, G.: Empirical comparison of regression test selection algorithms. The Journal of Systems and Software 57(1), 79–90 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, S., Gotlieb, A., Ali, S., Liaaen, M. (2013). Automated Test Case Selection Using Feature Model: An Industrial Case Study. In: Moreira, A., Schätz, B., Gray, J., Vallecillo, A., Clarke, P. (eds) Model-Driven Engineering Languages and Systems. MODELS 2013. Lecture Notes in Computer Science, vol 8107. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41533-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-41533-3_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41532-6
Online ISBN: 978-3-642-41533-3
eBook Packages: Computer ScienceComputer Science (R0)