Managing Execution Environment Variability during Software Testing: An Industrial Experience

  • Aymeric Hervieu
  • Benoit Baudry
  • Arnaud Gotlieb
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7641)


Nowadays, telecom software applications are expected to run on a tremendous variety of execution environments. For example, network connection software must deliver the same functionalities on distinct physical platforms, which themselves run several distinct operating systems, with various applications and physical devices. Testing those applications is challenging as it is simply impossible to consider every possible environment configuration. This paper reports on an industrial case study called BIEW (Business and Internet EveryWhere) where the combinatorial explosion of environment configurations has been tackled with a dedicated and original methodology devised by KEREVAL, a french SME focusing on software testing services. The proposed solution samples a subset of configurations to be tested, based on environment modelling, requirement analysis and systematic traceability. From the experience on this case study, we outline the challenges to develop means to select relevant environment configurations from variability modelling and requirement analysis in the testing processes of telecom software.


Environmental Feature Functional Domain Software Test Software Product Line Physical Device 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Olsen, P., Foederer, J., Tretmans, J.: Model-Based Testing of Industrial Transformational Systems. In: Wolff, B., Zaïdi, F. (eds.) ICTSS 2011. LNCS, vol. 7019, pp. 131–145. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  2. 2.
    Kang, K., Cohen, S., Hess, J., Novak, W., Peterson, S.: Feature-Oriented Domain Analysis (FODA) Feasibility Study, Software Engineering Institute, Tech. Rep. CMU/SEI-90-TR-21 (November 1990)Google Scholar
  3. 3.
    Benavides, D., Segura, S., Ruiz-Cortés, A.: Automated analysis of feature models: A detailed literature review. Information Systems (35), 615–636 (2010)Google Scholar
  4. 4.
    Hartmann, H., Trew, T.: Using feature diagrams with context variability to model multiple product lines for software supply chains. In: Proceedings of the 2008 12th International Software Product Line Conference (2008)Google Scholar
  5. 5.
    Disambiguating the Documentation of Variability in Software Product Lines: A Separation of Concerns, Formalization and Automated Analysis (2007)Google Scholar
  6. 6.
    Than Tun, T., Boucher, Q., Classen, A., Hubaux, A., Heymans, P.: Relating requirements and feature configurations: a systematic approach. In: Proceedings of the 13th International Software Product Line ConferenceTha (2009)Google Scholar
  7. 7.
    Cohen, D.M., Dalal, S.R., Fredman, M.L., Patton, G.C.: The aetg system: An approach to testing based on combinatorial design. IEEE Transactions on Software Engineering 23(7), 437–444 (1997)CrossRefGoogle Scholar
  8. 8.
    Lei, Y., Tai, K.-C.: In-parameter-order: A test generation strategy for pairwise testing. In: HASE 1998, pp. 254–261 (1998)Google Scholar
  9. 9.
    Oster, S., Markert, F., Ritter, P.: Automated Incremental Pairwise Testing of Software Product Lines. In: Bosch, J., Lee, J. (eds.) SPLC 2010. LNCS, vol. 6287, pp. 196–210. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
    Perrouin, G., Sen, S., Klein, J., Baudry, B., Le Traon, Y.: Automated and scalable t-wise test case generation strategies for software product lines. In: ICST 2010, Paris, France (2010)Google Scholar
  11. 11.
    Lamancha, B.P., Usaola, M.P.: Testing Product Generation in Software Product Lines Using Pairwise for Features Coverage. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 111–125. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  12. 12.
    Johansen, M.F., Haugen, Ø., Fleurey, F.: Properties of Realistic Feature Models Make Combinatorial Testing of Product Lines Feasible. In: Whittle, J., Clark, T., Kühne, T. (eds.) MODELS 2011. LNCS, vol. 6981, pp. 638–652. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  13. 13.
    Hervieu, A., Baudry, B., Gotlieb, A.: Pacogen: Automatic generation of pairwise test configurations from feature models. In: Proc. of the 22nd IEEE Int. Symp. on Softw. Reliability Engineering (ISSRE 2011), Hiroshima, Japan (November 2011)Google Scholar
  14. 14.
    Garvin, B.J., Cohen, M.B., Dwyer, M.B.: An improved meta-heuristic search for constrained interaction testing. In: Proceedings of the 2009 1st International Symposium on Search Based Software Engineering, SSBSE 2009, pp. 13–22. IEEE Computer Society, Washington, DC (2009), Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2012

Authors and Affiliations

  • Aymeric Hervieu
    • 1
    • 2
  • Benoit Baudry
    • 2
  • Arnaud Gotlieb
    • 3
  1. 1.KEREVALThorigné FouillardFrance
  2. 2.INRIA Rennes Bretagne AtlantiqueRennesFrance
  3. 3.Certus Software V&V CenterSIMULA RESEARCH LAB.LysakerNorway

Personalised recommendations