Feature Location Benchmark for Software Families Using Eclipse Community Releases

  • Jabier MartinezEmail author
  • Tewfik Ziadi
  • Mike Papadakis
  • Tegawendé F. Bissyandé
  • Jacques Klein
  • Yves Le Traon
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9679)


It is common belief that high impact research in software reuse requires assessment in realistic, non-trivial, comparable, and reproducible settings. However, real software artefacts and common representations are usually unavailable. Also, establishing a representative ground truth is a challenging and debatable subject. Feature location in the context of software families is a research field that is becoming more mature with a high proliferation of techniques. We present EFLBench, a benchmark and a framework to provide a common ground for this field. EFLBench leverages the efforts made by the Eclipse Community which provides real feature-based family artefacts and their implementations. Eclipse is an active and non-trivial project and thus, it establishes an unbiased ground truth. EFLBench is publicly available and supports all tasks for feature location techniques integration, benchmark construction and benchmark usage. We demonstrate its usage and its simplicity and reproducibility by comparing four techniques.


Feature location Software product lines Benchmark Static analysis Information retrieval 



Supported by the National Research Fund Luxembourg (FNR), under the AFR grant 7898764.


  1. 1.
    Acher, M., Cleve, A., Collet, P., Merle, P., Duchien, L., Lahire, P.: Extraction and evolution of architectural variability models in plugin-based systems. Softw. Syst. Model. 13(4), 1367–1394 (2014)CrossRefGoogle Scholar
  2. 2.
    AL-Msie’deen, R.F., Seriai, A., Huchard, M., Urtado, C., Vauttier, S., Salman, H.E.: Feature location in a collection of software product variants using formal concept analysis. In: Favaro, J., Morisio, M. (eds.) ICSR 2013. LNCS, vol. 7925, pp. 302–307. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  3. 3.
    Apache: Opennlp (2010).
  4. 4.
    Apel, S., Batory, D.S., Kästner, C., Saake, G.: Feature-Oriented Software Product Lines - Concepts and Implementation. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  5. 5.
    Assunção, W.K.G., Vergilio, S.R.: Feature location for software product line migration: a mapping study. In: International Software Product Line Conference: Companion Volume for Workshop, Tools and Demo papers, SPLC, pp. 52–59 (2014)Google Scholar
  6. 6.
    Benavides, D., Segura, S., Cortés, A.R.: Automated analysis of feature models 20 years later: a literature review. Inf. Syst. 35(6), 615–636 (2010)CrossRefGoogle Scholar
  7. 7.
    Chen, K., Rajlich, V.: Case study of feature location using dependence graph, after 10 years. In: The 18th IEEE International Conference on Program Comprehension, ICPC 2010, Braga, Minho, Portugal, 30 June–2 July, pp. 1–3 (2010)Google Scholar
  8. 8.
    Couto, M.V., Valente, M.T., Figueiredo, E.: Extracting software product lines: a case study using conditional compilation. In: European Conference on Software Maintenance and Reengineering, CSMR 2011, pp. 191–200 (2011)Google Scholar
  9. 9.
    Dubinsky, Y., Rubin, J., Berger, T., Duszynski, S., Becker, M., Czarnecki, K.: An exploratory study of cloning in industrial software product lines. In: 17th European Conference on Software Maintenance and Reengineering, CSMR 2013, Genova, Italy, 5–8 March, pp. 25–34. IEEE Computer Society (2013)Google Scholar
  10. 10.
    Falleri, J.R., Dolques, X.: Erca - eclipse’s relational concept analysis (2010).
  11. 11.
    Fischer, S., Linsbauer, L., Lopez-Herrejon, R.E., Egyed, A.: Enhancing clone-and-own with systematic reuse for developing software variants. In: Proceedings of International Conference on Software Maintenance and Evolution (ICSME 2014), pp. 391–400 (2014)Google Scholar
  12. 12.
    Font, J., Ballarín, M., Haugen, O., Cetina, C.: Automating the variability formalization of a model family by means of common variability language. In: SPLC, pp. 411–418 (2015)Google Scholar
  13. 13.
    Ganter, B., Wille, R.: Formal Concept Analysis: Mathematical Foundations, 1st edn. Springer-Verlag New York Inc., Secaucus (1997)zbMATHGoogle Scholar
  14. 14.
    Grünbacher, P., Rabiser, R., Dhungana, D., Lehofer, M.: Model-based customization and deployment of eclipse-based tools: Industrial experiences. In: International Conference on Automated Software Engineering (ASE), pp. 247–256 (2009)Google Scholar
  15. 15.
    Kang, K.C., Cohen, S.G., Hess, J.A., Novak, W.E., Peterson, A.S.: Feature-oriented domain analysis (foda) feasibility study. Technical report, Carnegie-Mellon University Software Engineering Institute (1990)Google Scholar
  16. 16.
    Kästner, C., Apel, S., Kuhlemann, M.: Granularity in software product lines. In: Proceedings of the 30th International Conference on Software Engineering (ICSE), pp. 311–320 (2008)Google Scholar
  17. 17.
    Lopez-Herrejon, R.E., Batory, D.: A standard problem for evaluating product-line methodologies. In: Dannenberg, R.B. (ed.) GCSE 2001. LNCS, vol. 2186, pp. 10–24. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  18. 18.
    Lopez-Herrejon, R.E., Ferrer, J., Chicano, F., Haslinger, E.N., Egyed, A., Alba, E.: Towards a benchmark and a comparison framework for combinatorial interaction testing of software product lines. CoRR abs/1401.5367 (2014)Google Scholar
  19. 19.
    Lopez-Herrejon, R.E., Ziadi, T., Martinez, J., Thurimella, A.K., Acher, M.: Third international workshop on reverse variability engineering (REVE 2015). In: Proceedings of the 19th International Conference on Software Product Line, SPLC 2015, Nashville, TN, USA, 20–24 July, p. 394 (2015)Google Scholar
  20. 20.
    Martinez, J., Ziadi, T., Bissyandé, T.F., Klein, J., Traon, Y.L.: Automating the extraction of model-based software product lines from model variants. In: ASE 2015, Lincoln, Nebraska, USA (2015)Google Scholar
  21. 21.
    Martinez, J., Ziadi, T., Bissyandé, T.F., Klein, J., Traon, Y.L.: Bottom-up adoption of software product lines: a generic and extensible approach. In: Proceedings of International Conference on Software Product Line, SPLC 2015, pp. 101–110 (2015)Google Scholar
  22. 22.
    Porter, M.F.: Snowball: A language for stemming algorithms, Accessed 19 Nov 2015
  23. 23.
    Rubin, J., Chechik, M.: Locating distinguishing features using diff sets. In: IEEE/ACM International Conference on Automated Software Engineering, ASE 2012, Essen, Germany, 3–7 September, pp. 242–245 (2012)Google Scholar
  24. 24.
    Rubin, J., Chechik, M.: A survey of feature location techniques. In: Domain Engineering, Product Lines, Languages, and Conceptual Models, pp. 29–58 (2013)Google Scholar
  25. 25.
    Salman, H.E., Seriai, A., Dony, C.: Feature location in a collection of product variants: combining information retrieval and hierarchical clustering. In: International Conference on Software Engineering and Knowledge Engineering, SEKE, pp. 426–430 (2014)Google Scholar
  26. 26.
    Salman, H.E., Seriai, A., Dony, C.: Feature-to-code traceability in a collection of software variants: combining formal concept analysis and information retrieval. In: International Conference on Information Reuse and Integration, IRI, pp. 209–216 (2013)Google Scholar
  27. 27.
    Salton, G., Wong, A., Yang, C.S.: A vector space model for automatic indexing. Commun. ACM 18(11), 613–620 (1975)CrossRefzbMATHGoogle Scholar
  28. 28.
    Segura, S., Galindo, J.A., Benavides, D., Parejo, J.A., Cortés, A.R.: Betty: benchmarking and testing on the automated analysis of feature models. In: Proceedings of Sixth International Workshop on Variability Modelling of Software-Intensive Systems, Leipzig, Germany, 25–27 January, pp. 63–71 (2012)Google Scholar
  29. 29.
    Shatnawi, A., Seriai, A., Sahraoui, H.: Recovering architectural variability of a family of product variants. In: Schaefer, I., Stamelos, I. (eds.) ICSR 2015. LNCS, vol. 8919, pp. 17–33. Springer, Heidelberg (2014)Google Scholar
  30. 30.
    Sim, S.E., Easterbrook, S.M., Holt, R.C.: Using benchmarking to advance research: a challenge to software engineering. In: Proceedings of the 25th International Conference on Software Engineering, Portland, Oregon, USA, 3–10 May, pp. 74–83 (2003)Google Scholar
  31. 31.
    Souza, I.S., Fiaccone, R., de Oliveira, R.P., Almeida, E.S.D.: On the relationship between features granularity and non-conformities in software product lines: an exploratory study. In: 27th Brazilian Symposium on Software Engineering, SBES 2013, Brasilia, Brazil, 1–4 October, pp. 147–156 (2013)Google Scholar
  32. 32.
    Xing, Z., Xue, Y., Jarzabek, S.: A large scale linux-kernel based benchmark for feature location research. In: Proceedings of International Conference on Software Engineering, ICSE, pp. 1311–1314 (2013)Google Scholar
  33. 33.
    Xue, Y., Xing, Z., Jarzabek, S.: Feature location in a collection of product variants. In: Proc. of Working Conference on Reverse Engineering, WCRE 2012, pp. 145–154 (2012)Google Scholar
  34. 34.
    Ziadi, T., Henard, C., Papadakis, M., Ziane, M., Traon, Y.L.: Towards a language-independent approach for reverse-engineering of software product lines. In: Symposium on Applied Computing, SAC 2014, pp. 1064–1071 (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Jabier Martinez
    • 1
    • 2
    Email author
  • Tewfik Ziadi
    • 2
  • Mike Papadakis
    • 1
  • Tegawendé F. Bissyandé
    • 1
  • Jacques Klein
    • 1
  • Yves Le Traon
    • 1
  1. 1.SnTUniversity of LuxembourgLuxembourgLuxembourg
  2. 2.LiP6, Sorbonne Universités, UPMC Univ Paris 06ParisFrance

Personalised recommendations