Towards a Comprehensive Approach for Assessing Open Source Projects

  • Marcus Ciolkowski
  • Martín Soto
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5338)


Open Source Software (OSS) has an increasing importance for the software industry. Similar to traditional (closed) software acquisition, OSS acquisition requires an assessment of whether quality is sufficient for the intended purpose. This includes assessing a software component’s intrinsic quality, as well as its supplier’s maturity (i.e., ability to deliver high quality) and sustainability (i.e., whether the supplier will continue to exist). For traditional software acquisition, established procedures are available for evaluating these aspects. These procedures need to be adapted for OSS projects, because they have no traditional supplier, but an underlying OSS community. The openness of OSS development presents both challenges and opportunities for project evaluation. In particular, a variety of data sources are available that potentially allow for in-depth analysis, but it is not clear how to use them effectively.

In this paper, we present an approach toward a comprehensive measurement framework for OSS projects, developed in the EU project QualOSS. This approach takes into account product quality as well as process maturity and sustainability of the underlying OSS community.


Open Source quality process assessment process maturity 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Software Engineering Institute (SEI): Capability Maturity Model Integration (CMMI) for Development, Version 1.2 (2006)Google Scholar
  2. 2.
    ISO/IEC 15504-5:2006, Software Process Improvement and Capability Determination, Part 5Google Scholar
  3. 3.
    Michlmayr, M.: Software Process Maturity and the Success of Free Software Projects. In: Zieliński, K., Szmuc, T. (eds.) Software Engineering: Evolution and Emerging TechnologiesGoogle Scholar
  4. 4.
    International Society for Pharmaceutical Engineering (ISPE): Good Automated Manufacturing Practice (GAMP-4) Supplier Guide for Validation of Automated Systems in Pharmaceutical Manufacture (1995)Google Scholar
  5. 5.
    ISO/IEC 61508:1998, Functional safety of electrical/electronic/programmable electronic safety-related systemsGoogle Scholar
  6. 6.
    Software Engineering Institute (SEI): CMMI for acquisition, Technical Report DMU/SEI-2007-TR-017 (2007)Google Scholar
  7. 7.
    Method for Qualification and Selection of Open Source software (QSOS) version 1.6 ©, Atos Origin (April 2006),
  8. 8., Business Readiness Rating for Open Source©: A Proposed Open Standard to Facilitate Assessment and Adoption of Open Source Software, BRR (2005),
  9. 9.
    Golden, B.: Open Source Maturity Model © Navica,
  10. 10.
    Duijnhouwer, F., Widdows, C.: Open Source Maturity Model, Capgemini Expert Letter (August. 2003),
  11. 11.
    Deprez, J.C., Alexandre, S.: Comparing Assessment Methodologies for Free/Open Source Software: OpenBRR and QSOS. In: Jedlitschka, A., Salo, O. (eds.) PROFES 2008. LNCS, vol. 5089, pp. 189–203. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  12. 12.
    Ciolkowski, M., Guzmán, L., Soto, M., Kamseu, F.: Validation and Calibration of User Manual, QualOSS Deliverable D1.6 (2008)Google Scholar
  13. 13.
    Ciolkowski, M., Soto, M., Monfils, F.F., García, C., Izquierdo, D., Kamseu, F., del Castillo, A.: Calibration of the Prototype QualOSS Model, QualOSS Deliverable D1.5 (2007)Google Scholar
  14. 14.
    Ciolkowski, M., Soto, M., Deprez, J.-C., Monfils, F.F., Kamseu, F., Ruiz, J., del Castillo, A., Izquierdo, D.: Metrics System and Prototype QualOSS Models. QualOSS Deliverable D1.3 (2007)Google Scholar
  15. 15.
    ISO/IEC 9126 International Standard, Software engineering – Product quality, Part 1: Quality model (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Marcus Ciolkowski
    • 1
  • Martín Soto
    • 1
  1. 1.Fraunhofer Institute for Experimental Software EngineeringKaiserslauternGermany

Personalised recommendations