Selection of Projects in Software Process Assessment: New Perspectives

  • Alberto Sampaio
  • Edwin Gray
  • Helena Moreira
  • Mário Martins


Since the starting point of successful software process improvement begins with a formal software process assessment, it is essential that this assessment is effective and reliable. However, recently, critics and proponents have engaged in a debate on the reliability of such assessments. Until more scientific assessment methods are ready, how can the current kind of assessments be made more reliable? This paper argues that it is possible to reduce variability of software process assessments and improve their reliability. An assessment is an intensive human process of decision taking, subject to different interpretations by the evaluators. To overcome this, automation and quantitative data that could help in decision taking are needed. This paper discusses the need for widespread participation by everybody from the organisational unit (the part of the organisation being assessed), its implications for project selection and proposes the use of an automatic method for a better project selection. Better selection of projects for the visit, and better information about the organisation, could imply a visit in less time and with fewer costs.


Software Process Organisational Unit Process Instance Project Selection Decision Taking 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aldenderfer, M.S., Blashfield, R.K., 1984, Cluster analysis, in: Sage University Paper, Sage Publications.Google Scholar
  2. Bach, J., 1994, The immaturity of the CMM, American Programmer 7:9, 13–18.Google Scholar
  3. Bollinger, T.B., and McGowan, C., 1991, A critical look at software capability evaluations, IEEE Software 8:4,25–41.CrossRefGoogle Scholar
  4. Bouguettaya, A., 1996, On-Line clustering, IEEE Transactions on Knowledge and Data Engineering 8:2, 333–339.CrossRefGoogle Scholar
  5. Craigmyle, M., 1998, Process assessment using SPICE: the ratings framework, in: SPICE: The Theory and Practice of Software Process Improvement and Capability Determination, K. El Emam, J. Drouin, and W. Melo, eds., IEEE CS Press.Google Scholar
  6. Daskalantonakis, M. K., 1994, Achieving higher SEI levels, IEEE Software, 11:4, 17–24CrossRefGoogle Scholar
  7. Demirors, O., and Demirors, E., 1998, Software process improvement in a small organisation: difficulties and suggestions, in: Software Process Technology, V. Gruhn, ed., Springer Verlag, Lecture Notes in Com-puter Science, Vol. 1487.Google Scholar
  8. Diaz, M and Sligo, J, 1997, How software process improvement helped Motorola, IEEE Software, 14:5, 75–81.CrossRefGoogle Scholar
  9. Diday, E., Lemaire, J., Pouget, J., and Testu, F., 1985, Eléments d’ Analyse de Données, Dunod.Google Scholar
  10. Dion., R, 1993, Process improvement and the corporate balance sheet, IEEE Sofhvare, July, pp. 28–35.Google Scholar
  11. Dunaway, D. K., and Masters, S., 1996, CMM-Based Appraisal for Internal Process Improvement (CBA-IPI)VIA: Method Description, Software Engineering Institute, CMU/SEI-96-TR-007.Google Scholar
  12. El Emam[l]
    El Emam, K., Simon, J., Rouseau, S., and Jacquet, E., 1998, Cost implications of interrater agreement for software process assessments, in Proceedings of the Fifth Int. Software Metrics Sympo-sium, IEEE CS Press.Google Scholar
  13. El Emam[2]
    El Emam, K., Drouin, J., and Melo, W., 1998, SPICE: The Theory and Practice of Software Process Improvement and Capability Determination, K. El Emam, K., J. Drouin, and W. Melo, eds.,IEEE CS Press.Google Scholar
  14. El Emam[3]
    El Emam, K.. and Madhavji, N., 1995, The reliability of measuring organisational maturity,Software Process-Improvement and Practice, 1:1, pp. 3–25.Google Scholar
  15. El Emam [4]
    El Emam, K., 1998, The internal consistency of the ISO/IEC 15504 software process capabilityscale, in: Proceedings of the Fifth Int. Software Metrics Symposium, IEEE CS.Google Scholar
  16. El Emam [5]
    El Emam, K., and Marshall, P., 1998, Interrater agreement in assessment ratings, in: SPICE: The Theory and Practice of Software Process Improvement and Capability Determination, K. El Emam, K.,J. Drouin, and W. Melo, eds., IEEE CS Press.Google Scholar
  17. Fusaro, P., El Eman, K., Smith, B., 1997, The Internal Consistencies of the 1987 SEI Maturity Questionnaire and the SPICE Capability Dimension, Technical Report, International Software Engineering Research Network, ISER-97-01.Google Scholar
  18. Goldenson, D. R. and Herbsleb, J. D., 1995 After the Appraisal: A Systematic Survey of Process Improvement,Its Benefits, and Factors that Influence Success, Technical Report CMU/SEI-95-TR-009.Google Scholar
  19. Gray, E.M. and Smith, W.L., 1998, On the limitations of software process assessment and the recognition of a required re-orientation for global process improvement, Software Quality Journal,7.Google Scholar
  20. Hayes. W and Zubrow, D, 1995, Moving On Up: Data and Experience Doing CMM-Based Process Improvement, Technical Report CMU/SEI-95-TR-008.Google Scholar
  21. Herbsleb, J., et al., 1994, Software process improvement: state of the payoff, American Programmer, Septem-ber.Google Scholar
  22. Hetzel, B., 1995, The sorry state of software practice measurement and evaluation, in: Software Quality Assur-ance and Measurement: A Worldwide Perspective, N Fenton, R Whitty, Y. Iizuka, eds., Thomson Comp.Press.Google Scholar
  23. Hirscheim, R., Newman, M., 1985, Information systems and user resistance: theory and practice, Comp. Jour-nal, 31:5.Google Scholar
  24. Hsia, P., Hsu, C. T., Kung, D. C., and Holder, L. B., 1996, User-centered system decomposition: Z-based requirements clustering, in: Proceedings of the International Conference on Requirements Engineering,IEEE Press.Google Scholar
  25. Humphrey, W.S., and Curtis, B., 1991, Comments on ‘a critical look’, IEEE Software, 8:4, July.Google Scholar
  26. Humphrey, W.S., Snyder T.R., and Willis R.R., 1991 Software process improvement at Hughes aircraft,IEEE Software, July .pp 11–23.Google Scholar
  27. ISO/IEC, 1998, Information technology — Software process assessment — Part 7: Guide for use in process improvement, ISO/IEC TR 15504.Google Scholar
  28. Iversen, J. J., Nielsen, P. A., and Heje, J. P., 1998, Combining quantitative and qualitative assessment methods in software process improvement, in: Proceedings of 6th European Conference on Information Systems (ECIS), Aix-en-Provence, France, pp. 451–466Google Scholar
  29. Jones, C, 2001, Measuring software process improvement, in: Software Process Improvement, R. Hunter, and R. Thayer, eds., IEEE Computer Society Press, Under review.Google Scholar
  30. Lawlis, P.K., Flowe, R.M., and Thordahl, J.B., 1995, A Correlational Study of the CMM and Software Devel-opment Performance, Crosstalk, STSC, Hill Air Force Base, Utah, pp. 21–25.Google Scholar
  31. Tou, J., and Gonzalez, R., 1974, Pattern Recognition Principles, Addison-Wesley.MATHGoogle Scholar
  32. Wohlwend, H., and Rosenbaum, S.,1994, Schlumberger’s software improvement program, IEEE Transactions on Software Engineering, 11, pp. 833–839.CrossRefGoogle Scholar
  33. Woodman, I., and Hunter, R., 1998, Analysis of assessment ratings from the trials, in: SPICE: The Theory and Practice of Software Process Improvement and Capability Determination, K. El Emam, K., J. Drouin,and W. Melo, eds., IEEE CS Press.Google Scholar
  34. Yamamura, G., and Wigle, G. B., 1997, SEI CMM Level 5: For the Right Reasons, Crosstalk, STSC, Hill Air Force Base, Utah, pp. 3–6.Google Scholar
  35. Zubrow, D., Hayes, W., Siegel, J., and Goldenson, D., 1994, Maturity Questionnaire, Software Engineering Institute, CMU/SEI-94-SR-007.Google Scholar

Copyright information

© Springer Science+Business Media New York 2002

Authors and Affiliations

  • Alberto Sampaio
    • 1
  • Edwin Gray
    • 2
  • Helena Moreira
  • Mário Martins
    • 3
  1. 1.Departamento de Engenharia InformáticaInstituto Superior de Engenharia do PortoPortoPortugal
  2. 2.Department of ComputingGlasgow Caledonian UniversityGlasgowUK
  3. 3.Departamento de InfomáticaUniversidade do MinhoBragaPortugal

Personalised recommendations