Assessment of Software Process and Metrics to Support Quantitative Understanding: Experience from an Undefined Task Management Process

  • Ayca Tarhan
  • Onur Demirors
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 155)


Software engineering management demands the measurement, evaluation and improvement of the software processes and products. However, the utilization of measurement and analysis in software engineering is not very straightforward. It requires knowledge on the concepts of measurement, process management, and statistics as well as on their practical applications. We developed a systematic approach to evaluate the suitability of a software process and its measures for quantitative analysis, and have applied the approach in several industrial contexts. This paper explains the experience of evaluating a task management process and related measures of a government research agency. The agency had not defined the task management and measurement processes, and the performance data were gathered from a change management tool. We spent six person-days performing the assessment and analyzing data from 92 process executions. We observed that as systematic approaches have become available, software organizations are able to readily apply quantitative techniques.


Software measurement quantitative analysis control chart 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Weller, E.: Practical Applications of Statistical Process Control. IEEE Software 17(3), 48–55 (2000)CrossRefGoogle Scholar
  2. 2.
    Florac, A.W., Carleton, A.D.: Statistical Process Control: Analyzing a Space Shuttle Onboard Software Process. IEEE Software 17(4), 97–106 (2000)CrossRefGoogle Scholar
  3. 3.
    Baldassarre, M.T., Boffoli, N., Caivano, D., Visaggio, G.: Managing Software Process Improvement (SPI) through Statistical Process Control (SPC). In: Bomarius, F., Iida, H. (eds.) PROFES 2004. LNCS, vol. 3009, pp. 30–46. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Demirors, O., Sargut, K.U.: Utilization of Statistical Process Control (SPC) in Emergent Software Organizations: Pitfalls and Suggestions. Software Quality Journal 14(2), 135–157 (2006)CrossRefGoogle Scholar
  5. 5.
    Card, D.N., Domzalski, K., Davies, G.: Making Statistics Part of Decision Making in an Engineering Organization. IEEE Software 25(3), 37–47 (2008)CrossRefGoogle Scholar
  6. 6.
    Jalote, P., Dinesh, K., Raghavan, S., Bhashyam, M.R., Ramakrishnan, M.: Quantitative Quality Management through Defect Prediction and Statistical Process Control. In: 2nd World Quality Congress for Software, Yokohama, Japan (2000)Google Scholar
  7. 7.
    Pitterman, B.: Telcordia Technologies: The Journey to High Maturity. IEEE Sw. 17(4), 89–96 (2000)Google Scholar
  8. 8.
    McGarry, F., Decker, B.: Attaining Level 5 in CMM Process Maturity. IEEE Sw. 19(6), 87–96 (2002)Google Scholar
  9. 9.
    Jacob, L., Pillai, S.K.: Statistical Process Control to Improve Coding and Code Review. IEEE Software 20(3), 50–55 (2003)CrossRefGoogle Scholar
  10. 10.
    Agrawal, M., Chari, K.: Software Effort, Quality, and Cycle Time: A Study of CMM Level 5 Projects. IEEE Transactions on Software Engineering 33(3), 145–156 (2007)CrossRefGoogle Scholar
  11. 11.
    CMU/SEI-CMMI Product Team. CMMI for Development V1.3. CMU/SEI-2010-TR-033 (2010)Google Scholar
  12. 12.
    ISO/IEC. ISO/IEC 15504 Information Technology – Process Assessment, parts 1-7 (2003-2008)Google Scholar
  13. 13.
    Florac, A.W., Carleton, A.D.: Measuring the Software Process: Statistical Process Control for Software Process Improvement. Pearson Education, London (1999)Google Scholar
  14. 14.
    ISO/IEC. ISO/IEC 15939 Software Engineering – Software Measurement Process (2002)Google Scholar
  15. 15.
    McGarry, J., Card, D., Jones, C., Layman, B., Clark, E., Dean, J., Hall, F.: Practical Software Measurement: Objective Information for Decision Makers, 1st edn. Addison-Wesley, Reading (2001)Google Scholar
  16. 16.
    Dyba, T., Kampenes, V.B., Sjoberg, D.I.K.: A systematic review of statistical power in software engineering experiments. Information and Software Technology 48(8), 745–755 (2006)CrossRefGoogle Scholar
  17. 17.
    Chang, C., Chu, C.: Improvement of causal analysis using multivariate statistical process control. Software Quality Control 16(3), 377–409 (2008)CrossRefGoogle Scholar
  18. 18.
    Jalote, P., Saxena, A.: Optimum Control Limits for Employing Statistical Process Control in Software Process. IEEE Transactions on Software Engineering 28(12), 1126–1134 (2002)CrossRefGoogle Scholar
  19. 19.
    Lantzy, M.A.: Application of Statistical Process Control to Software Processes. In: 9th Washington Ada Symposium on Empowering Software Users and Developers, pp. 113–123 (1992)Google Scholar
  20. 20.
    Card, D.: Statistical Process Control for Software? IEEE Software 11(3), 95–97 (1994)CrossRefGoogle Scholar
  21. 21.
    Radice, R.: Statistical Process Control for Software Projects. In: 10th Software Engineering Process Group Conference, Chicago, Illinois (1998)Google Scholar
  22. 22.
    Paulk, M.C., Goldenson, D., White, D.M.: The 1999 Survey of High Maturity Organizations. Special Report, CMU/SEI-2000-SR-002 (2000)Google Scholar
  23. 23.
    Paulk, M.C., Chrissis, M.B.: The November 1999 High Maturity Workshop. Special Report, CMU/SEI-2000-SR-003 (2000)Google Scholar
  24. 24.
    Paulk, M.C., Chrissis, M.B.: The 2001 High Maturity Workshop. Special Report. CMU/SEI-2001-SR-014 (2000)Google Scholar
  25. 25.
    Kasunic, M.: The State of Software Measurement Practice: Results of 2006 Survey. Technical Report, CMU/SEI-2006-TR-009 (2006)Google Scholar
  26. 26.
    Stoddard II, R.W., Goldenson, D.R., Zubrow, D., Harper, E.: CMMI High Maturity Measurement and Analysis Workshop Report: March 2008. Technical Note, CMU/SEI-2008-TN-027 (2008)Google Scholar
  27. 27.
    Tarhan, A., Demirors, O.: Investigating the Effect of Variations in Test Development Process: A Case from a Safety-Critical System. Software Quality Journal (in press)Google Scholar
  28. 28.
    Tarhan, A., Demirors, O.: Assessment of Software Process and Metrics for Quantitative Understanding. In: Cuadrado-Gallego, J.J., Braungarten, R., Dumke, R.R., Abran, A. (eds.) IWSM-Mensura 2007. LNCS, vol. 4895, pp. 102–113. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  29. 29.
    Kirbas, S., Tarhan, A., Demirors, O.: An Assessment and Analysis Tool for Statistical Process Control of Software Processes. In: SPICE 2007 Conference, Seoul, Korea (2007)Google Scholar
  30. 30.
    Wheeler, D.J.: Advanced Topics in Statistical Process Control. SPC Press (1995)Google Scholar
  31. 31.
    ISO. ISO 9001, Quality Management System – Requirements (2008) Google Scholar
  32. 32.
    MINITAB Statistical Software (Release 14),

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Ayca Tarhan
    • 1
  • Onur Demirors
    • 2
  1. 1.Computer Engineering DepartmentHacettepe UniversityBeytepe YerleskesiTurkey
  2. 2.Middle East Technical University Informatics InstituteInonu BulvariTurkey

Personalised recommendations