Quantifying the benefits of software testing: an experience report from the GSM application domain

Abstract

This paper aims at describing the quantitative results gained from a systematic integration test campaign undergone at Siemens Telecomunicazioni Italia on parts of a GSM (Global System for Mobile Communications) Phase 2 system. In particular, the following topics are dealt with: basics of the Siemens development process, with emphasis on testing issues; a high level description of the system under test; the actions taken in order to improve the effectiveness of software testing practices; the methods and techniques applied during the project; the measurement system that was set-up in order to track the effectiveness of the improvements; the quantitative results collected, with associated interpretations; the future goals, in the light of continuous improvement. The results clearly show a positive impact of systematic testing on timeliness, reliability and documentation, underlining also the importance of anticipating error detection as much as possible in the development cycle and the need for tools supporting the various activities. The data collected provide also models and baselines onto which it will be possible to base estimates for new projects.