Reliability modeling for safety critical software

Abstract

We show how software reliability predictions can increase confidence in the reliability of safety critical software such as the NASA Space Shuttle Primary Avionics Software System (Shuttle flight software). This objective was achieved with our novel approach of integrating software safety criteria, risk analysis, reliability prediction, and a stopping rule for testing. This approach is applicable to other safety critical software. We encourage practitioners to apply this approach. Only the safety of the software in a safely critical system is covered. The hardware and human operator parts of such systems are not covered. Our concern is with reducing the risk of failures in the software, which could cause loss of life or mission, to an acceptable level. Thus, our use of the word safety refers to software safety and not to system safety. By improving the reliability of the software, where the reliability measurements and predictions are directly related to mission and crew safety, we contribute to system safety.

Remaining failures, total failures, test time required to attain a given fraction of remaining failures, and time to next failure are useful reliability measurements and predictions for: 1) providing confidence that the software has achieved safety goals; 2) rationalizing how long to test a piece of software; and 3) analyzing the risk of not achieving remaining failure and time to next failure goals. Having predictions of the extent that the software is not fault free (remainingfailures) and whether it is likely to survive a mission (time to next failure) provide criteria for assessing the risk of deploying the software. Furthermore, fractfon of remaining failures can be used as both an operational quality goal in predicting test time requirements and, conversely, as an indicator of operational qualify as a function of test time expended.

Software reliability models provide one of several tools that software reliability managers of the Shuttle flight software are using to provide confidence that the software meets required safety goals. Other tools are inspections, software reviews, testing, change control boards, and perhaps most important — experience and judgement.