Abstract
From commercial to life-critical applications, the proliferation of computing systems in everyday life has substantially increased our dependence on them. Failures in air traffic control systems, nuclear reactors, or hospital patient monitoring systems can bring catastrophic consequences. In order to enhance the dependability of computing systems, an effective evaluation of their reliability is desired. This chapter presents methods for evaluating system reliability, and indicates that stochastic modeling has provided an effective and unified framework for analyzing various aspects of reliability.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
O.O. Aalen, Ø. Borgan, H.K. Gjessing, Survival and Event History Analysis: A Process Point of View (Springer, New York, 2008)
A.A. Abdel-Ghaly, P.Y. Chan, B. Littlewood, Evaluation of competing software reliability predictions. IEEE Trans. Softw. Eng. 12, 950–967 (1986)
D. Al-Mutairi, Y. Chen, N.D. Singpurwalla, An adaptative concatenated failure rate model for software reliability. J. Am. Stat. Assoc. 93, 1150–1163 (1998)
S.V. Amari, R.B. Misra, Closed-form expressions for distribution of sum of exponential random variables. IEEE Trans. Reliab. 46(4), 519–522 (1997)
S.V. Amari, J.B. Dugan, R.B. Misra, Optimal reliability of systems subject to imperfect fault coverage. IEEE Trans. Reliab. 48(3), 275–284 (1999)
S.V. Amari, K.B. Misra, H. Pham, Tampered failure rate load-sharing systems: status and perspectives, in Handbook on Performability Engineering, ed. by K.B. Misra (Springer, London, 2008)
P.M. Anderson, S.K. Agarwal, An improved model for protective-system reliability. IEEE Trans. Reliab. 41(3), 422–426 (1992)
H. Ascher, H. Feingold, Repairable Systems Reliability: Modeling, Inference, Misconceptions and Their Causes (Marcel Dekker, New York, 1984)
N.T.J. Bailey, The Elements of Stochastic Processes (Wiley, New York, 1964)
N. Balakrishnan, C.D. Lai, Continuous Bivariate Distributions (Springer, New York, 2009)
R.E. Barlow, F. Prochan, Statistical Theory of Reliability and Life Testing: Probability Models (To Begin With, Silver Spring, 1981)
H.W. Block, A.P. Basu, A continuous bivariate exponential extension. J. Am. Stat. Assoc. 69, 1031–1037 (1974)
P.J. Boland, N.N. ChuÃv, Optimal times for software release when repair is imperfect. Stat. Probab. Lett. 77, 1176–1184 (2007)
L.C. Briand, K.E. Emam, B.G. Freimut, A comprehensive evaluation of capture-recapture models for estimating software defect content. IEEE Trans. Softw. Eng. 26(6), 518–540 (2000)
S.S. Brilliant, J.C. Knight, N.G. Leveson, Analysis of faults in an N-version software experiment. IEEE Trans. Softw. Eng. 16(2), 238–247 (1990)
X. Cai, M.R. Lyu, M.A. Vouk, An experimental evaluation on reliability features of N-version programming, in Proceeding of the 16th International Symposium on Software Reliability Engineering, Chicago, 2005
K.C. Chae, G.M. Clark, System reliability in the presence of common-cause failures. IEEE Trans. Reliab. 35(1), 32–35 (1986)
Y. Chen, N.D. Singpurwalla, Unification of software reliability models by self-exciting point processes. Adv. Appl. Probab. 29, 337–352 (1997)
D.R. Cox, V. Isham, Point Processes (Chapman & Hall, London, 1980)
D.R. Cox, P.A.W. Lewis, The Statistical Analysis of Series of Events (Chapman & Hall, London, 1966)
L.H. Crow, Reliability analysis for complex repairable systems, in Reliability and Biometry: Statistical Analysis of Lifelength, ed. by F. Proschan, R.J. Serfling (SIAM, Philadelphia, 1974)
M.J. Crowder, A.C. Kimber, R.L. Smith, T.J. Sweeting, Statistical Analysis of Reliability Data (Chapman & Hall, London, 1991)
T. Dohi, S. Osaki, K.S. Trivedi, An infinite server queueing approach for describing software reliability growth: unified modeling and estimation framework, in Proceedings of the 11th Asia-Pacific Software Engineering Conference, Busan, 2004
J.T. Duane, Learning curve approach to reliabilitymonitoring. IEEE Trans. Aerosp. 2, 563–566 (1964)
D.E. Eckhardt, L.D. Lee, A theoretical basis for the analysis of multiversion software subject to coincident errors. IEEE Trans. Softw. Eng. 11(12), 1511–1517 (1985)
D.E. Eckhardt, A.K. Caglavan, J.C. Knight, L.D. Lee, D.F. McAllister, M.A. Vouk, J.P. Kelly, An experimental evaluation of software redundancy as a strategy for improving reliability. IEEE Trans. Softw. Eng. 17(7), 692–702 (1991)
M. Ege, M.A. Eyler, M.U. Karakas, Reliability analysis in N-version programming with dependent failures, in Proceedings of the 27th Euromicro Conference, Warsaw, 2001
K.N. Fleming, N. Mosleh, R.K. Deremer, A systematic procedure for incorporation of common cause events into risk and reliability models. Nucl. Eng. Des. 93, 245–273 (1986)
J.E. Freund, A bivariate extension of the exponential distribution. J. Am. Stat. Assoc. 56, 971–977 (1961)
O. Gaudoin, J.L. Soler, Statistical analysis of the geometric de-eutrophication software-reliability model. IEEE Trans. Reliab. 41(4), 518–524 (1992)
O. Gaudoin, C. Lavergne, J.L. Soler, A generalized geometric de-eutrophication software-reliability model. IEEE Trans. Reliab. 43(4), 536–541 (1994)
B.V. Gnedenko, Y.K. Belyayev, A.D. Solovyev, Mathematical Methods of Reliability Theory (Academic, New York, 1969)
A.L. Goel, Software reliability models: assumptions, limitations, and applicability. IEEE Trans. Softw. Eng. 11(12), 1411–1423 (1985)
A.L. Goel, K. Okumoto, Time-dependent error-detection rate model for software reliability and other performance measures. IEEE Trans. Reliab. 28(3), 206–211 (1979)
S.S. Gokhale, P.N. Marinos, K.S. Trivedi, Important milestomes in software reliability modeling, in Proceedings of Software Engineering and Knowledge Engineering (SEKE) ’96, Lake Tahoe, 1996
K. Go\(\breve{\mbox{ s}}\)eva-Popstojanova, K.S. Trivedi, Architecture-based approaches to reliability assessment of software systems. Perform. Eval. 45, 179–204 (2001)
M.H. Halstead, Elements of Software Science (North-Holland, Amsterdam, 1977)
P. Hokstad, M. Rausand, Common cause failure modeling: status and trends, in Handbook of Performability Engineering, ed. by K.B. Misra (Springer, London, 2008)
C.Y. Huang, W.C. Huang, Software reliability analysis and measurement using finite and infinite server queueing models. IEEE Trans. Reliab. 57(1), 192–203 (2008)
C.Y. Huang, M.R. Lyu, S.Y. Kuo, A unified scheme of some nonhomogenous Poisson process models for software reliability estimation. IEEE Trans. Softw. Eng. 29(3), 261–269 (2003)
L. Huang, Q. Xu, Lifetime reliability for load-sharing redundant systems with arbitrary failure distributions. IEEE Trans. Reliab. 59(2), 319–330 (2010)
Z. Jelinski, P.B. Moranda, Software reliability research, in Statistical Computer Performance Evaluation, ed. by W. Freiberger (Academic, New York, 1972)
H. Joe, Multivariate Models and Dependence Concepts (Chapman & Hall, London, 1997)
P.K. Kapur, A. Gupta, P.C. Jha, Reliability growth modeling and optimal release policy under fuzzy environment of an N-version programming system incorporating the effect of fault removal efficiency. Int. J. Autom. Comput. 4(4), 369–379 (2007)
M. Kijima, Markov Processes for Stochastic Modeling (Chapman & Hall, London, 1997)
B.A. Kitchenham, B. Littlewood, Measurement for Software Control and Assurance (Elsevier, London, 1989)
J.C. Knight, N.G. Leveson, An experimental evaluation of the assumption of independence in multiversion programming. IEEE Trans. Softw. Eng. 12(1), 96–109 (1986)
K.A.H. Kobbacy, D.N.P. Murthy, Complex System Maintenance Handbook (Springer, London, 2008)
W. Kremer, Birth-death and bug counting. IEEE Trans. Reliab. 32(1), 37–47 (1983)
P.H. Kvam, E.A. Peña, Estimating load-sharing properties in a dynamic reliability system. J. Am. Stat. Assoc. 100, 262–272 (2005)
N. Langberg, N.D. Singpurwalla, A unification of some software reliabilitymodels. SIAM J. Sci. Stat. Comput. 6, 781–790 (1985)
J. Ledoux, Software reliability modeling, in Handbook of Reliability Engineering, ed. by H. Pham (Springer, London, 2003)
H.H. Lin, K.H. Chen, R.T. Wang, A multivariate exponential shared-load model. IEEE Trans. Reliab. 42(1), 165–171 (1993)
B.H. Lindqvist, On the statistical modeling and analysis of repairable systems. Stat. Sci. 21(4), 532–551 (2006)
B.H. Lindqvist, G. Elvebakk, K. Heggland, The trend-renewal process for statistical analysis of repairable systems. Technometrics 45, 31–44 (2003)
B. Littlewood, Theories of software reliability: how good are they and how can they be improved? IEEE Trans. Softw. Eng. 6(5), 489–500 (1980)
B. Littlewood, Stochastic reliability-growth: a model for fault-removal in computer-programs and hardware-designs. IEEE Trans. Reliab. 30(4), 313–320 (1981)
B. Littlewood, Rationale for a modified Duane model. IEEE Trans. Reliab. 33(2), 157–159 (1984)
B. Littlewood, D. Miller, Conceptual modeling of coincident failures in multiversion software. IEEE Trans. Softw. Eng. 15(12), 1596–1614 (1989)
B. Littlewood, J.L. Verrall, A Bayesian reliability growth model for computer software. J. R. Stat. Soc. C 22, 332–346 (1973)
B. Littlewood, J.L. Verrall, A Bayesian reliability model with a stochastically monotone failure rate. IEEE Trans. Reliab. 23(2), 108–114 (1974)
H. Liu, Reliability of a load-sharing k-out-of-n:G system: non-iid components with arbitrary distributions. IEEE Trans. Reliab. 47(3), 279–284 (1998)
J.C. Lu, Weibull extensions of the Freund and Marshall-Olkin bivariate exponential models. IEEE Trans. Reliab. 38(5), 615–619 (1989)
M.R. Lyu, Handbook of Software Reliability Engineering (McGraw-Hill, New York/IEEE Computer Society, Los Angeles, 1996)
M.R. Lyu, Y. He, Improving the n-version programming process through the evolution of a design paradigm. IEEE Trans. Reliab. 42(2), 179–189 (1993)
M.R. Lyu, Z. Huang, K.S. Sze, X. Cai, An empirical study on testing and fault tolerance of software reliability engineering, in Proceedings of the 14th International Symposium on Software Reliability Engineering (ISSRE 2003), Denver, 2003
A.W. Marshall, I. Olkin, A multivariate exponential distribution. J. Am. Stat. Assoc. 62, 30–44 (1967)
T.J. McCabe, A compexity measure. IEEE Trans. Softw. Eng. 2(4), 308–320 (1976)
W.Q. Meeker, L.A. Escobar, Statistical Methods for Reliability Data (Wiley, New York, 1998)
D.R. Miller, Exponential order statistic models of software reliability growth. IEEE Trans. Softw. Eng. 12(1), 12–24 (1986)
K.B. Misra, Handbook of Performability Engineering (Springer, London, 2008)
P.B. Moranda, Predictions of software reliability during debugging, in Proceedings Annual Reliability and Maintainability Symposium, Washington, DC, 1975
P.B. Moranda, Event-altered rate models for general reliability analysis. IEEE Trans. Reliab. 28(5), 376–381 (1979)
J.D. Musa, A.F. Ackerman, Quantifying software validation: when to stop testing? IEEE Softw. 6, 19–27 (1989)
J.D. Musa, K. Okumoto, A logarithmic Poisson execution time model for software reliability measurement, in Proceedings 7th International Conference on Software Engineering, Orlando, 1984
J.D. Musa, A. Iannino, K. Okumoto, Software Reliability: Measurement, Prediction, Application (McGraw-Hill, New York, 1987)
A. Myers, Complex System Reliability: Multichannel Systems with Imperfect Fault Coverage (Springer, London, 2010)
T. Nakagawa, Maintenance Theory of Reliability (Springer, London, 2005)
NEA, International common-cause failure data exchange. ICDE general coding guidelines. Technical note NEA/CSNI/R(2004)4. Nuclear Energy Agency (2004)
V.F. Nicola, A. Goyal, Modelling of correlated failures and community error recovery in multiversion software. IEEE Trans. Softw. Eng. 16(3), 350–359 (1990)
E. Parzen, Stochastic Processes (Holden Day, San Francisco, 1962)
E.A. Peña, Dynamic modeling and statistical analysis of event times. Stat. Sci. 21(4), 487–500 (2006)
H. Pham, Software Reliability (Springer, New York, 2000)
H. Pham, Handbook of Reliability Engineering (Springer, London, 2003)
H. Pham, Recent Advances in Reliability and Quality in Design (Springer, London, 2008)
H. Pham, L. Nordmann, X. Zhang, A general imperfect-software-debugging model with s-shaped fault-detection rate. IEEE Trans. Reliab. 48(2), 169–175 (1999)
P.T. Popov, L. Strigini, J. May, S. Kuball, Estimating bounds on the reliability of diverse systems. IEEE Trans. Softw. Eng. 29(4), 345–359 (2003)
P. Pukite, J. Pukite, Modeling for Reliability Analysis (IEEE, New York, 1998)
L. Rade, Expected time to failure of reliability systems. Math. Sci. 14, 24–37 (1989)
A.E. Raftery, Inference and prediction for a general order statistic model with unknown population size. J. Am. Stat. Assoc. 82, 1163–1168 (1987)
S. Ramani, S.S. Gokhale, K.S. Trivedi, in Computer Performance Evaluation: Modelling Techniques and Tools, SREPT: Software Reliability Estimation and Prediction Tool, ed. by R. Puigjaner et al. Lecture Notes in Computer Science (LNCS 1469) (Springer, New York, 1998)
M. Rausand, A. Høyland, System Reliability Theory: Models, Statistical Methods, and Applications (Wiley, Hoboken, 2004)
S.M. Ross, Introduction to Probability Models, 9th edn. (Academic, London, 2007)
E.M. Scheuer, Reliability of an m-out-of-n system when component failure induces higher failure rates in survivors. IEEE Trans. Reliab. 37(1), 73–74 (1988)
G.J. Schick, R.W. Wolverton, Assessment of software reliability, in Proceedings of the Operations Research (Physica, Wirzberg-Wien, 1973). September 1972 in Hamburg
G.J. Schick, R.W. Wolverton, An analysis of competing software reliability models. IEEE Trans. Softw. Eng. 4, 104–120 (1978)
J. Shao, L.R. Lamberson, Modeling a shared-load k-out-of-n:G system. IEEE Trans. Reliab. 40(2), 205–209 (1991)
K. Sharma, R. Garg, C.K. Nagpal, R.K. Garg, Selection of optimal software reliability growth models using a distance based approach. IEEE Trans. Reliab. 59(2), 266–276 (2010)
N.D. Singpurwalla, Foundational issues in reliability and risk analysis. SIAM Rev. 30(2), 264–282 (1988)
N.D. Singpurwalla, Reliability and Risk: A Bayesian Perspective (Wiley, New York, 2006)
N.D. Singpurwalla, R. Soyer, Non-homogeneous autoregressive processes for tracking (software) reliability growth, and their Bayesian analysis. J. R. Stat. Soc. B 54, 145–156 (1992)
N.D. Singpurwalla, S.P. Wilson, Software reliability modeling. Int. Stat. Rev. 62, 289–317 (1994)
D.L. Snyder, M.I. Miller, Random Point Processes in Time and Space, 2nd edn. (Springer, New York, 1991)
H.S. Son, M.C. Kim, Software faults and reliability, in Reliability and Risk Issues in Large Scale Safety-Critical Digital Control Systems, ed. by P.H. Seong (Springer, London, 2009)
Z. Tang, J.B. Dugan, An integrated method for incorporating common cause failures in system analysis, in Proceedings of the 50th Annual Reliability and Maintainability Symposium, Los Angeles, 2004
X. Teng, H. Pham, A software-reliability growth model for N-version programming systems. IEEE Trans. Reliab. 51(3), 311–321 (2002)
K. Tokuno, S. Yamada, An imperfect debugging model with two types of hazard rates for software reliability measurement and assessment. Math. Comput. Model. 31, 343–352 (2000)
U.S. Nuclear Regulatory Commission, Reactor Safety: An Assessment of Accident Risks in U.S. Commercial Nuclear Power Plants. WASH-1400, NUREG-75/014 (U.S. Nuclear Regulatory Commission, Washington, DC, 1975)
E.A. van Doom, On the α-classification of birth-death and quasi-bith-death processes. Stoch Model 22, 411–421 (2006)
J.K. Vaurio, An implicit method for incorporating common-cause failures in system analysis. IEEE Trans. Reliab. 47(2), 173–180 (1998)
A.K. Verma, S. Ajit, M. Kumar, Dependability of Networked Computer-Based Systems (Springer, London, 2011)
W.L. Wagoner, The final report on a software reliability measurement study. TOR-0074(4112)-1. Aerospace Corporation, El Segundo (1973)
R. Wang, A mixture and self-exciting model for software reliability. Stat. Probab. Lett. 72, 187–194 (2005)
R.T. Wang, A reliability model for multivariate exponential distributions. J. Multivar. Anal. 98, 1033–1042 (2007)
R.T. Wang, A dependent model for fault tolerant software systems during debugging. IEEE Trans. Reliab. 61(2), 504–515 (2012)
M. Xie, Software Reliability Modelling (World Scientific, Singapore, 1991)
L. Xing, Fault-tolerant network reliability and importance analysis using binary decision diagrams, in Proceedings of the 50th Annual Reliability and Maintainability Symposium, Los Angeles, 2004
S. Yamada, Software reliability growth models incorporating imperfect debugging with introduced faults. Electron. Commun. Jpn. Part 3 81(4), 33–40 (1998)
S. Yamada, S. Osaki, Software reliability growth modeling: models and applications. IEEE Trans. Softw. Eng. 11(12), 1431–1437 (1985)
S. Yamada, M. Ohba, S. Osaki, S-shaped reliability growth modeling for software error detection. IEEE Trans. Reliab. 32(5), 475–478 (1983)
S. Yamada, M. Ohba, S. Osaki, S-shaped software reliability growth models and their applications. IEEE Trans. Reliab. 33(4), 289–292 (1984)
K.Z. Yang, An infinite server queueing model for software readiness assessment and related performance measures. Ph.D. Dissertation, Syracuse University, 1996
S. Zack, Introduction to Reliability Analysis: Probability Models and Statistical Methods (Springer, New York, 1992)
P. Zeephongsekul, G. Xia, S. Kumar, Software reliability growth models: primary failures generate secondary-faults under imperfect debugging. IEEE Trans. Reliab. 43(3), 408–413 (1994)
T. Zhang, M. Horigome, Availability and reliability of system with dependent components and time-varying failure and repair rates. IEEE Trans. Reliab. 50(2), 151–158 (2001)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media New York
About this chapter
Cite this chapter
Wang, RT. (2014). Reliability Evaluation Techniques. In: Mathew, J., Shafik, R., Pradhan, D. (eds) Energy-Efficient Fault-Tolerant Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4193-9_2
Download citation
DOI: https://doi.org/10.1007/978-1-4614-4193-9_2
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-4192-2
Online ISBN: 978-1-4614-4193-9
eBook Packages: EngineeringEngineering (R0)