Basic Stochastic Process Theory

  • Alessandro Birolini


Stochastic processes are powerful tools for the investigation of the reliability and availability of repairable equipment and systems. They can be considered as families of time-dependent random variables or as random functions in time, and thus have a theoretical foundation based on probability theory (Appendix A6). The use of stochastic processes allows the analysis of the influence of the failure-free operating and repair time distributions of elements, as well as of the system’s structure, repair strategy, and logistical support, on the reliability and availability of a given system. Considering the applications given in Chapter 6, and for reasons of mathematical tractability, this appendix mainly deals with regenerative stochastic processes with a finite state space, to which belong renewal processes, Markov processes, semi-Markov processes, and semi-regenerative processes. The theoretical presentation will be supported by examples taken from practical applications. This appendix is a compendium of the theory of stochastic processes, consistent from a mathematical point of view but still with engineering applications in mind.


Markov Process Renewal Process Sojourn Time Embed Markov Chain Active Redundancy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


A7 Stochastic Processes

  1. [A7.1]
    Asmussen S., Applied Probability and Queues, 1986, Wiley, Chicherster.Google Scholar
  2. [A7.2]
    Birolini A., “Some appl. of regen. stoch. proc. to rel. theory — Part one and two”, IEEE Trans. Rel, 23 (1974) 3, pp. 186–194 and 24(1975)5, pp. 336–340; Semi-Markoff und verwandte Prozesse: Erzeugung und Anwendungen auf Probleme der Zuverlässigkeits- und Übertragungsth, 1974, Ph.D. Thesis 5375, ETH Zurich, also in AGEN-Mitt., 18(1975), pp. 3–52; “Hardware simulation of semi-Markov and related proc, Math. & Comp. in Simul., 19(1977), pp. 75–97 and 183–191; On the Use of Stoch. Proc. in Modeling Rel. Problems, 1985, Springer, Berlin (Lect. Notes Ec. & Math. Systems Nr. 252); Quality and Reliability of Technical Systems, 2nd Ed. 1997, Springer, Berlin; Zuverlässigkeit von Geräten und Systemen, 4th Ed. 1997, Springer, Berlin.MathSciNetCrossRefGoogle Scholar
  3. [A7.3]
    Cinlar E., Introduction to Stochastic Processes, 1975, Prentice Hall, Englewood Cliffs NJ.MATHGoogle Scholar
  4. [A7.4]
    Cox D.R., “The analysis of non-markovian stoch. proc. by the inclusion of sup. variables”, Proc. Cambridge Phil Soc, 51(1955), pp. 433–441; Renewal Theory, 1962, Methuen, London.MATHCrossRefGoogle Scholar
  5. [A7.5]
    Cox D.R., Lewis P.A., The Stat. Analysis of Series of Events, 2nd Ed. 1968, Methuen, London.Google Scholar
  6. [A7.6]
    Cramér H., “Model building with the aid of stoch. proc.”, Technometrics, 6 (1964), pp. 133–159.MathSciNetMATHCrossRefGoogle Scholar
  7. [A7.7]
    Cramér H., Leadbetter M.R., Stationary and Related Stochastic Processes, 1967, Wiley, New York.MATHGoogle Scholar
  8. [A7.8]
    Doob I.L., Stochastic Processes, 7th Ed. 1967, Wiley, New York.Google Scholar
  9. [A7.9]
    Feller W., “On the integral equation of renewal theory”, Ann. Math. Statistics, 12(1941), pp.243–67; “On semi-Markov-processes”, Proc. Nat. Acad. Scient. (USA), 51(1964), pp. 653–659; An Introduction to Prob. Theory and its Applic, Vol 13th Ed. 1968, Vol II 2nd Ed. 1966, Wiley, N.Y.MathSciNetCrossRefGoogle Scholar
  10. [A7.10]
    Franken P., Streller A., “Reliability analysis of complex repairable systems by means of marked point processes”, J. Appl. Prob., 17(1980), pp. 154–167; — and Kirsten B.M., “Reliability analysis of complex systems with repair”, EIK, 20(1984), pp. 407–422.MathSciNetMATHCrossRefGoogle Scholar
  11. [A7.11]
    Franken P. et al, Queues and Point Processes, 1981, Akademie, Berlin.Google Scholar
  12. [A7.12]
    Gnedenko B.W., König D. (Ed.), Handbuch der Bedienungstheorie, Vol.1 & II1983, Akad., Berlin.MATHGoogle Scholar
  13. [A7.13]
    Gnedenko B.W., Kovalenko I.N., Introduction to Quening Theory, 1989, Birkhäuser, Basel.CrossRefGoogle Scholar
  14. [A7.14]
    Grigelionis B.I., “Limit theorems for sums of repair processes”, Cybernetics in the Serv. of Comm., 2(1964), pp. 316–341.Google Scholar
  15. [A7.15]
    Hunter J.J., Mathematical Techniques of Applied Probability, Vol. 2, Discrete Time Models, 1983, Academic Press, New York.Google Scholar
  16. [A7.16]
    Karlin S., McGregor J.L., “The differential equations of birth and death processes, and the Stieltjes moment problem”, Trans. Amer. Math. Soc, 86(1957), pp. 489–546;MathSciNetCrossRefGoogle Scholar
  17. [A7.16a]
    Karlin S., McGregor J.L., “The classification of birth and death processes”, Trans. Amer. Math. Soc, 85(1957), pp. 366–400;CrossRefGoogle Scholar
  18. [A7.16]
    Karlin S., McGregor J.L.,“Coincidence properties of birth and death processes”, Pacific J. Math., 9(1959), pp. 1109–1140.MathSciNetMATHCrossRefGoogle Scholar
  19. [A7.17]
    Khintchine A.Y., Mathematical Methods in the Theory of Queuing, 1960, Griffin, London.Google Scholar
  20. [A7.18]
    Lévy P., “Processus semi-markoviens”, Proc. Int. Congr. Math. Amsterdam, 3(1954), pp. 416–26.Google Scholar
  21. [A7.19]
    Neuts M.F., Matrix-geometric Solutions in Stochastic Models: an Algorithmic Approach, 1981, Hopkins Univ. Press, Baltimore MA.MATHGoogle Scholar
  22. [A7.20]
    Osaki S. and Hatoyama Y. (Eds.), Stochastic Models in Reliability Theory, 1984, Springer, Berlin (Lect. Notes in Ec. and Math. Syst. Nr. 235).MATHGoogle Scholar
  23. [A7.21]
    Parzen E., Stochastic Processes, 3rd Printing 1967, Holden-Day, San Francisco.Google Scholar
  24. [A7.22]
    Pavlov I.V., “The asymptotic distribution of the time until a semi-Markov process gets out of a kernel”, Eng. Cybernetics, (1978)5, pp. 68–72.Google Scholar
  25. [A7.23]
    Pyke R., “Markov renewal processes: definitions and preliminary properties”, Annals Math. Statistics, 32(1961), pp. 1231–1242;MathSciNetMATHCrossRefGoogle Scholar
  26. [A7.23a]
    Pyke R., “Markov renewal proc. with finitely many states”, Annals Math. Stat., 32(1961), pp.1243–1259MathSciNetMATHCrossRefGoogle Scholar
  27. [A7.23b]
    Pyke R.,#x2014; and Schaufele R., “Limit theorems for Markov renewal proc”, Annals Math. Stat., 35(1964), pp. 1746–1764; “The existence and uniqueness of stationary measures for Markov renewal proc.”, Annals Math. Stat., 37(1966), pp. 1439–1462.MathSciNetMATHCrossRefGoogle Scholar
  28. [A7.24]
    Smith W.L., “Asymptotic renewal theorems”, Proc. Roy. Soc. Edinbourgh, 64(1954), pp. 9–48; “Regenerative stochastic processes, Proc. Int. Congress Math. Amsterdam, 3(1954), pp. 304–305; “Regenerative stochastic processes”, Proc. Roy. Soc. London, Ser. A, 232(1955), pp. 6–31; “Remarks on the paper: Regenerative stochastic processes”, Proc. Roy. Soc. London, Ser. A, 256(1960), pp. 496–501; “Renewal theory and its ramifications”, J. Roy. Stat. Soc, Ser.B, 20(1958), pp. 243–302.MATHGoogle Scholar
  29. [A7.25]
    Snyder D.L., Miller M.I., Random Point Processes in Time and Space, 2nd Ed. 1991, Springer, Berlin.MATHCrossRefGoogle Scholar
  30. [A7.26]
    Solovyev A.D., “The problem of optimal servicing”, Eng. Cybernetics, 5(1970), pp. 859–868; “Asymptotic distribution of the moment of first crossing of a high level by a birth and death process”, Proc sixth Berkeley Symp. Math. Stat. Prob., 3(1970), pp. 71–86; “Asymptotic behavior of the time of first occurr. of a rare event in a reg. proc”, Eng. Cybernetics, 6(1971)9, pp. 1038–048.MathSciNetGoogle Scholar
  31. [A7.27]
    Srinivasan S.K., Mehata K.M., Stochastic processes, 2nd Ed. 1988, Tata McGraw-Hill, New Delhi.Google Scholar
  32. [A7.28]
    Störnier H., Semi-Markoff-Prozesse mit endlich vielen Zuständen, 1970, Springer, Berlin (Lect. Notes in Op. Res. and Math. Syst. Nr. 34).CrossRefGoogle Scholar
  33. [A7.29]
    Takács L., “On certain sojurn time problems in the theory of stoch. proc”, Acta Math. (Hungar), 8(1957), pp. 169–191; Stochastic processes, problems and solutions, 4th Ed. 1968, Methuen, London.MATHCrossRefGoogle Scholar
  34. [A7.30]
    Thompson W.A., Jr., Point Processes Models with Applications to Safety and Reliability, 1988, Chapman & Hall, New York.CrossRefGoogle Scholar
  35. [6.1]
    Ascher H., Feingold H., Repairable Systems Reliability, 1984, Dekker, New York.MATHGoogle Scholar
  36. [6.2]
    Barlow R.E., Proschan F., Mathematical Theory of Reliability, 1965, Wiley, New York; Statistical Theory of Reliability and Life Testing, 1975, Holt Rinehart & Winston, New York.MATHGoogle Scholar
  37. [6.3]
    Beichelt F., Franken P., Zuverlässigkeit und Instandhaltung — Math. Methoden, 1983, Technik, Berlin;Google Scholar
  38. [6.3a]
    Beichelt F., Zuverlässigkeits- und Instandhaltbarkeitstheorie, 1993, Teubner, Stuttgart.Google Scholar
  39. [6.4]
    Birolini A., “Comments on renewal theoretic aspects of two-unit redundant systems”, IEEE Trans. Rel., 21(1972)2, pp 122–123; “Generalization of the expressions for the rel. and availability of rep. items”, Proc. 2. Int. Conf. on Struct. Mech. in Reactor Techn., Berlin: 1973, Vol. VI, pp. 1–16; “Some appl. of regen. stoch. processes to reliability theory — part two: rel. and availability of 2-item redundant systems”, IEEE Trans. Rel., 24(1975)5, pp. 336–340; On the Use of Stochastic Proc. in Modeling Rel. Problems, 1985, Springer, Berlin (Lecture Notes in Ec. and Math. Systems Nr. 252); Quality and Reliability of Technical Systems, 2nd Ed. 1997, Springer, Berlin; Zuverlässigkeit von Geräten und Systemen, 4th Ed. 1997, Springer, Berlin.CrossRefGoogle Scholar
  40. [6.5]
    Bobbio A., Roberti L., “Distribution of the minimal completition time of parallel tasks in multi-reward semi-Markov models”, Performance Eval., 14(1992), pp. 239–256.MathSciNetMATHCrossRefGoogle Scholar
  41. [6.6]
    Brenner A., Performability and Dependability of Fault-Tolerant Systems, 1996, Ph. D. Thesis 11623, ETH Zurich.Google Scholar
  42. [6.7]
    Choi, C.Y. et al, “Safety issues in the comparative analysis of dependable architectures”, IEEE Trans. Rel., 46(1997)3, pp. 316–322.CrossRefGoogle Scholar
  43. [6.8]
    Dhillon B.S., Rayapati S.N., “Common-cause failures in repairable systems”, Proc. Ann. Rel. & Maint. Symp., 1988, pp. 283–289.Google Scholar
  44. [6.9]
    Dyer D., “Unification of rel./availab. mod. for Markov syst.”, IEEE Trans. Rel, 38(1989)2, pp.246–52.MATHCrossRefGoogle Scholar
  45. [6.10]
    Gaede K.W., Zuverlässigkeit Mathematische Modelle, 1977, Hanser, Munich.MATHGoogle Scholar
  46. [6.11]
    Gnedenko B.V., Beljajev J.K., Soloviev A.D., Mathematical Methods of Reliability Theory, 1969, Academic, New York (1968, Akademie, Berlin).MATHGoogle Scholar
  47. [6.12]
    Kovalenko I., Birolini A., “Uniform exponential boundes for the availability of a repairable system”, in Exploring Stochastic laws, Homage to V.S. Korolyuk, 1995, VSP, Utrecht, pp.233–242.Google Scholar
  48. [6.13]
    Kullstam A., “Availability, MTBF and MTTR for repairable M-out-of-N Systems”, IEEE Trans. Rel., 30(1981)4, pp. 393–394.MATHCrossRefGoogle Scholar
  49. [6.14]
    Kumar A., Agarwal M., “A review of standby red. syst”, IEEE Trans. Rel, 29(1980)4, pp. 290–294.MATHCrossRefGoogle Scholar
  50. [6.15]
    Osaki S., Nakagawa T., “Bibliography for reliability and availability of stochastic systems”, IEEE Trans. Rel. 25(1976)4, pp. 284–287.MathSciNetMATHCrossRefGoogle Scholar
  51. [6.16]
    Rai S., Agrawal D.P. (Ed.), Advances in Distributed Systems Reliability and Distributed Computing Network Reliability, 1990, IEEE Press, Piscataway NJ.Google Scholar
  52. [6.17]
    Ravichandran N., Stochastic Methods in Reliability Theory, 1990, Wiley Eastern, New Dehli.MATHGoogle Scholar
  53. [6.18]
    Schneeweiss W.G., “Mean time to first failure of repairable systems with one cold spare”, IEEE Trans. Rel., 44(1995)4, pp. 567–574.MathSciNetCrossRefGoogle Scholar
  54. [6.19]
    Ushakov I.A., Harrison R., Handbook of Reliability Engineering, 1994, Wiley, New York.MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1999

Authors and Affiliations

  • Alessandro Birolini
    • 1
  1. 1.ETH ZurichLuganoSwitzerland

Personalised recommendations