Transient Solutions for Markov Chains

  • Edmundo de Souza e Silva
  • H. Richard Gail
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 24)

Abstract

Much of the theory developed for solving Markov chain models is devoted to obtaining steady state measures, that is, measures for which the observation interval (0, t) is “sufficiently large” (t → ∞). These measures are indeed approximations of the behavior of the system for a finite, but long, time interval, where long means with respect to the interval of time between occurrences of events in the system. However, an increasing number of applications requires the calculation of measures during a relatively “short” period of time. These are the so-called transient measures. In these cases the steady state measures are not good approximations for the transient, and one has to resort to different techniques to obtain the desired quantities.

Keywords

Markov Chain Krylov Subspace Transition Probability Matrix Computational Probability Transient Solution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [Abate and Whitt, 1989]
    Abate, J. and Whitt, W. (1989). Calculating time-dependent performance measures for the M/M/1 queue. IEEE Trans. on Communications,37(10):891–904.Google Scholar
  2. [Abdallah and Marie, 1993]
    Abdallah, H. and Marie, R. (1993). The uniformized power method for transient solutions of Markov processes. Computers E4 Operations Research, 20 (5): 515–526.CrossRefGoogle Scholar
  3. [Ajmone Marsan and Chiola, 1987]
    Ajmone Marsan, M. and Chiola, G. (1987). On Petri nets with deterministic and exponentially distributed firing times. In Rozenberg, G., editor, Lecture Notes in Computer Science 266: Advances in Petri Nets 1987, pages 132–145. Springer-Verlag.Google Scholar
  4. [Bobbio and Telek, 1993]
    Bobbio, A. and Telek, M. (1993). Task completion time. In Proceedings of the 2nd Int’l Workshop on Performability Modeling of Computer and Communication Systems, Le Mont Saint-Michel, France.Google Scholar
  5. [Bobbio and Trivedi, 1986]
    Bobbio, A. and Trivedi, K. (1986). An aggregation technique for the transient analysis of stiff Markov chains. IEEE Trans. on Computers, C-35(9): 803–814.Google Scholar
  6. [Carmo et al., 1998]
    Carmo, R., de Carvalho, L., de Souza e Silva, E., Diniz, M., and Muntz, R. (1998). Performance/availability modeling with the TANGRAM-II modeling environment. Performance Evaluation, 33: 45–65.CrossRefGoogle Scholar
  7. [Carmo et al., 1996]
    Carmo, R., de Souza e Silva, E., and Marie, R. (1996). Efficient solutions for an approximation technique for the transient analysis of Markovian models. Technical report, IRISA, no. 3055, Campus universitaire de Beaulieu.Google Scholar
  8. [Carrasco and Calderón, 1995]
    Carrasco, J. and Calderón, A. (1995). Regenerative randomization: Theory and application examples. In Proc. Performance ‘85 and 1995 ACM SIGMETRICS Conf., pages 241–262.Google Scholar
  9. [Çinlar, 1975]
    Çinlar, E. (1975). Introduction to Stochastic Processes. Prentice-Hall.Google Scholar
  10. [Choi et al., 1994]
    Choi, H., Kulkarni, V., and Trivedi, K. (1994). Markov regenerative stochastic Petri nets. Performance Evaluation, 20: 337–357.CrossRefGoogle Scholar
  11. [Choudhury et al., 1993]
    Choudhury, G., Lucantoni, D., and Whitt, W. (1993). Multi-dimensional transform inversion with applications to the transient M/G/1 queue. Technical report, ATamp;T Bell Labs, ATamp;T Bell Labs, Holmdel.Google Scholar
  12. [Ciardo et al., 1993]
    Ciardo, G., Blakemore, A., Chimento, P., Muppala, J., and Trivedi, K. (1993). Automated generation and analysis of Markov reward models using stochastic reward nets. In Meyer, C. and Plemmons, R., editors, Linear Algebra, Markov Chains, and Queueing Models,pages 145191. Springer-Verlag.Google Scholar
  13. [Ciardo et al., 1989]
    Ciardo, G., Muppala, J., and Trivedi, K. (1989). SPNP: stochastic Petri net package. In Proceedings of the Third International Workshop on Petri-nets and Performance Models, pages 142–151.Google Scholar
  14. [de Souza e Silva and Gail, 1986]
    de Souza e Silva, E. and Gail, H. (1986). Calculating cumulative operational time distributions of repairable computer systems. IEEE Trans. on Computers, C-35(4): 322–332.Google Scholar
  15. [de Souza e Silva and Gail, 1989]
    de Souza e Silva, E. and Gail, H. (1989). Calculating availability and performability measures of repairable computer systems using randomization. Journal of the ACM, 36 (1): 171–193.CrossRefGoogle Scholar
  16. [de Souza e Silva and Gail, 1990]
    de Souza e Silva, E. and Gail, H. (1990). Analyzing scheduled maintenance policies for repairable computer systems. IEEE Trans. on Computers, 39 (11): 1309–1324.CrossRefGoogle Scholar
  17. [de Souza e Silva and Gail, 1992]
    de Souza e Silva, E. and Gail, H. (1992). Per-formability analysis of computer systems: from model specification to solution. Performance Evaluation, 14: 157–196.CrossRefGoogle Scholar
  18. [de Souza e Silva and Gail, 1996]
    de Souza e Silva, E. and Gail, H. (1996). The uniformization method in performability analysis. Technical report, IBM Research Report RC 20383, Yorktown Heights, N. Y.Google Scholar
  19. [de Souza e Silva and Gail, 1998]
    de Souza e Silva, E. and Gail, H. (1998). An algorithm to calculate transient distributions of cumulative rate and impulse based reward. Communications in Statistics–Stochastic Models, 14 (3): 509–536.CrossRefGoogle Scholar
  20. [de Souza e Silva et al., 1995a]
    de Souza e Silva, E., Gail, H., and Campos, R. V. (1995a). Calculating transient distributions of cumulative reward. In Proc. Performance ‘85 and 1995 ACM SIGMETRICS Conf., pages 231–240.Google Scholar
  21. [de Souza e Silva et al., 1995b]
    de Souza e Silva, E., Gail, H., and Muntz, R. (1995b). Efficient solutions for a class of non-Markovian models. In Stewart, W., editor, Computations with Markov Chains,pages 483–506. Kluwer Academic Publishers.Google Scholar
  22. [de Souza e Silva et al., 1995c]
    de Souza e Silva, E., Gail, H., and Muntz, R. (1995c). Polling systems with server timeouts and their application to token passing networks. IEEE Trans. on Networking, 3 (5): 560–575.CrossRefGoogle Scholar
  23. [de Souza e Silva et al., 1998]
    de Souza e Silva, E., Gail, H., and Muntz, R. (1998). Gated time-limited polling systems. In Hasegawa, T., Takagi, H., and Takahashi, Y., editors, Performance and Management of Complex Ciommunication Networks, pages 253–274. Chapman amp; Hall.Google Scholar
  24. [Donatiello and Grassi, 1991]
    Donatiello, L. and Grassi, V. (1991). On evaluating the cumulative performance distribution of fault-tolerant computer systems. IEEE Trans. on Computers, 40 (11): 1301–1307.CrossRefGoogle Scholar
  25. [Golub and van Loan, 1989]
    Golub, G. and van Loan, C. (1989). Matrix Computations. Johns Hopkins University Press, second edition.Google Scholar
  26. [Goyal et al., 1986]
    Goyal, A., Carter, W., de Souza e Silva, E., Lavenberg, S., and Trivedi, K. (1986). The system availability estimator. In Proceedings of FTCS-16, pages 84–89.Google Scholar
  27. [Grassmann, 1977a]
    Grassmann, W. (1977a). Transient solutions in Markovian queueing systems. Computers ê4 Operations Research, 4: 47–53.CrossRefGoogle Scholar
  28. [Grassmann, 1977b]
    Grassmann, W. (1977b). Transient solutions in Markovian queues. European Journal of Operational Research, 1: 396–402.CrossRefGoogle Scholar
  29. [Grassmann, 1982]
    Grassmann, W. (1982). The GI/PH/1 queue: a method to find the transition matrix. INFOR, 20 (2): 144–156.Google Scholar
  30. [Grassmann, 1987]
    Grassmann, W. (1987). Means and variances of time averages in Markovian environments. European Journal of Operational Research, 31: 132–139.CrossRefGoogle Scholar
  31. [Grassmann, 1991]
    Grassmann, W. (1991). Finding transient solutions in Markovian event systems through randomization. In Stewart, W. J., editor, Numerical Solution of Markov Chains, pages 357–371. Marcel Dekker, Inc.Google Scholar
  32. [Grassmann, 1993]
    Grassmann, W. (1993). Means and variances in Markov reward systems. In Meyer, C. and Plemmons, R., editors, Linear Algebra, Markov Chains, and Queueing Models, pages 193–204. Springer-Verlag.Google Scholar
  33. [Heidelberger and Goyal, 1988]
    Heidelberger, P. and Goyal, A. (1988). Sensitivity analysis of continuous time Markov chains using uniformization. In fazeolla, G., Courtois, P., and Boxma, O., editors, Computer Performance and Reliability,pages 93–104. North-Holland.Google Scholar
  34. [Horn and Johnson, 1985]
    Horn, R. and Johnson, C. (1985). Matrix Analysis. Cambridge Univ. Press.Google Scholar
  35. [Islam and Ammar, 1991]
    Islam, S. and Ammar, H. (1991). Performability analysis of distributed real-time systems. IEEE Trans. on Computers, 40 (11): 1239–1251.CrossRefGoogle Scholar
  36. [Iyer et al., 1986]
    Iyer, B., Donatiello, L., and Heidelberger, P. (1986). Analysis of performability for stochastic models of fault-tolerant systems. IEEE Trans. on Computers, C-35(10): 902–907.Google Scholar
  37. [Jensen, 1953]
    Jensen, A. (1953). Markoff chains as an aid in the study of Markoff processes. Skandinaysk Aktuarietidskrift, 36: 87–91.Google Scholar
  38. [Johnson Jr. and Malek, 1988]
    Johnson Jr., A. and Malek, M. (1988). Survey of software tools for evaluating reliability, availability and serviceability. ACM Computing Surveys, 20: 227–271.CrossRefGoogle Scholar
  39. [Kulkarni et al., 1986]
    Kulkarni, V., Nicola, V., Smith, R., and Trivedi, K. (1986). Numerical evaluation of performability and job completion time in repairable fault-tolerant systems. In Proceedings of FTCS-16, pages 252–257.Google Scholar
  40. [Leguesdron et al., 1993]
    Leguesdron, P., Pellaumail, J., Rubino, G., and Sericola, B. (1993). Transient analysis of the M/M/1 queue. Technical report, IRISA, no. 720, Campus universitaire de Beaulieu.Google Scholar
  41. [Lindemann, 1991]
    Lindemann, C. (1991). An improved numerical algorithm for calculating steady-state solutions of deterministic and stochastic Petri net models. In Proceedings of the 4th International Workshop on Petri-nets and Performance Models, pages 176–184.Google Scholar
  42. [Lindemann, 1992]
    Lindemann, C. (1992). DSPNexpress: A software package for the efficient solution of deterministic and stochastic Petri nets. In Proceedings of the Sixth International Conference on Modelling Techniques and Tools for Computer Systems Performance Evaluation, pages 15–29, Edinburgh, Great Britain.Google Scholar
  43. [Lopez-Benitez and Trivedi, 1993]
    Lopez-Benitez, N. and Trivedi, K. (1993). Multiprocessor performability analysis. IEEE Trans. on Reliability, 42 (4): 579–587.CrossRefGoogle Scholar
  44. [Lucantoni et al., 1994]
    Lucantoni, D., Choudhury, G., and Whitt, W. (1994). The transient BMAP/G/1 queue. Communications in Statistics - Stochastic Models, 10 (1).Google Scholar
  45. [Matsunawa, 1985]
    Matsunawa, T. (1985). The exact and approximate distributions of linear combinations of selected order statistics from a uniform distribution. Ann. Inst. Statist. Math., 37: 1–16.CrossRefGoogle Scholar
  46. [Meyer, 1980]
    Meyer, J. (1980). On evaluating the performability of degradable computing systems. IEEE Trans. on Computers, C-29(8): 720–731.Google Scholar
  47. [Meyer, 1982]
    Meyer, J. (1982). Closed-form solutions of performability. IEEE Trans. on Computers, C-31(7): 648–657.Google Scholar
  48. [Meyer, 1992]
    Meyer, J. (1992). Performability: A retrospective and some pointers to the future. Performance Evaluation, 14: 139–156.CrossRefGoogle Scholar
  49. [Meyer, 1995]
    Meyer, J. (1995). Performability evaluation: Where it is and what lies ahead. In IPDS’95, pages 334–343.Google Scholar
  50. [Meyer et al., 1980]
    Meyer, J., Furchtgott, D., and Wu, L. (1980). Performability evaluation of the SIFT computer. IEEE Trans. on Computers, C29 (6): 501–509.CrossRefGoogle Scholar
  51. [Moler and van Loan, 1978]
    Moler, C. and van Loan, C. (1978). Nineteen dubious ways to compute the exponential of a matrix. SIAM Review, 20 (4): 801–836.CrossRefGoogle Scholar
  52. [Nabli and Sericola, 1996]
    Nabli, H. and Sericola, B. (1996). Performability analysis: a new algorithm. IEEE Trans. on Computers, 45 (4): 491–494.CrossRefGoogle Scholar
  53. [Pattipati et al., 1993]
    Pattipati, K., Li, Y., and Blom, H. (1993). A unified framework for the performability evaluation of fault-tolerant computer systems. IEEE Trans. on Computers, 42 (3): 312–326.CrossRefGoogle Scholar
  54. [Pattipati and Shah, 1990]
    Pattipati, K. and Shah, S. (1990). On the computational aspects of performability models of fault-tolerant computer systems. IEEE Trans. on Computers, 39 (6): 832–836.CrossRefGoogle Scholar
  55. [Philippe and Sidge, 1995]
    Philippe, B. and Sidge, R. (1995). Transient solutions of Markov processes by Krylov subspaces. In Stewart, W., editor, Computations with Markov Chains,pages 95–119. Kluwer Academic Publishers.Google Scholar
  56. [Prodromides and Sanders, 1993]
    Prodromides, K. and Sanders, W. (1993). Performability evaluation of CSMA/CD and CSMA/CR protocols under transient fault conditions. IEEE Trans. on Reliability, 42 (1): 116–127.CrossRefGoogle Scholar
  57. [Puri, 1971]
    Puri, P. (1971). A method for studying the integral functionals of stochastic processes with applications: I. Markov chain case. Journal of Applied Probability, 8 (10): 331–343.CrossRefGoogle Scholar
  58. [Qureshi and Sanders, 1994]
    Qureshi, M. and Sanders, W. (1994). Reward model solution methods with impulse and rate rewards: An algorithm and numerical results. Performance Evaluation, 20 (4): 413–436.CrossRefGoogle Scholar
  59. [Reibman et al., 1989]
    Reibman, A., Trivedi, K., Kumar, S., and Ciardo, G. (1989). Analysis of stiff Markov chains. ORSA Journal on Computing, 1 (2): 126–133.CrossRefGoogle Scholar
  60. [Ren and Kobayashi, 1995]
    Ren, Q. and Kobayashi, H. (1995). Transient solutions for the buffer behavior in statistical multiplexing. Performance Evaluation, 23: 65–87.CrossRefGoogle Scholar
  61. [Ross, 1983]
    Ross, S. (1983). Stochastic Processes. John Wiley amp; Sons.Google Scholar
  62. [Ross, 1987]
    Ross, S. (1987). Approximating transition probabilities and mean occupation times in continuous-time Markov chains. Probability in the Engineering and Informational Sciences, 1: 251–264.CrossRefGoogle Scholar
  63. [Rubino and Sericola, 1993]
    Rubino, G. and Sericola, B. (1993). Interval availability distribution computation. In Proceedings of FTCS-23, pages 48–55.Google Scholar
  64. [Saad, 1992]
    Saad, Y. (1992). Analysis of some Krylov subspace approximations to the matrix exponential operator. SIAM Journal on Numerical Analysis, 29 (1): 208–227.CrossRefGoogle Scholar
  65. [Saad, 1995]
    Saad, Y. (1995). Iterative Methods for Sparse Linear Systems. PWS Publishing Company.Google Scholar
  66. [Sanders et al., 1995]
    Sanders, W., II, W. O., Qureshi, M., and Widjanarko, F. (1995). The UltraSAN modeling environment. Performance Evaluation, 24: 89–115.CrossRefGoogle Scholar
  67. [Sericola, 1999]
    Sericola, B. (1999). Availability analysis and stationary regime detection of Markov processes. IEEE Trans. on Computers, (to appear).Google Scholar
  68. [Stewart, 1991]
    Stewart, W. (1991). MARCA Markov chain analyzer, a software package for Markov chains. In Numerical Solution of Markov Chains, pages 37–61. Marcel Dekker, Inc.Google Scholar
  69. [Stewart, 1994]
    Stewart, W. (1994). Introduction to the Numerical Solution of Markov Chains. Princeton University Press.Google Scholar
  70. [Tai and Meyer, 1996]
    Tai, A. and Meyer, J. (1996). Performability management in distributed database systems: an adaptive concurrency control protocol. In Proceedings of the 4th Int’l Workshop on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, pages 212–216, San Jose, California.Google Scholar
  71. [Tanaka et al., 1995]
    Tanaka, T., Hashida, O., and Takahashi, Y. (1995). Transient analysis of fluid model for ATM statistical multiplexer. Performance Evaluation, 23: 145–162.CrossRefGoogle Scholar
  72. [Trivedi, 1982]
    Trivedi, K. (1982). Probability and Statistics with Reliability, Queueing and Computer Science Applications. Prentice-Hall.Google Scholar
  73. [van As, 1986]
    van As, H. (1986). Transient analysis of Markovian queueing systems and its application to congestion control modeling. IEEE Journal on Selected Areas in Communications, SAC-4(6): 891–904.Google Scholar
  74. [van Dijk, 1990]
    van Dijk, N. (1990). The importance of bias-terms for error bounds and comparison results. In The First International Conference on the Numerical Solution of Markov Chains, pages 640–663.Google Scholar
  75. [van Dijk, 1992a]
    van Dijk, N. (1992a). Approximate uniformization for continuous-time Markov chains with an application to performability analysis. Stochastic Processes and Their Applications, 40 (2): 339–357.CrossRefGoogle Scholar
  76. [van Dijk, 1992b]
    van Dijk, N. (1992b). Uniformization for nonhomogeneous Markov chains. Operations Research Letters, 12: 283–291.CrossRefGoogle Scholar
  77. [van Moorsel and Sanders, 1994]
    van Moorsel, A. and Sanders, W. (1994). Adaptive uniformization. Communications in Statistics–Stochastic Models, 10 (3): 619–648.CrossRefGoogle Scholar
  78. [Weisberg, 1971]
    Weisberg, H. (1971). The distribution of linear combinations of order statistics from the uniform distribution. Annals Math. Stat., 42: 704709.Google Scholar
  79. [Yoon and Shanthikumar, 1989]
    Yoon, B. and Shanthikumar, J. (1989). Bounds and approximations for the transient behavior of continuous time Markov chains. Probability in the Engineering and Informational Sciences, 3: 175–198.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2000

Authors and Affiliations

  • Edmundo de Souza e Silva
    • 1
  • H. Richard Gail
    • 2
  1. 1.NCE and CS DepartmentFederal University of Rio de JaneiroRio de JaneiroBrazil
  2. 2.IBMT. J. Watson Research CenterYorktown HeightsUSA

Personalised recommendations