Use of Characteristic Roots for Solving Infinite State Markov Chains

  • H. Richard Gail
  • Sidney L. Hantler
  • B. Alan Taylor
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 24)

Abstract

In this chapter, our interest is in determining the stationary distribution of an irreducible positive recurrent Markov chain with an infinite state space. In particular, we consider the solution of such chains using roots or zeros. A root of an equation f (z) = 0 is a zero of the function f (z),and so for notational convenience we use the terms root and zero interchangeably. A natural class of chains that can be solved using roots are those with a transition matrix that has an almost Toeplitz structure. Specifically, the classes of M/G/1 type chains and G/M/1 type chains lend themselves to solution methods that utilize roots. In the M/G/1 case, it is natural to transform the stationary equations and solve for the stationary distribution using generating functions. However, in the G/M/1 case the stationary probability vector itself is given directly in terms of roots or zeros. Although our focus in this chapter is on the discrete-time case, we will show how the continuous-time case can be handled by the same techniques. The M/G/1 and G/M/1 classes can be solved using the matrix analytic method [Neuts, 1981, Neuts, 1989], and we will also discuss the relationship between the approach using roots and this method.

Keywords

Markov Chain Unit Disk Characteristic Root Simple Zero Open Unit Disk 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [Akar and Sohraby, 1997a]
    Akar, N. and Sohraby, K. (1997a). An invariant subspace approach in M/G/1 and G/M/1 type Markov chains. Communications in Statistics-Stochastic Models, 13: 381–416.CrossRefGoogle Scholar
  2. [Akar and Sohraby, 1997b]
    Akar, N. and Sohraby, K. (1997b). Matrix geometric solutions in M/G/1 type Markov chains with multiple boundaries: A generalized state-space approach. In Ramaswami, V. and Wirth, P. E., editors, Teletraf c Contributions for the Information Age,pages 487–496. Elsevier.Google Scholar
  3. [Andrew et al., 1993]
    Andrew, A. L., Chu, K.-W. E., and Lancaster, P. (1993). Derivatives of eigenvalues and eigenvectors of matrix functions. SIAM Journal on Matrix Analysis and Applications, 14 (4): 903–926.CrossRefGoogle Scholar
  4. [Bailey, 1954]
    Bailey, N. T. J. (1954). On queueing processes with bulk service. Journal of the Royal Statistical Society, Series B, 16: 80–87.Google Scholar
  5. [Ball et al., 1990]
    Ball, J. A., Gohberg, I., and Rodman, L. (1990). Interpolation of Rational Matrix Functions. Birkhäuser.Google Scholar
  6. [Berenstein and Gay, 1991]
    Berenstein, C. A. and Gay, R. (1991). Complex Variables: An Introduction. Springer-Verlag.Google Scholar
  7. [Bini and Meini, 1996]
    Bini, D. and Meini, B. (1996). On the solution of a nonlinear matrix equation arising in queueing problems. SIAM Journal on Matrix Analysis and Applications, 17: 906–926.CrossRefGoogle Scholar
  8. [Bini and Meini, 1998]
    Bini, D. and Meini, B. (1998). Using displacement structure for solving non-skip-free M/G/1 type Markov chains. In Alfa, A. S. and Chakravarthy, S. R., editors, Advances in Matrix Analytic Methods for Stochastic Models, pages 17–37. Notable Publications.Google Scholar
  9. [Coffman et al., 1983]
    Coffman, Jr., E. G., Fayolle, G., and Mitrani, I. (1983). Two queues with alternating service periods. In Courtois, P. J. and Latouche, G., editors, Performance 87,pages 227–237. North-Holland.Google Scholar
  10. [Cohen and Down, 1996]
    Cohen, J. W. and Down, D. G. (1996). On the role of Rouché’s theorem in queueing analysis. Queueing Systems, 23: 281–291.CrossRefGoogle Scholar
  11. [Daigle and Lucantoni, 1991]
    Daigle, J. N. and Lucantoni, D. M. (1991). Queueing systems having phase-dependent arrival and service rates. In Stewart, W. J., editor, Numerical Solution of Markov Chains,pages 161–202. Marcel Dekker.Google Scholar
  12. [Dukhovny, 1994]
    Dukhovny, A. (1994). Multiple roots in some equations of queueing theory. Communications in Statistics-Stochastic Models, 10 (2): 519–524.CrossRefGoogle Scholar
  13. [Dukhovny, 1998]
    Dukhovny, A. (1998). Multiple roots in queueing equations: How realistic must examples get? Communications in Statistics-Stochastic Models, 14 (3): 763–765.CrossRefGoogle Scholar
  14. [Feller, 1968]
    Feller, W. (1968). An Introduction to Probability Theory and its Applications,volume I. John Wiley und Sons, 3rd edition.Google Scholar
  15. [Feller, 1971]
    Feller, W. (1971). An Introduction to Probability Theory and its Applications,volume II. John Wiley Sc Sons, 2nd edition.Google Scholar
  16. [Franklin, 1968]
    Franklin, J. N. (1968). Matrix Theory. Prentice-Hall.Google Scholar
  17. [Friedberg et al., 1989]
    Friedberg, S. H., Insel, A. J., and Spence, L. E. (1989). Linear Algebra. Prentice-Hall, 2nd edition.Google Scholar
  18. [Gail et al., 1994]
    Gail, H. R., Hantler, S. L., Konheim, A. G., and Taylor, B. A. (1994). An analysis of a class of telecommunications models. Performance Evaluation, 21: 151–161.CrossRefGoogle Scholar
  19. [Gail et al., 1995]
    Gail, H. R., Hantler, S. L., Sidi, M., and Taylor, B. A. (1995). Linear independence of root equations for M/G/1 type Markov chains Queueing Systems, 20: 321–339.CrossRefGoogle Scholar
  20. [Gail et al., 1992]
    Gail, H. R., Hantler, S. L., and Taylor, B. A. (1992). On a preemptive Markovian queue with multiple servers and two priority classes. Mathematics of Operations Research, 17 (2): 365–391.CrossRefGoogle Scholar
  21. [Gail et al., 1996]
    Gail, H. R., Hantler, S. L., and Taylor, B. A. (1996). Spectral analysis of M/G/1 and G/M/1 type Markov chains. Advances in Applied Probability, 28: 114–165.CrossRefGoogle Scholar
  22. [Gail et al., 1997]
    Gail, H. R., Hantler, S. L., and Taylor, B. A. (1997). Nonskip-free M/G/1 and G/M/1 type Markov chains. Advances in Applied Probability, 29: 733–758.CrossRefGoogle Scholar
  23. [Gail et al., 1998]
    Gail, H. R., Hantler, S. L., and Taylor, B. A. (1998). Matrix-geometric invariant measures for G/M/1 type Markov chains. Communications in Statistics-Stochastic Models, 14 (3): 537–569.CrossRefGoogle Scholar
  24. [Gantmacher, 1959]
    Gantmacher, F. R. (1959). Applications of the Theory of Matrices. Interscience.Google Scholar
  25. [Glicksberg, 1976]
    Glicksberg, I. (1976). A remark on Rouché’s theorem. The American Mathematical Monthly, 83: 186–187.CrossRefGoogle Scholar
  26. [Gohberg et al., 1982]
    Gohberg, I., Lancaster, P., and Rodman, L. (1982). Matrix Polynomials. Academic Press.Google Scholar
  27. [Gohberg and Feldman, 1974]
    Gohberg, I. C. and Fel’dman, I. A. (1974). Convolution Equations and Projection Methods for their Solution. American Mathematical Society.Google Scholar
  28. [Goldberg, 1958]
    Goldberg, S. (1958). Introduction to Difference Equations. John Wiley und Sons.Google Scholar
  29. [Grassmann, 1977a]
    Grassmann, W. K. (1977a). Transient solutions in Markovian queueing systems. Computers und Operations Research, 4: 47–53.CrossRefGoogle Scholar
  30. [Grassmann, 1977b]
    Grassmann, W. K. (1977b). Transient solutions in Marko-vian queues. European Journal of Operational Research, 1: 396–402.CrossRefGoogle Scholar
  31. [Grassmann, 1991]
    Grassmann, W. K. (1991). Finding transient solutions in Markovian event systems through randomization. In Stewart, W. J., editor, Numerical Solution of Markov Chains,pages 357–371. Marcel Dekker.Google Scholar
  32. [Grassmann, 1998]
    Grassmann, W. K. (1998). Finding test data for solving the equilibrium equations of Markov chains with repeating columns. In Alfa, A. S. and Chakravarthy, S. R., editors, Advances in Matrix Analytic Methods for Stochastic Models, pages 261–278. Notable Publications.Google Scholar
  33. [Grassmann and Heyman, 1990]
    Grassmann, W. K. and Heyman, D. P. (1990). Equilibrium distribution of block-structured Markov chains with repeating rows. Journal of Applied Probability, 27: 557–576.CrossRefGoogle Scholar
  34. [Grassmann and Heyman, 1993]
    Grassmann, W. K. and Heyman, D. P. (1993). Computation of steady-state probabilities for infinite-state Markov chains with repeating rows. ORSA Journal on Computing, 5: 292–303.CrossRefGoogle Scholar
  35. [H. R. Gail and Taylor, 1998]
    H. R. Gail, S. L. H. and Taylor, B. A. (1998). M/G/1 type Markov chains with rational generating functions. In Alfa, A. S. and Chakravarthy, S. R., editors, Advances in Matrix Analytic Methods for Stochastic Models, pages 1–16. Notable Publications.Google Scholar
  36. [Horn and Johnson, 1985]
    Horn, R. A. and Johnson, C. R. (1985). Matrix Analysis. Cambridge University Press.Google Scholar
  37. [Jensen, 1953]
    Jensen, A. (1953). Markoff chains as an aid in the study of markoff processes. Skandinaysk Aktuarietidskrift, 36: 87–91.Google Scholar
  38. [Keilson and Wishart, 1964]
    Keilson, J. and Wishart, D. M. G. (1964). A central limit theorem for processes defined on a finite Markov chains. In Proceedings of the Cambridge Philosophical Society, volume 60, pages 547–567.Google Scholar
  39. [Kemeny et al., 1966]
    Kemeny, J. G., Snell, J. L., and Knapp, A. W. (1966). Denumerable Markov Chains. Van Nostrand.Google Scholar
  40. [Kingman, 1961]
    Kingman, J. F. C. (1961). A convexity property of positive matrices. Quarterly Journal of Mathematics, 12: 283–284.CrossRefGoogle Scholar
  41. [Kleinrock, 1975]
    Kleinrock, L. (1975). Queueing Systems Volume I: Theory. John Wiley und Sons.Google Scholar
  42. [Konheim and Reiser, 1976]
    Konheim, A. G. and Reiser, M. (1976). A queueing model with finite waiting room and blocking. Journal of the ACM,23(2):328341.Google Scholar
  43. [Konheim and Reiser, 1978]
    Konheim, A. G. and Reiser, M. (1978). Finite capacity queueing systems with applications in computer modeling. SIAM Journal on Computing, 7 (2): 210–229.CrossRefGoogle Scholar
  44. [Latouche and Ramaswami, 1993]
    Latouche, G. and Ramaswami, V. (1993). A logarithmic reduction algorithm for quasi-birth-death processes. Journal of Applied Probability, 30: 650–674.CrossRefGoogle Scholar
  45. [Meini, 1997]
    Meini, B. (1997). An improved FFT-based version of Ramaswami’s formula. Communications in Statistics—Stochastic Models, 13: 23238.Google Scholar
  46. [Meyer, 1989]
    Meyer, C. D. (1989). Stochastic complementation, uncoupling Markov chains, and the theory of nearly reducible systems. SIAM Review, 31 (2): 40–272.CrossRefGoogle Scholar
  47. [Mitrani and Chakka, 1995]
    Mitrani, I. and Chakka, R. (1995). Spectral expansion solution for a class of Markov models: Application and comparison with the matrix-geometric method. Performance Evaluation, 23 (3): 241–260.CrossRefGoogle Scholar
  48. [Mitrani and Mitra, 1992]
    Mitrani, I. and Mitra, D. (1992). A spectral expansion method for random walks on semi-infinite strips. In Iterative Methods in Linear Algebra,pages 141–149. North-Holland.Google Scholar
  49. [Mitrany and Avi-Itzhak, 1968]
    Mitrany, I. L. and Avi-Itzhak, B. (1968). A many-server queue with service interruptions. Operations Research, 16 (3): 628–638.CrossRefGoogle Scholar
  50. [Neuts, 1981]
    Neuts, M. F. (1981). Matrix-Geometric Solutions in Stochastic Models. Johns Hopkins University Press.Google Scholar
  51. [Neuts, 1989]
    Neuts, M. F. (1989). Structured Stochastic Matrices of M/G/1 Type and Their Applications. Marcel Dekker.Google Scholar
  52. [Ramaswami, 1988]
    Ramaswami, V. (1988). A stable recursion for the steady state vector in Markov chains of M/G/1 type. Communications in Statistics—Stochastic Models, 4: 183–188.CrossRefGoogle Scholar
  53. [Takács, 1962]
    Takács, L. (1962). Introduction to the Theory of Queues. Oxford University Press.Google Scholar
  54. [Wuyts and Boel, 1998]
    Wuyts, K. and Boel, R. K. (1998). Efficient matrix geometric methods for B-ISDN by using the spectral decomposition of the rate matrices. In Alfa, A. S. and Chakravarthy, S. R., editors, Advances in Matrix Analytic Methods for Stochastic Models,pages 341–359. Notable Publications.Google Scholar

Copyright information

© Springer Science+Business Media New York 2000

Authors and Affiliations

  • H. Richard Gail
    • 1
  • Sidney L. Hantler
    • 1
  • B. Alan Taylor
    • 2
  1. 1.IBM Thomas J. Watson Research CenterYorktown HeightsUSA
  2. 2.Department of MathemticsThe University of MichiganAnn ArborUSA

Personalised recommendations