Queueing Systems

, Volume 92, Issue 1–2, pp 173–200 | Cite as

A sequential update algorithm for computing the stationary distribution vector in upper block-Hessenberg Markov chains

  • Hiroyuki MasuyamaEmail author


This paper proposes a new algorithm for computing the stationary distribution vector in continuous-time upper block-Hessenberg Markov chains. To this end, we consider the last-block-column-linearly-augmented (LBCL-augmented) truncation of the (infinitesimal) generator of the upper block-Hessenberg Markov chain. The LBCL-augmented truncation is a linearly augmented truncation such that the augmentation distribution has its probability mass only on the last block column. We first derive an upper bound for the total variation distance between the respective stationary distribution vectors of the original generator and its LBCL-augmented truncation. Based on the upper bound, we then establish a series of linear fractional programming (LFP) problems to obtain augmentation distribution vectors such that the bound converges to zero. Using the optimal solutions of the LFP problems, we construct a matrix-infinite-product (MIP) form of the original (i.e., not approximate) stationary distribution vector and develop a sequential update algorithm for computing the MIP form. Finally, we demonstrate the applicability of our algorithm to BMAP/M/\(\infty \) queues and M/M/s retrial queues.


Upper block-Hessenberg Markov chain Level-dependent M/G/1-type Markov chain Matrix-infinite-product (MIP) form Last-block-column-linearly-augmented truncation (LBCL-augmented truncation) BMAP/M/\(\infty \) queue M/M/s retrial queue 

Mathematics Subject Classification

60J22 60K25 



The author thanks Mr. Masatoshi Kimura and Dr. Tetsuya Takine for providing the counterexample presented in Sect. 2.3. The author also thanks an anonymous referee for his/her valuable comments that helped to improve the paper.


  1. 1.
    Anderson, W.J.: Continuous-Time Markov Chains: An Applications-Oriented Approach. Springer, New York (1991)Google Scholar
  2. 2.
    Baumann, H., Sandmann, W.: Numerical solution of level dependent quasi-birth-and-death processes. Procedia Comput. Sci. 1(1), 1561–1569 (2012)Google Scholar
  3. 3.
    Brémaud, P.: Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues. Springer, New York (1999)Google Scholar
  4. 4.
    Bright, L., Taylor, P.G.: Calculating the equilibrium distribution in level dependent quasi-birth-and-death processes. Stoch. Models 11(3), 497–525 (1995)Google Scholar
  5. 5.
    Falin, G.I., Templeton, J.G.C.: Retrial Queues. Chapman & Hall, London (1997)Google Scholar
  6. 6.
    Gibson, D., Seneta, E.: Augmented truncations of infinite stochastic matrices. J. Appl. Probab. 24(3), 600–608 (1987)Google Scholar
  7. 7.
    Horn, R.A., Johnson, C.R.: Matrix Analysis, 2nd edn. Cambridge University Press, Cambridge (2013)Google Scholar
  8. 8.
    Kim, J., Kim, J., Kim, B.: Tail asymptotics of the queue size distribution in the M/M/\(m\) retrial queue. J. Comput. Appl. Math. 236(14), 3445–3460 (2012)Google Scholar
  9. 9.
    Kimura, M., Takine, T.: Private communication (2016)Google Scholar
  10. 10.
    Kimura, M., Takine, T.: Computing the conditional stationary distribution in Markov chains of level-dependent M/G/1-type. Stoch. Models 34(2), 207–238 (2018)Google Scholar
  11. 11.
    Klimenok, V., Dudin, A.: Multi-dimensional asymptotically quasi-Toeplitz Markov chains and their application in queueing theory. Queueing Syst. 54(4), 245–259 (2006)Google Scholar
  12. 12.
    Kontoyiannis, I., Meyn, S.P.: On the \(f\)-norm ergodicity of Markov processes in continuous time. Electron. Commun. Probab. 21, Paper no. 77, 1–10 (2016)Google Scholar
  13. 13.
    Latouche, G., Ramaswami, V.: Introduction to Matrix Analytic Methods in Stochastic Modeling. SIAM, Philadelphia, PA (1999)Google Scholar
  14. 14.
    Le Boudec, J.Y.: An efficient solution method for Markov models of ATM links with loss priorities. IEEE J. Sel. Areas Commun. 9(3), 408–417 (1991)Google Scholar
  15. 15.
    Li, H., Zhao, Y.Q.: Stochastic block-monotonicity in the approximation of the stationary distribution of infinite Markov chains. Stoch. Models 16(2), 313–333 (2000)Google Scholar
  16. 16.
    Li, Q.L., Lian, Z., Liu, L.: An \(RG\)-factorization approach for a BMAP/M/1 generalized processor-sharing queue. Stoch. Models 21(2–3), 507–530 (2005)Google Scholar
  17. 17.
    Liu, Y., Li, W., Masuyama, H.: Error bounds for augmented truncation approximations of continuous-time Markov chains. Oper. Res. Lett. 46(4), 409–413 (2018)Google Scholar
  18. 18.
    Lucantoni, D.M.: New results on the single server queue with a batch Markovian arrival process. Stoch. Models 7(1), 1–46 (1991)Google Scholar
  19. 19.
    Masuyama, H.: Limit formulas for the normalized fundamental matrix of the northwest-corner truncation of Markov chains: Matrix-infinite-product-form solutions of block-Hessenberg Markov chains (2016). Preprint arXiv:1603.07787
  20. 20.
    Masuyama, H.: Error bounds for last-column-block-augmented truncations of block-structured Markov chains. J. Oper. Res. Soc. Jpn. 60(3), 271–320 (2017)Google Scholar
  21. 21.
    Phung-Duc, T., Masuyama, H., Kasahara, S., Takahashi, Y.: A simple algorithm for the rate matrices of level-dependent QBD processes. In: Proceedings of the 5th International Conference on Queueing Theory and Network Applications (QTNA2010), pp. 46–52. ACM, New York (2010)Google Scholar
  22. 22.
    Shin, Y.W.: Fundamental matrix of transient QBD generator with finite states and level dependent transitions. Asia-Pac. J. Oper. Res. 26(5), 697–714 (2009)Google Scholar
  23. 23.
    Shin, Y.W., Pearce, C.E.M.: An algorithmic approach to the Markov chain with transition probability matrix of upper block-Hessenberg form. Korean J. Comput. Appl. Math. 5(2), 361–384 (1998)Google Scholar
  24. 24.
    Takine, T.: Analysis and computation of the stationary distribution in a special class of Markov chains of level-dependent M/G/1-type and its application to BMAP/M/\(\infty \) and BMAP/M/\(c+M\) queues. Queueing Syst. 84(1–2), 49–77 (2016)Google Scholar
  25. 25.
    Yajima, M., Phung-Duc, T., Masuyama, H.: The stability condition of BMAP/M/\(\infty \) queues. In: Proceedings of the 11th International Conference on Queueing Theory and Network Applications (QTNA2016), Article no. 5, pp. 1–6. ACM, New York (2016)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019
corrected publication 2019

Authors and Affiliations

  1. 1.Department of Systems Science, Graduate School of InformaticsKyoto UniversityKyotoJapan

Personalised recommendations