Queueing Systems

, Volume 92, Issue 1–2, pp 173–200

A sequential update algorithm for computing the stationary distribution vector in upper block-Hessenberg Markov chains

Article

Abstract

This paper proposes a new algorithm for computing the stationary distribution vector in continuous-time upper block-Hessenberg Markov chains. To this end, we consider the last-block-column-linearly-augmented (LBCL-augmented) truncation of the (infinitesimal) generator of the upper block-Hessenberg Markov chain. The LBCL-augmented truncation is a linearly augmented truncation such that the augmentation distribution has its probability mass only on the last block column. We first derive an upper bound for the total variation distance between the respective stationary distribution vectors of the original generator and its LBCL-augmented truncation. Based on the upper bound, we then establish a series of linear fractional programming (LFP) problems to obtain augmentation distribution vectors such that the bound converges to zero. Using the optimal solutions of the LFP problems, we construct a matrix-infinite-product (MIP) form of the original (i.e., not approximate) stationary distribution vector and develop a sequential update algorithm for computing the MIP form. Finally, we demonstrate the applicability of our algorithm to BMAP/M/$$\infty$$ queues and M/M/s retrial queues.

Keywords

Upper block-Hessenberg Markov chain Level-dependent M/G/1-type Markov chain Matrix-infinite-product (MIP) form Last-block-column-linearly-augmented truncation (LBCL-augmented truncation) BMAP/M/$$\infty$$ queue M/M/s retrial queue

60J22 60K25

References

1. 1.
Anderson, W.J.: Continuous-Time Markov Chains: An Applications-Oriented Approach. Springer, New York (1991)Google Scholar
2. 2.
Baumann, H., Sandmann, W.: Numerical solution of level dependent quasi-birth-and-death processes. Procedia Comput. Sci. 1(1), 1561–1569 (2012)Google Scholar
3. 3.
Brémaud, P.: Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues. Springer, New York (1999)Google Scholar
4. 4.
Bright, L., Taylor, P.G.: Calculating the equilibrium distribution in level dependent quasi-birth-and-death processes. Stoch. Models 11(3), 497–525 (1995)Google Scholar
5. 5.
Falin, G.I., Templeton, J.G.C.: Retrial Queues. Chapman & Hall, London (1997)Google Scholar
6. 6.
Gibson, D., Seneta, E.: Augmented truncations of infinite stochastic matrices. J. Appl. Probab. 24(3), 600–608 (1987)Google Scholar
7. 7.
Horn, R.A., Johnson, C.R.: Matrix Analysis, 2nd edn. Cambridge University Press, Cambridge (2013)Google Scholar
8. 8.
Kim, J., Kim, J., Kim, B.: Tail asymptotics of the queue size distribution in the M/M/$$m$$ retrial queue. J. Comput. Appl. Math. 236(14), 3445–3460 (2012)Google Scholar
9. 9.
Kimura, M., Takine, T.: Private communication (2016)Google Scholar
10. 10.
Kimura, M., Takine, T.: Computing the conditional stationary distribution in Markov chains of level-dependent M/G/1-type. Stoch. Models 34(2), 207–238 (2018)Google Scholar
11. 11.
Klimenok, V., Dudin, A.: Multi-dimensional asymptotically quasi-Toeplitz Markov chains and their application in queueing theory. Queueing Syst. 54(4), 245–259 (2006)Google Scholar
12. 12.
Kontoyiannis, I., Meyn, S.P.: On the $$f$$-norm ergodicity of Markov processes in continuous time. Electron. Commun. Probab. 21, Paper no. 77, 1–10 (2016)Google Scholar
13. 13.
Latouche, G., Ramaswami, V.: Introduction to Matrix Analytic Methods in Stochastic Modeling. SIAM, Philadelphia, PA (1999)Google Scholar
14. 14.
Le Boudec, J.Y.: An efficient solution method for Markov models of ATM links with loss priorities. IEEE J. Sel. Areas Commun. 9(3), 408–417 (1991)Google Scholar
15. 15.
Li, H., Zhao, Y.Q.: Stochastic block-monotonicity in the approximation of the stationary distribution of infinite Markov chains. Stoch. Models 16(2), 313–333 (2000)Google Scholar
16. 16.
Li, Q.L., Lian, Z., Liu, L.: An $$RG$$-factorization approach for a BMAP/M/1 generalized processor-sharing queue. Stoch. Models 21(2–3), 507–530 (2005)Google Scholar
17. 17.
Liu, Y., Li, W., Masuyama, H.: Error bounds for augmented truncation approximations of continuous-time Markov chains. Oper. Res. Lett. 46(4), 409–413 (2018)Google Scholar
18. 18.
Lucantoni, D.M.: New results on the single server queue with a batch Markovian arrival process. Stoch. Models 7(1), 1–46 (1991)Google Scholar
19. 19.
Masuyama, H.: Limit formulas for the normalized fundamental matrix of the northwest-corner truncation of Markov chains: Matrix-infinite-product-form solutions of block-Hessenberg Markov chains (2016). Preprint arXiv:1603.07787
20. 20.
Masuyama, H.: Error bounds for last-column-block-augmented truncations of block-structured Markov chains. J. Oper. Res. Soc. Jpn. 60(3), 271–320 (2017)Google Scholar
21. 21.
Phung-Duc, T., Masuyama, H., Kasahara, S., Takahashi, Y.: A simple algorithm for the rate matrices of level-dependent QBD processes. In: Proceedings of the 5th International Conference on Queueing Theory and Network Applications (QTNA2010), pp. 46–52. ACM, New York (2010)Google Scholar
22. 22.
Shin, Y.W.: Fundamental matrix of transient QBD generator with finite states and level dependent transitions. Asia-Pac. J. Oper. Res. 26(5), 697–714 (2009)Google Scholar
23. 23.
Shin, Y.W., Pearce, C.E.M.: An algorithmic approach to the Markov chain with transition probability matrix of upper block-Hessenberg form. Korean J. Comput. Appl. Math. 5(2), 361–384 (1998)Google Scholar
24. 24.
Takine, T.: Analysis and computation of the stationary distribution in a special class of Markov chains of level-dependent M/G/1-type and its application to BMAP/M/$$\infty$$ and BMAP/M/$$c+M$$ queues. Queueing Syst. 84(1–2), 49–77 (2016)Google Scholar
25. 25.
Yajima, M., Phung-Duc, T., Masuyama, H.: The stability condition of BMAP/M/$$\infty$$ queues. In: Proceedings of the 11th International Conference on Queueing Theory and Network Applications (QTNA2016), Article no. 5, pp. 1–6. ACM, New York (2016)Google Scholar