Abstract
In this chapter, the censoring technique is applied to be able to deal with any irreducible block-structured Markov chain, which is either discrete-time or continuous-time. The R-, U- and G-measures are iteratively defined from two different censored directions: UL-type and LU-type. An important censoring invariance for the R- and G-measures is obtained. Using the censoring invariance, the Wiener-Hopf equations are derived, and then the UL- and UL-types of RG-factorizations are given. The stationary probability vector is given an R-measure expression; while the transient probability can be computed by means of the R-, U- and G-measures. Finally, the A- and B-measures are proposed in order to discuss the state classification of the block-structured Markov chain.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bini D.A., G. Latouche and B. Meini (2005). Numerical Methods for Structured Markov Chains, Oxford University Press
Dudin A. and V.I. Klimenok (1999). Multi-dimensional quasi-Toeplitz Markov chains. Journal of Applied Mathematics and Stochastic Analysis 12: 393–415
Dudin A., C. Kim and V.I. Klimenok (2008). Markov chains with hybrid repeating rows-upper-Hessenberg, quasi-Toeplitz structure of the block transition probability matrix. Journal of Applied Probability 45: 211–225
Freedman D. (1983) Approximating Countable Markov Chains (Second Edition), Springer: New York
Grassmann W.K. and D.P. Heyman (1990). Equilibrium distribution of block-structured Markov chains with repeating rows. Journal of Applied Probability 27: 557–576
Grassmann W.K. (1993). Means and variances in Markov reward systems. In Linear Algebra, Markov Chains, and Queueing Models, C.D. Meyer and R.J. Plemmons (eds.), Springer-Verlag: New York, 193–204
Grassmann W.K. and D.P. Heyman (1993). Computation of steady-state probabilities for infinite-state Markov chains with repeating rows. ORSA Journal on Computing 5: 282–303
Heyman D.P. (1995). A decomposition theorem for infinite stochastic matrices. Journal of Applied Probability 32: 893–901
Karlin S. and H.W. Taylor (1975). A First Course in Stochastic Processes (Second Edition), Academic Press, INC.
Karlin S. and H.W. Taylor (1981). A Second Course in Stochastic Processes, Academic Press, INC.
Kemeny J.G., J.L. Snell and A.W. Knapp (1976). Denumerable Markov Chains (Second Edition), Springer-Verlag: NewYork
Klimenok V.I. and A. Dudin (2006). Multi-dimensional asymptotically quasi-Toeplitz Markov chains and their application in queueing theory. Queueing Systems 54: 245–259
Latouche G. (1992). Algorithms for infinite Markov chains with repeating columns. In Linear algebra, Markov chains, and queueing models, Minneapolis, MN, 231–265; IMA Vol. Math. Appl., 48, Springer: New York, 1993
Latouche G. and V. Ramaswami (1999). Introduction to Matrix Analytic Methods in Stochastic Models, ASA-SIAM
Lévy P. (1951). Systèmes markoviens et stationaires. Cas dénombrable. Ann. Sci. École Norm. Sup. 68: 327–381
Lévy P. (1952). Complément à l’étude des processus de Markoff. Ann. Sci. École Norm. Sup. 69: 203–212
Lévy P. (1958). Processus markoviens et stationaires. Cas dénombrable. Ann. Inst. H. Poincaré 18: 7–25
Li Q.L. and J. Cao (2004). Two types of RG-factorizations of quasi-birth-and-death processes and their applications to stochastic integral functionals. Stochastic Models 20:299–340
Li Q.L. and L.M. Liu (2004). An algorithmic approach on sensitivity analysis of perturbed QBD processes. Queueing Systems 48: 365–397
Li Q.L. and Y.Q. Zhao (2002). A constructive method for finding β-invariant measures for transition matrices of M/G/1 type. In Matrix Analytic Methods: Theory and Applications, G. Latouche and P.G. Taylor (eds), World Scientific: New Jersey, 237–263
Li Q.L. and Y.Q. Zhao (2002). The RG-factorizations in block-structured Markov renewal processes with applications. Technical Report 381, Laboratory for Research in Statistics and Probability, Carleton University and University of Ottawa, Canada, 1–40
Li Q.L. and Y.Q. Zhao (2003). β-invariant measures for transition matrices of GI/M/1 type. Stochastic Models 19: 201–233
Li Q.L. and Y.Q. Zhao (2004). The RG-factorization in block-structured Markov renewal processes with applications. In Observation, Theory and Modeling of Atmospheric Variability, X. Zhu (ed), World Scientific, 545–568
Li Q.L. and Y.Q. Zhao (2005). Light-tailed asymptotics of stationary probability vectors of Markov chains of GI/G/1 type. Advances in Applied Probability 37: 1075–1093
Li Q.L. and Y.Q. Zhao (2005). Heavy-tailed asymptotics of stationary probability vectors of Markov chains of GI/G/1 type. Advances in Applied Probability 37: 482–509
Neuts M.F. (1981). Matrix-Geometric Solutions in Stochastic Models-An Algorithmic Approach, The Johns Hopkins University Press: Baltimore
Neuts M.F. (1989). Structured Stochastic Matrices of M/G/1 Type and Their Applications, Marcel Dekker: New York
Zhao Y.Q. (2000). Censoring technique in studying block-structured Markov chains. In Advances in Algorithmic Methods for Stochastic Models, G. Latouche and P.G. Taylor (eds), Notable Publications Inc., NJ., 417–433
Zhao Y.Q. and D. Liu (1996). The censored Markov chain and the best augmentaion. Journal of Applied Probability 33: 623–629
Zhao Y.Q., W. Li and A.S. Alfa (1999). Duality results for block-structured transition matrices. Journal of Applied Probability 36: 1045–1057
Zhao Y.Q., W. Li and W.J. Braun (1998). Infinite block-structured transition matrices and their properties. Advances in Applied Probability 30: 365–384
Zhao Y.Q., W. Li and W.J. Braun (2003). Censoring, factorizations and spectral analysis for transition matrices with block-repeating entries. Methodology and Computing in Applied Probability 5: 35–58
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2010 Tsinghua University Press, Beijing and Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Li, QL. (2010). Block-Structured Markov Chains. In: Constructive Computation in Stochastic Models with Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-11492-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-11492-2_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-11491-5
Online ISBN: 978-3-642-11492-2
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)