Singularly Perturbed Markov Chains and Applications to Large-Scale Systems under Uncertainty
This chapter is concerned with large-scale hybrid stochastic systems, in which the dynamics involve both continuously evolving components and discrete events. Corresponding to different discrete states, the dynamic behavior of the underlying system could be markedly different. To reduce the complexity of these systems, singularly perturbed Markov chains are used to characterize the system. Asymptotic expansions of probability vectors and the structural properties of these Markov chains are provided. The ideas of decomposition and aggregation are presented using two typical optimal control problems. Such an approach leads to control policies that are simple to obtain and perform nearly as well as the optimal ones with substantially reduced complexity.
Key wordssingularly perturbation Markov chain near optimality optimal control LQG MDP
Unable to display preview. Download preview PDF.
- Gershwin, S.B. (1994). Manufacturing Systems Engineering, Prentice Hall, Englewood Cliffs, 1994.Google Scholar
- Hansen, B.E. (1992). The likelihood ratio test under nonstandard conditions: Testing Markov trend model of GNP, J. Appl. Economics, 7, S61–S82.Google Scholar
- Kendrick, D. (1972). On the Leontief dynamic inverse, Quart. J. Economics, 86, 693–696.Google Scholar
- Kushner, H.J. (1972). Introduction to Stochastic Control Theory, Holt, Rinehart and Winston, New York.Google Scholar
- Liu, R.H., Q. Zhang, and G. Yin. (2001). Nearly optimal control of singularly perturbed Markov decision processes in discrete time, to appear in Appl. Math. Optim.Google Scholar
- Pervozvanskii, A.A. and V.G. Gaitsgori. (1988). Theory of Suboptimal Decisions: Decomposition and Aggregation, Kluwer, Dordrecht.Google Scholar
- Tse, D.N.C., R.G. Gallager, and J.N. Tsitsiklis. (1995). Statistical multiplexing of multiple time-scale Markov streams, IEEE J. Selected Areas Comm.13, 1028–1038.Google Scholar
- Yang, H., G. Yin, K. Yin, and Q. Zhang. (2001). Control of singularly perturbed Markov chains: A numerical study, to appear in J. Australian Math. Soc. Ser. B: Appl. Math.Google Scholar
- Yin, G. (2001). On limit results for a class of singularly perturbed switching diffusions, to appear in J. TheoreticalProbab.Google Scholar
- Yin, G. and J.F. Zhang. (2001). Hybrid singular systems of differential equations, to appear in Scientia Sinica.Google Scholar
- Yin, G. and Q. Zhang (Eds.) (1997b). Mathematics of Stochastic Manufacturing Systems, Proc. 1996 AMS-SIAM Summer Seminar in Applied Mathematics, Lectures in Applied Mathematics, LAM 33, Amer. Math. Soc., Providence, RI, Providence, RI.Google Scholar
- Yin, G., Q. Zhang, and G. Badowski. (2000d). Decomposition and aggregation of large-dimensional Markov chains in discrete time, preprint.Google Scholar