Part of the International Centre for Mechanical Sciences book series (CISM, volume 280)
Optimal Control of Perturbed Markov Chains: The Multitime Scale Case
Given a controled perturbed Markov chain of transition matrix mu(ε), where ε is the perturbation scale and u the control, we study the solution expansion in ε, wε, of the dynamic programming equation:
mu(ε), cu(ε), λ(ε) are polynomials in ε. The case λ(ε) = εℓ leads to study Markov chains on a time scale of order 1/εℓ. The state space and the control set are finite.
KeywordsMarkov Chain Dynamic Programming Equation Implicit System Aggregate Chain Recurrent Class
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Unable to display preview. Download preview PDF.
- P. BERNHARD. Sur les systèmes dynamiques linéaires implicites singuliers, à paraître SIAM J. on control and optimization et rapport INRIA 69, 1981.Google Scholar
- M. CODERCH, A.S. WILLSKY, S.S. SASTRY, D.A. CASTANON. Hierarchical aggregation of linear systems with multiple time scales, MIT Report LIDS-P-1187, mars 1982.Google Scholar
- M. CODERCH, A.S. WILLSKY, S.S. SASTRY. Hierarchical aggregation of singularly perturbed finite state Markov chains submitted to stochastics.Google Scholar
- P.J. COURTOIS. Decomposability, ACM Monograph Series, Academic Press, 1977.Google Scholar
- F. DELEBECQUE. A reduction process for perturbed Markov chains, à paraître SIAM J. of applied math. to appear.Google Scholar
- F. DELEBECQUE,- J.P. QUADRAT. The optimal cost expansion of finite controls finite states Markov chains with weak and stong interactions. Analysis and optimization of systems, Lecture Notes an control and Inf. Science 28 Springer Verlag, 1980.Google Scholar
- A.A. PERVOZVANSKII, A.V. GAITSGORI. Decomposition aggregation and approximate optimization en Russe, Nauka, Moscou, 1979.Google Scholar
- T. KATO. Perturbation theory for linear operator, Springer Verlag, 1976.Google Scholar
- R. PHILIPS, P.KOKOTOVIC. A singular perturbation approach to modelling and control of Markov chains IEEE A.C. Bellman issue, 1981.Google Scholar
- J. KEMENY, L. SNELL. Finite Markov chains, Van Nostrand, 1960.Google Scholar
© Springer-Verlag Wien 1983