Abstract
In this paper adaptive control of partially observed discrete time Markov processes with transition probability depending on an unknown random variable is studied. Although the techniques and methods used in the paper can be extended in various directions, the author concentrated himself on an analysis of three models called respectively: mixed observation model, model with observed regeneration, and rich observation model for which partially observed control problem with known transition probability and average cost per unit time functional can be solved.
Supported by the Center of Excellence IMPAN-BC
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Borkar, V. S. and Mundra, S. M. (1998) Bayesian Parameter Estimation and Adaptive Control of Markov Processes with Time-Averaged Costs, Applicationes Math. 25, 339–358
Borkar, V. S. (2000) Average Cost Dynamic Programming Equations for Controlled Markov Chains with Partial Observations, SICON 39, 673–681
Borkar, V. S. (1998) Ergodic control of partially observed Markov processes, Systems & Control Letters 34, 185–189
Di Masi, G. B. and Stettner, Ł. (1994) On Adaptive Control of a Partially Observed Markov Chain, Applicationes Math. 22, 165–180
Di Masi, G. B. and Stettner, Ł. (1995) Bayesian Ergodic Control of Discrete Time Markov Processes, Stochastics and Stochastics Rep. 54, 301–316.
Di Masi, G. B. and Stettner, Ł. (1997) Bayesian Ergodic Adaptive Control of Diffusion Processes, Stochastics and Stochastics Rep. 60, 155–183.
Di Masi, G. B. and Stettner, Ł. (1998) Bayesian adaptive control of discrete-time Markov processes with long-run average cost, Systems and Control Letters 34, 55–62
Duncan, T. E., Pasik-Duncan, B., and Stettner, Ł. (1998) Adaptive Control of a Partially Observed Discrete Time Markov Process, JAMO 37, 269–293
Ikeda, N. and Watanabe, S. (1981) Stochastic Differential Equations and Diffusion Processes, North Holland.
Liverani, C. (1995) Decay of correlation, Ann. Math. 142, 239–301.
Runggaldier, W. J. and Stettner, Ł. (1994) Approximations of Discrete Time Partially Observed Control Problems, Giardini, Pisa.
Stettner, Ł. (1993) Ergodic Control of Partially Observed Markov Processes With Equivalent Transition Probabilities, Applicationes Math. 22, 25–38.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Additional information
Dedicated to Prof. Tyrone Duncan on his 60th birthday
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Stettner, L. (2002). Bayesian Adaptive Control of Discrete Time Partially Observed Markov Processes. In: Pasik-Duncan, B. (eds) Stochastic Theory and Control. Lecture Notes in Control and Information Sciences, vol 280. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48022-6_29
Download citation
DOI: https://doi.org/10.1007/3-540-48022-6_29
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43777-2
Online ISBN: 978-3-540-48022-8
eBook Packages: Springer Book Archive