Skip to main content
SpringerLink
Log in

Adaptive control of Markov chains

  • Part II: Research Reports
  • Conference paper
  • First Online:
Stochastic Control Theory and Stochastic Differential Systems

Part of the book series: Lecture Notes in Control and Information Sciences ((LNCIS,volume 16))

Abstract

Consider a controlled Markov chain whose transition probabilities are parameterized by α known to be in a finite set A. To each α is associated a prespecified control law φ(α). The adaptive controller selects at each time t the control action indicated by φ(αt), where αt is the maximum likelihood estimate of α. The asymptotic behavior of αt is studied.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  1. M.H. Degroot, Optimal Statistical Decisions, McGraw-Hill, New York, 1970.

    Google Scholar 

  2. Y. Bar-Shalom and E. Tse, Dual effect, certainty equivalence, and separation in stochastic control, IEEE Trans. Aut. Cont. AC-19(5), 494–500, 1974.

    Google Scholar 

  3. P. Mandl, Estimation and control in Markov chains, Adv. Appl. Prob. 6, 40–60, 1974.

    Google Scholar 

  4. K.J. Astrom and B. Wittenmark, On self-tuning regulators, Automatica 9, 185–199, 1973.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departement of Electrical Engineering and Computer Sciences and the Electronics Research Laboratory, University of California, 94720, Berkeley, CA

    Vivek Borkar & Pravin Varaiya

Authors
  1. Vivek Borkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pravin Varaiya
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

M. Kohlmann W. Vogel

Rights and permissions

Reprints and permissions

Copyright information

© 1979 Springer-Verlag

About this paper

Cite this paper

Borkar, V., Varaiya, P. (1979). Adaptive control of Markov chains. In: Kohlmann, M., Vogel, W. (eds) Stochastic Control Theory and Stochastic Differential Systems. Lecture Notes in Control and Information Sciences, vol 16. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0009389

Download citation

  • DOI: https://doi.org/10.1007/BFb0009389

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-09480-7

  • Online ISBN: 978-3-540-35211-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation