Skip to main content
SpringerLink
Log in

Certainty equivalence with uncertainty adjustments in stochastic adaptive control

  • Conference paper
  • First Online:
Stochastic Theory and Adaptive Control

Part of the book series: Lecture Notes in Control and Information Sciences ((LNCIS,volume 184))

  • 188 Accesses

Abstract

A useful technique for finding relatively simple and yet asymptotically optimal solutions to stochastic adaptive control problems is to incorporate into certainty-equivalence rules suitable adjustments for parameter uncertainty. We review some results in sequential testing and estimation theories in statistics and discuss their applications to the assessment of parameter uncertainty and to efficient adjustments of certainty-equivalence rules in stochastic adaptive control.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. T. L. Lai, “Information bounds, certainty equivalence and learning in asymptotically efficient adaptive control of time-invariant stochastic systems”, in Topics in Stochastic Systems, Modelling, Estimation and Adaptive Control (L. Gerencser and P. E. Caines, Eds.), Springer-Verlag, 1991, pp. 335–368.

    Google Scholar 

  2. T. L. Lai, “Adaptive treatment allocation and the multi-armed bandit problem”, Ann. Statist., vol. 15 (1987), pp. 1091–1114.

    MATH  MathSciNet  Google Scholar 

  3. T. L. Lai, “Asymptotic solutions of bandit problems”, in Stochastic Differential Systems, Stochastic Control Theory and Applications (W. Fleming and P. L. Lions, Eds.), Springer-Verlag, 1988, pp. 275–292.

    Google Scholar 

  4. J. C. Gittins, “Bandit processes and dynamic allocation indices”, J. Roy. Statist. Soc. Ser. B, vol. 41 (1979), pp. 148–177.

    MATH  MathSciNet  Google Scholar 

  5. T. L. Lai and H. Robbins, “Asymptotically efficient adaptive allocation rules,” Adv. Appl. Math., vol. 6 (1985), pp. 4–22.

    Article  MATH  MathSciNet  Google Scholar 

  6. T. L. Lai and H. Robbins, “Asymptotically optimal allocation of treatments in sequential experiments”, in Design of Experiments, Ranking and Selection (T. J. Santner and A. C. Tamhane, Eds.), Marcel-Dekker, 1984, pp. 127–142.

    Google Scholar 

  7. Y. S. Chow, H. Robbins and D. Siegmund, Great Expectations: The Theory of Optimal Stopping. Houghton Mifflin, 1972.

    Google Scholar 

  8. T. L. Lai, “Nearly optimal sequential tests of composite hypotheses,” Ann. Statist., vol. 16 (1988), pp. 856–886.

    MATH  MathSciNet  Google Scholar 

  9. T. L. Lai, “Boundary crossing problems for sample means”, Ann. Probability, vol. 16 91988), pp. 375–396.

    MATH  Google Scholar 

  10. C. D. Fuh and T. L. Lai, “Asymptotically efficient adaptive control of Markov chains”, Tech. Report, Department of Statistics, Stanford Univ., 1992.

    Google Scholar 

  11. V. Borkar and P. Varaiya, “Adaptive control of Markov chains, I: Finite parameter set”, IEEE Trans. Automat. Contr., vol. AC-24 (1979), pp. 953–958.

    Article  MathSciNet  Google Scholar 

  12. V. Borkar and P. Varaiya, “Identification and adaptive control of Markov chains”, SIAM J. Contr. Optimiz., vol. 20 (1982), pp. 470–489.

    Article  MATH  MathSciNet  Google Scholar 

  13. R. Agrawal, D. Teneketzis and V. Anantharam, “Asymptotically efficient adaptive allocation schemes for controlled Markov chains: Finite parameter space”, IEEE Trans. Automat. Contr., vol. 34 (1989), pp. 1249–1259.

    Article  MATH  MathSciNet  Google Scholar 

  14. K. J. Åström, “Theory and applications of adaptive control — A survey”, Automatica, vol. 19 (1983), pp. 471–486.

    Article  MATH  Google Scholar 

  15. T. L. Lai and C. Z. Wei, “Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems”, Ann. Statist., vol. 10 (1982), pp. 154–166.

    MathSciNet  Google Scholar 

  16. P. R. Kumar, “A survey of some results in stochastic adaptive control”, SIAM J. Contr. Optimiz., vol. 23 (1985), pp. 329–380.

    Article  MATH  Google Scholar 

  17. T. L. Lai, “Asymptotically efficient adaptive control in stochastic regression models”, Adv. Appl. Math., vol. 7 (1986), pp. 23–45.

    Article  MATH  Google Scholar 

  18. T. L. Lai and C. Z. Wei, “Asymptotically efficient self-tuning regulators”, SIAM J. Contr. Optimiz., vol. 25 (1987), pp. 466–481.

    Article  MATH  MathSciNet  Google Scholar 

  19. T. L. Lai and Z. Ying, “Parallel recursive algorithms in asymptotically efficient adaptive control of linear stochastic systems”, SIAM J. Contr. Optimiz., vol. 29 (1991), pp. 1091–1127.

    Article  MATH  MathSciNet  Google Scholar 

  20. T. L. Lai and Z. Ying, “Recursive identification and adaptive prediction in linear stochastic systems”, SIAM J. Contr. Optimiz., vol. 29 (1991), pp. 1061–1090.

    Article  MATH  MathSciNet  Google Scholar 

  21. G. Zhu, “Least squares estimation and adaptive prediction in non-linear stochastic regression models with applications to time series and stochastic systems”, Ph.D. Dissertation, Department of Statistics, Stanford Univ., 1992.

    Google Scholar 

  22. T. L. Lai and G. Zhu, “Adaptive prediction in non-linear autoregressive models and control systems”, Statistica Sinica, vol. 1 (1991), pp. 309–334.

    MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, Stanford University, 94305, Stanford, CA

    Tze Leung Lai

Authors
  1. Tze Leung Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

T. E. Duncan B. Pasik-Duncan

Rights and permissions

Reprints and permissions

Copyright information

© 1992 Springer-Verlag

About this paper

Cite this paper

Lai, T.L. (1992). Certainty equivalence with uncertainty adjustments in stochastic adaptive control. In: Duncan, T.E., Pasik-Duncan, B. (eds) Stochastic Theory and Adaptive Control. Lecture Notes in Control and Information Sciences, vol 184. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0113247

Download citation

  • DOI: https://doi.org/10.1007/BFb0113247

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-55962-7

  • Online ISBN: 978-3-540-47327-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation