Encyclopedia of Systems and Control

Living Edition
| Editors: John Baillieul, Tariq Samad

Stochastic Adaptive Control

Living reference work entry

Latest version View entry history

DOI: https://doi.org/10.1007/978-1-4471-5102-9_231-2


Stochastic adaptive control focuses on the control of partially known stochastic systems. These systems occurs in both continuous and discrete time and are described by Markov chains, stochastic difference equations, and stochastic differential equations. Two major goals for the solutions are self-tuning and self-optimizing. These two goals are typically determined asymptotically so that self-optimality denotes the convergence of the average costs to the optimal long-run average cost for the true system.


Stochastic systems; Stochastic control; System identification; Parameter estimation; adaptive control 
This is a preview of subscription content, log in to check access.


  1. Astrom KJ, Wittenmark B (1973) On self-tuning regulators. Automatica 9:185–199CrossRefGoogle Scholar
  2. Astrom KJ, Wittenmark B (1989) Adaptive control. Addison-Wesley, ReadingzbMATHGoogle Scholar
  3. Bercu B (1995) Weighted estimation and tracking for ARMAX models. SIAM J Control Optim 33:89–106MathSciNetCrossRefGoogle Scholar
  4. Borkar V, Varaiya P (1982) Identification and adaptive control of Markov chains. SIAM J Control Optim 20:470–489MathSciNetCrossRefGoogle Scholar
  5. Chen HF, Guo L (1991) Identification and stochastic adaptive control. Birkhauser, BostonCrossRefGoogle Scholar
  6. Davis MHA, Vinter RD (1985) Stochastic modelling and control. Chapman and Hall, LondonCrossRefGoogle Scholar
  7. Duncan TE, Pasik-Duncan B (1990) Adaptive control of continuous time linear systems. Math Control Signals Syst 3:43–60MathSciNetCrossRefGoogle Scholar
  8. Duncan TE, Maslowski B, Pasik-Duncan B (1994a) Adaptive boundary and point control of linear stochastic distributed parameter systems. SIAM J Control Optim 32:648–672MathSciNetCrossRefGoogle Scholar
  9. Duncan TE, Pasik-Duncan B, Stettner L (1994b) Almost self-optimizing strategies for the adaptive control of diffusion processes. J Optim Theory Appl 81:470–507MathSciNetzbMATHGoogle Scholar
  10. Duncan TE, Guo L, Pasik-Duncan B (1999) Adaptive continuous-time linear quadratic Gaussian control. IEEE Trans Autom Control 44:1653–1662MathSciNetCrossRefGoogle Scholar
  11. Duncan TE, Tembine T (2018) Linear-quadratic mean field type games: a direct method. Games J 9:7–16MathSciNetCrossRefGoogle Scholar
  12. Duncan TE, Pasik-Duncan B (2013) Linear-quadratic fractional Gaussian control. SIAM J Control Optim 51:4604–4619MathSciNetCrossRefGoogle Scholar
  13. Fleming WH, Rishel RW (1975) Deterministic and stochastic optimal control. Springer, New YorkCrossRefGoogle Scholar
  14. Goodwin G, Ramadge P, Caines PE (1981) Discrete time stochastic adaptive control. SIAM J Control Optim 19:820–853MathSciNetCrossRefGoogle Scholar
  15. Guo L (1995) Convergence and logarithm laws of self-tuning regulators. Automatica 31:435–450MathSciNetCrossRefGoogle Scholar
  16. Guo L (1996) Self-convergence of weighted least squares with applications. IEEE Trans Autom Control 41: 79–89CrossRefGoogle Scholar
  17. Guo L (1997) On critical stability of discrete time adaptive nonlinear control. IEEE Trans Autom Control 42:1488–1499MathSciNetCrossRefGoogle Scholar
  18. Guo L, Chen HF (1991) The Astrom-Wittenmark self-tuning regulator revisited and ELS based adaptive trackers. IEEE Trans Autom Control 36:802–812CrossRefGoogle Scholar
  19. Huang M, Caines, P E, Malhame, R (2006) Large population stochastic dyname games, closed loop McKean-Vlasov systems and the Nash certainty equivalence,. IEEE Trans Autom Control 52:1560–1571CrossRefGoogle Scholar
  20. Kumar PR (1985) A survey of some results in stochastic adaptive control. SIAM J Control Optim 23:329–380MathSciNetCrossRefGoogle Scholar
  21. Kumar PR (1990) Convergence of adaptive control schemes with least squares estimates. IEEE Trans Autom Control 35:416–424MathSciNetCrossRefGoogle Scholar
  22. Kumar PR, Varaiya P (1986) Stochastic systems, estimation, identification and adaptive control. Prentice-Hall, Englewood CliffszbMATHGoogle Scholar
  23. Lai TL, Wei CZ (1982) Least square estimation is stochastic regression models with applications to identification and control of dynamic systems. Ann Stat 10:154–166CrossRefGoogle Scholar
  24. Lasry JM, Lions PL (2007) Mean field games. Jpn J Math 2:227–269MathSciNetCrossRefGoogle Scholar
  25. Ljung L, Soderstrom T (1983) Theory and practice of recursive identification. MIT Press, CambridgezbMATHGoogle Scholar
  26. Mandl P (1973) On the adaptive control of finite state Markov processes. Z Wahr Verw Geb 27:263–276MathSciNetCrossRefGoogle Scholar
  27. Nourian M, Caines PE, Malhame RP (2012) Mean field LQG control in leader-follower stochastic multi-agent systems: likelihood ratio based adaptation. IEEE Trans Autom Control 57:2801–2816MathSciNetCrossRefGoogle Scholar

Authors and Affiliations

  1. 1.Department of MathematicsUniversity of KansasLawrenceUSA

Section editors and affiliations

  • Lei Guo
    • 1
  1. 1.Academy of Mathematics and Systems Science, Chinese Academy of Sciences (CAS)BeijingChina