Observer-Based Online Adaptive Regulation for a Class of Uncertain Nonlinear Systems

  • Ding WangEmail author
  • Chaoxu Mu
Part of the Studies in Systems, Decision and Control book series (SSDC, volume 167)


A novel observer-based online control strategy is proposed for a class of uncertain continuous-time nonlinear systems based on solving the HJB equation. Due to the dynamics complexity, the approximate optimal control for affine uncertain continuous-time nonlinear systems is pursued by policy iteration algorithm. Considering that only output variables can be measured in control practice, an observer is designed to reconstruct all system states by relying on output information and then is used to develop the policy iteration control scheme. The observer-based policy iteration algorithm can approximately solve the HJB equation within the ADP framework, where a critic neural network is constructed to approximate the optimal cost function. Then, the approximate expression of the optimal control policy can be directly derived from solving the HJB equation. Additionally, the stability of the closed-loop system is provided based on the Lyapunov theory. Two simulation examples are presented to verify the effectiveness of the proposed control approach.


  1. 1.
    Abdollahi, F., Talebi, H.A., Patel, R.V.: A stable neural network-based observer with application to flexible-joint manipulators. IEEE Trans. Neural Netw. 17(1), 118–129 (2006)CrossRefGoogle Scholar
  2. 2.
    Abu-Khalaf, M., Lewis, F.L.: Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5), 779–791 (2005)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bellman, R.E.: Dynamic Programming. Princeton University Press, Princeton (1957)zbMATHGoogle Scholar
  4. 4.
    Chen, C., Liu, Z., Zhang, Y., Chen, C.L.P., Xie, S.: Asymptotic fuzzy tracking control for a class of stochastic strict-feedback systems. IEEE Trans. Fuzzy Syst. 25(3), 556–568 (2017)CrossRefGoogle Scholar
  5. 5.
    Chen, C.L.P., Wen, G.X., Liu, Y.J., Liu, Z.: Observer-based adaptive backstepping consensus tracking control for high-order nonlinear semi-strict-feedback multiagent systems. IEEE Trans. Cybern. 46(7), 1591–1601 (2016)CrossRefGoogle Scholar
  6. 6.
    Chen, Z., Li, Z., Chen, C.L.P.: Adaptive neural control of uncertain MIMO nonlinear systems with state and input constraints. IEEE Trans. Neural Netw. Learn. Syst. 28(6), 1318–1330 (2017)CrossRefGoogle Scholar
  7. 7.
    Cui, L., Zhang, H., Chen, B., Zhang, Q.: Asymptotic tracking control scheme for mechanical systems with external disturbances and friction. Neurocomputing 73(7–9), 1293–1302 (2010)CrossRefGoogle Scholar
  8. 8.
    Dierks, T., Jagannathan, S.: Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update. IEEE Trans. Neural Netw. Learn. Syst. 23(7), 1118–1129 (2012)CrossRefGoogle Scholar
  9. 9.
    Hanselmann, T., Noakes, L., Zaknich, A.: Continuous-time adaptive critics. IEEE Trans. Neural Netw. 18(3), 631–647 (2007)CrossRefGoogle Scholar
  10. 10.
    He, H., Ni, Z., Fu, J.: A three-network architecture for on-line learning and optimization based on adaptive dynamic programming. Neurocomputing 78(1), 3–13 (2012)CrossRefGoogle Scholar
  11. 11.
    He, W., Dong, Y., Sun, C.: Adaptive neural impedance control of a robotic manipulator with input saturation. IEEE Trans. Syst. Man Cybern.: Syst. 46(3), 334–344 (2016)CrossRefGoogle Scholar
  12. 12.
    Hornik, K., Stinchcombe, M., White, H.: Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks. Neural Netw. 3(5), 551–560 (1990)CrossRefGoogle Scholar
  13. 13.
    Jiang, Y., Jiang, Z.P.: Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics. Automatica 48(10), 2699–2704 (2012)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Jiang, Y., Jiang, Z.P.: Robust adaptive dynamic programming for large-scale systems with an application to multimachine power systems. IEEE Trans. Circuits Syst.-II: Express Briefs 59(10), 693–697 (2012)CrossRefGoogle Scholar
  15. 15.
    Jiang, Y., Jiang, Z.P.: Robust adaptive dynamic programming and feedback stabilization of nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 25(5), 882–893 (2014)CrossRefGoogle Scholar
  16. 16.
    Kim, Y.H., Lewis, F.L., Abdallah, C.T.: A dynamic recurrent neural-network-based adaptive observer for a class of nonlinear systems. Automatica 33(8), 1539–1543 (1997)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Lewis, F.L., Syrmos, V.L.: Optimal Control. Wiley, New York (1995)Google Scholar
  18. 18.
    Lin, W.S., Sheu, J.W.: Optimization of train regulation and energy usage of metro lines using an adaptive-optimal-control algorithm. IEEE Trans. Autom. Sci. Eng. 8(4), 855–864 (2011)CrossRefGoogle Scholar
  19. 19.
    Liu, D., Huang, Y., Wang, D., Wei, Q.: Neural-network-observer-based optimal control for unknown nonlinear systems using adaptive dynamic programming. Int. J. Control 86(9), 1554–1566 (2013)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Liu, Y., Li, J., Tong, S., Chen, C.L.P.: Neural network control-based adaptive learning design for nonlinear systems with full-state constraints. IEEE Trans. Neural Netw. Learn. Syst. 27(7), 1562–1571 (2016)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Lv, Y., Na, J., Yang, Q., Wu, X., Guo, Y.: Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics. Int. J. Control 89(1), 99–112 (2016)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Modares, H., Lewis, F.L., Naghibi-Sistani, M.B.: Adaptive optimal control of unknown constrained-input systems using policy iteration and neural networks. IEEE Trans. Neural Netw. Learn. Syst. 24(10), 1513–1525 (2013)CrossRefGoogle Scholar
  23. 23.
    Mu, C., Ni, Z., Sun, C., He, H.: Air-breathing hypersonic vehicle tracking control based on adaptive dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 584–598 (2017)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Mu, C., Ni, Z., Sun, C., He, H.: Data-driven tracking control with adaptive dynamic programming for a class of continuous-time nonlinear systems. IEEE Trans. Cybern. 47(6), 1460–1470 (2017)CrossRefGoogle Scholar
  25. 25.
    Mu, C.: Observer-based online adaptive control for a class of uncertain continuous-time nonlinear systems. In preparation (2018)Google Scholar
  26. 26.
    Ni, Z., He, H., Wen, J., Xu, X.: Goal representation heuristic dynamic programming on maze navigation. IEEE Trans. Neural Netw. Learn. Syst. 24(12), 2038–2050 (2013)CrossRefGoogle Scholar
  27. 27.
    Prokhorov, D.V., Santiago, R.A., Wunsch, D.C.: Adaptive critic designs: a case study for neurocontrol. Neural Netw. 8(9), 1367–1372 (1995)CrossRefGoogle Scholar
  28. 28.
    Rudin, W.: Principles of Mathematical Analysis. McGraw-Hill, New York (1976)zbMATHGoogle Scholar
  29. 29.
    Si, J., Wang, Y.T.: On-line learning control by association and reinforcement. IEEE Trans. Neural Netw. 12(2), 264–276 (2001)CrossRefGoogle Scholar
  30. 30.
    Tong, S., Zhang, L., Li, Y.: Observed-based adaptive fuzzy decentralized tracking control for switched uncertain nonlinear large-scale systems with dead zones. IEEE Trans. Syst. Man Cybern.: Syst. 46(1), 37–47 (2016)CrossRefGoogle Scholar
  31. 31.
    Vamvoudakis, K.G., Lewis, F.L.: Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5), 878–888 (2010)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Vrabie, D., Lewis, F.L.: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Netw. 22(3), 237–246 (2009)CrossRefGoogle Scholar
  33. 33.
    Wang, D., Liu, D., Li, H.: Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems. IEEE Trans. Autom. Sci. Eng. 11(2), 627–632 (2014)CrossRefGoogle Scholar
  34. 34.
    Wang, D., Liu, D., Li, H., Ma, H.: Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming. Inf. Sci. 282, 167–179 (2014)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Wang, D., Liu, D., Zhang, Q., Zhao, D.: Data-based adaptive critic designs for nonlinear robust optimal control with uncertain dynamics. IEEE Trans. Syst. Man Cybern.: Syst. 46(11), 1544–1555 (2016)CrossRefGoogle Scholar
  36. 36.
    Wang, D., Mu, C., He, H., Liu, D.: Event-driven adaptive robust control of nonlinear systems with uncertainties through NDP strategy. IEEE Trans. Syst. Man Cybern.: Syst. 47(7), 1358–1370 (2017)CrossRefGoogle Scholar
  37. 37.
    Wang, D., Mu, C., Yang, X., Liu, D.: Event-based constrained robust control of affine systems incorporating adaptive critic mechanism. IEEE Trans. Syst. Man Cybern.: Syst. 47(7), 1602–1612 (2017)CrossRefGoogle Scholar
  38. 38.
    Werbos, P.J.: Approximate dynamic programming for real-time control and neural modeling. In: Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, pp. 493–526 (1992)Google Scholar
  39. 39.
    Yang, X., Liu, D., Ma, H., Xu, Y.: Online approximate solution of HJI equation for unknown constrained-input nonlinear continuous-time systems. Inf. Sci. 328, 435–454 (2016)CrossRefGoogle Scholar
  40. 40.
    Zhang, H., Cui, L., Luo, Y.: Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network ADP. IEEE Trans. Cybern. 43(1), 206–216 (2013)CrossRefGoogle Scholar
  41. 41.
    Zhang, H., Liu, D., Luo, Y., Wang, D.: Adaptive Dynamic Programming for Control: Algorithms and Stability. Springer, London (2013)CrossRefGoogle Scholar
  42. 42.
    Zhao, D., Zhang, Q., Wang, D., Zhu, Y.: Experience replay for optimal control of nonzero-sum game systems with unknown dynamics. IEEE Trans. Cybern. 46(3), 854–865 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.The State Key Laboratory of Management and Control for Complex SystemsInstitute of Automation, Chinese Academy of SciencesBeijingChina
  2. 2.School of Electrical and Information EngineeringTianjin UniversityTianjinChina

Personalised recommendations