Adaptive Tracking Control of Nonlinear Systems Subject to Matched Uncertainties

  • Ding WangEmail author
  • Chaoxu Mu
Part of the Studies in Systems, Decision and Control book series (SSDC, volume 167)


In this chapter, an adaptive tracking control scheme is designed for a class of continuous-time uncertain nonlinear systems based on the approximate solution of the HJB equation. Considering matched uncertainties, the tracking control of the continuous-time uncertain nonlinear system can be transformed to the optimal tracking control of the associated nominal system. By building the nominal error system and modifying its cost function, the solution of the relevant HJB equation can be contributed to the adaptive tracking control of the continuous-time uncertain nonlinear system. In view of the complexity on solving the HJB equation, its approximate solution is pursued by the policy iteration algorithm under the ADP framework, where a critic neural network is constructed to approximate the optimal cost function. Therein, an action network is used to directly calculate the approximate optimal control law, which constitutes the tracking control law for the original uncertain system together with the steady control law. The weight convergence of the critic network and the stability of the closed-loop system are provided as the theoretical guarantee based on the Lyapunov theory. Two simulation examples are studied to verify the theoretical results and the effectiveness of the proposed tracking control scheme.


  1. 1.
    Bellman, R.E.: Dynamic Programming. Princeton University Press, Princeton (1957)zbMATHGoogle Scholar
  2. 2.
    Cheng, L., Hou, Z.G., Tan, M., Zhang, W.J.: Tracking control of a closed-chain five-bar robot with two degrees of freedom by integration of an approximation-based approach and mechanical design. IEEE Trans. Syst. Man Cybern. Part B: Cybern. 42(5), 1470–1479 (2012)Google Scholar
  3. 3.
    Cimen, T., Banks, S.P.: Nonlinear optimal tracking control with application to super-tankers for autopilot design. Automatica 40(11), 1845–1863 (2004)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Dierks, T., Jagannathan, S.: Optimal tracking control of affine nonlinear discrete-time systems with unknown internal dynamics. In: Proceedings of the 48h IEEE Conference on Decision and Control held jointly with 28th Chinese Control Conference, pp. 6750–6755 (2009)Google Scholar
  5. 5.
    Faulwasser, T., Findeisen, R.: Nonlinear model predictive control for constrained output path following. IEEE Trans. Autom. Control 61(4), 1026–1039 (2016)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Hanselmann, T., Noakes, L., Zaknich, A.: Continuous-time adaptive critics. IEEE Trans. Neural Netw. 18(3), 631–647 (2007)CrossRefGoogle Scholar
  7. 7.
    He, H., Ni, Z., Fu, J.: A three-network architecture for on-line learning and optimization based on adaptive dynamic programming. Neurocomputing 78(1), 3–13 (2012)CrossRefGoogle Scholar
  8. 8.
    He, P., Jagannathan, S.: Reinforcement learning-based output feedback control of nonlinear systems with input constraints. IEEE Trans. Syst. Man Cybern. Part B: Cybern. 35(1), 150–154 (2005)Google Scholar
  9. 9.
    Howard, R.A.: Dynamic Programming and Markov Processes. Technology Press and Wiley, NewYork (1960)zbMATHGoogle Scholar
  10. 10.
    Jorge, D.: Exact tracking using backstepping control design and high-order sliding modes. IEEE Trans. Autom. Control 58(8), 2077–2081 (2013)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Kamalapurkar, R., Andrews, L., Walters, P., Dixon, W.E.: Model-based reinforcement learning for infinite-horizon approximate optimal tracking. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 753–758 (2017)CrossRefGoogle Scholar
  12. 12.
    Lewis, F.L., Ge, S.Z.: Neural networks in feedback control systems. In: Mechanical Engineer’s Handbook. Wiley, New York (2005)Google Scholar
  13. 13.
    Lewis, F.L., Syrmos, V.L.: Optimal Control. Wiley, New York (1995)Google Scholar
  14. 14.
    Lewis, F.L., Vrabie, D.: Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst. Mag. 9(3), 32–50 (2009)CrossRefGoogle Scholar
  15. 15.
    Liu, D., Xiong, X., Zhang, Y.: Action-dependent adaptive critic designs. In: Proceedings of 2001 International Joint Conference on Neural Networks, pp. 990–995 (2001)Google Scholar
  16. 16.
    Liu, Y., Gao, Y., Tong, S., Li, Y.: Fuzzy approximation-based adaptive backstepping optimal control for a class of nonlinear discrete-time systems with dead-zone. IEEE Trans. Fuzzy Syst. 24(1), 16–28 (2016)CrossRefGoogle Scholar
  17. 17.
    Liu, Y., Tang, L., Tong, S., Chen, C.P.: Adaptive NN controller design for a class of nonlinear MIMO discrete-time systems. IEEE Trans. Neural Netw. Learn. Syst. 26(5), 1007–1018 (2015)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Liu, Y., Tong, S.: Adaptive NN tracking control of uncertain nonlinear discrete-time systems with nonaffine dead-zone input. IEEE Trans. Cybern. 45(3), 497–505 (2015)CrossRefGoogle Scholar
  19. 19.
    Mobayen, S.: An adaptive chattering-free PID sliding mode control based on dynamic sliding manifolds for a class of uncertain nonlinear systems. Nonlinear Dyn. 82(1–2), 53–60 (2015)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Modares, H., Lewis, F.L.: Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning. Automatica 50(7), 1780–1792 (2014)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Mu, C., Ni, Z., Sun, C., He, H.: Data-driven tracking control with adaptive dynamic programming for a class of continuous-time nonlinear systems. IEEE Trans. Cybern. 47(6), 1460–1470 (2017)CrossRefGoogle Scholar
  22. 22.
    Mu, C., Sun, C., Wang, D., Song, A.: Adaptive tracking control for a class of continuous-time uncertain nonlinear systems using the approximate solution of HJB equation. Neurocomputing 260, 432–442 (2017)CrossRefGoogle Scholar
  23. 23.
    Mu, C., Wang, D.: Neural-network-based adaptive guaranteed cost control of nonlinear dynamical systems with matched uncertainties. Neurocomputing 245, 46–54 (2017)CrossRefGoogle Scholar
  24. 24.
    Narayanan V, Jagannathan S (2016) Approximate optimal distributed control of uncertain nonlinear interconnected systems with event-sampled feedback. In: Proceedings of the 55th IEEE Conference on Decision and Control, pp. 5827–5832Google Scholar
  25. 25.
    Park, Y.-M., Choi, M.-S., Lee, K.Y.: An optimal tracking neuro-controller for nonlinear dynamic systems. IEEE Trans. Neural Netw. 7(5), 1099–1110 (1996)CrossRefGoogle Scholar
  26. 26.
    Sahoo, A., Xu, H., Jagannathan, S.: Approximate optimal control of affine nonlinear continuous-time systems using event-sampled neurodynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 639–652 (2017)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Si, J., Wang, Y.-T.: Online learning control by association and reinforcement. IEEE Trans. Neural Netw. 12(2), 264–276 (2001)CrossRefGoogle Scholar
  28. 28.
    Vamvoudakis, K.G., Lewis, F.L.: Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5), 878–888 (2010)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Wang, D., Liu, D., Wei, Q.: Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach. Neurocomputing 78(1), 14–22 (2012)CrossRefGoogle Scholar
  30. 30.
    Wang, D., Liu, D., Zhang, Q., Zhao, D.: Data-based adaptive critic designs for nonlinear robust optimal control with uncertain dynamics. IEEE Trans. Syst. Man Cybern. Syst. 46(11), 1544–1555 (2016)CrossRefGoogle Scholar
  31. 31.
    Werbos, P.J.: Consistency of HDP applied to a simple reinforcement learning problem. Neural Netw. 3(2), 179–189 (1990)CrossRefGoogle Scholar
  32. 32.
    Werbos, P.J.: Approximate dynamic programming for real-time control and neural modeling. In: Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, pp. 493–526 (1992)Google Scholar
  33. 33.
    Yang, C., Li, Y., Ge, S.S., Lee, T.H.: Adaptive control of a class of discrete-time MIMO nonlinear systems with uncertain couplings. Int. J. Control 83(10), 2120–2133 (2010)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Yang, L., Si, J., Tsakalis, K.S., Rodriguez, A.: Direct heuristic dynamic programming for nonlinear tracking control with filtered tracking error. IEEE Trans. Syst. Man Cybern. Part B: Cybern. 39(6), 1617–1622 (2009)Google Scholar
  35. 35.
    Zhang, H., Cui, L., Zhang, X., Luo, Y.: Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method. IEEE Trans. Neural Netw. 22(12), 2226–2236 (2011)CrossRefGoogle Scholar
  36. 36.
    Zhang, H., Luo, Y., Liu, D.: Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans. Neural Netw. 20(9), 1490–1503 (2009)CrossRefGoogle Scholar
  37. 37.
    Zhang, H., Wei, Q., Luo, Y.: A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm. IEEE Trans. Syst. Man Cybern. Part B: Cybern. 38(4), 937–942 (2008)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.The State Key Laboratory of Management and Control for Complex SystemsInstitute of Automation, Chinese Academy of SciencesBeijingChina
  2. 2.School of Electrical and Information EngineeringTianjin UniversityTianjinChina

Personalised recommendations