Skip to main content
SpringerLink
Log in

Adaptive dynamic programming-based stabilization of nonlinear systems with unknown actuator saturation

  • Original Paper
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

This paper addresses the stabilizing control problem for nonlinear systems subject to unknown actuator saturation by using adaptive dynamic programming algorithm. The control strategy is composed of an online nominal optimal control and a neural network (NN)-based feed-forward saturation compensator. For nominal systems without actuator saturation, a critic NN is established to deal with the Hamilton–Jacobi–Bellman equation. Thus, the online approximate nominal optimal control policy can be obtained without action NN. Then, the unknown actuator saturation, which is considered as saturation nonlinearity by simple transformation, is compensated by employing a NN-based feed-forward control loop. The stability of the closed-loop nonlinear system is analyzed to be ultimately uniformly bounded via Lyapunov’s direct method. Finally, the effectiveness of the presented control method is demonstrated by two simulation examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Bellman, R.E.: Dynamic Programming. Princeton University Press, New Jersey (1957)

    MATH  Google Scholar 

  2. Werbos, P.J.: Approximate dynamic programming for real time control and neural modeling. In: White, D.A., Sofge, D.A. (eds.) Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches. Van Nostrand Reinhold, New York (1992)

    Google Scholar 

  3. Liu, D., Wei, Q., Wang, D., Yang, X., Li, H.: Adaptive dynamic programming with applications in optimal control. Springer, Cham (2017)

    Book  MATH  Google Scholar 

  4. Ge, S., Hang, C., Zhang, T.: Adaptive neural network control of nonlinear systems by state and output feedback. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 29(6), 818–828 (1999)

    Article  Google Scholar 

  5. Zhao, B., Li, Y.: Local joint information based active fault tolerant control for reconfigurable manipulator. Nonlinear Dyn. 77(3), 859–876 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  6. Zhao, B., Li, Y., Liu, D.: Self-tuned local feedback gain based decentralized fault tolerant control for a class of large-scale nonlinear systems. Neurocomputing 235, 147–156 (2017)

    Article  Google Scholar 

  7. Wang, F.Y., Zhang, H., Liu, D.: Adaptive dynamic programming: an introduction. IEEE Comput. Intell. Mag. 4(2), 39–47 (2009)

    Article  Google Scholar 

  8. Al-Tamimi, A., Lewis, F.L., Abu-Khalaf, M.: Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 38(4), 943–949 (2008)

    Article  Google Scholar 

  9. Prokhorov, D.V., Wunsch, D.C.: Adaptive critic designs. IEEE Trans. Neural Netw. 8(5), 997–1007 (1997)

    Article  Google Scholar 

  10. Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-dynamic programming: an overview. In: Proceedings of the 34th IEEE Conference on Decision and Control, pp. 560–564 (1995)

  11. Si, J., Wang, Y.T.: On-line learning control by association and reinforcement. IEEE Trans. Neural Netw. 12, 264–276 (2001)

    Article  Google Scholar 

  12. Li, D.J., Tang, L., Liu, Y.J.: Adaptive intelligence learning for nonlinear chaotic systems. Nonlinear Dyn. 73(4), 2103–2109 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  13. Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996)

    Article  Google Scholar 

  14. Liu, D., Wang, D., Li, H.: Decentralized stabilization for a class of continuous-time nonlinear interconnected systems using online learning optimal control approach. IEEE Trans. Neural Netw. Learn. Syst. 25(2), 418–428 (2014)

    Article  Google Scholar 

  15. Wang, D., He, H., Zhao, B., Liu, D.: Adaptive near-optimal controllers for nonlinear decentralised feedback stabilisation problems. IET Control Theory Appl. 11(6), 799–806 (2017)

    Article  MathSciNet  Google Scholar 

  16. Mehraeen, S., Jagannathan, S.: Decentralized optimal control of a class of interconnected nonlinear discrete-time systems by using online Hamilton–Jacobi–Bellman formulation. IEEE Trans. Neural Netw. 22(11), 1757–1769 (2011)

    Article  Google Scholar 

  17. Lin, Q., Wei, Q., Zhao, B.: Optimal control for discrete-time systems with actuator saturation. Optim. Contr. Appl. Met. 38, 1071–1080 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  18. Gao, W., Jiang, Y., Jiang, Z.P., Chai, T.: Output-feedback adaptive optimal control of interconnected systems based on robust adaptive dynamic programming. Automatica 72, 37–45 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  19. Fan, Q., Yang, G.: Adaptive actor-critic design-based integral sliding-mode control for partially unknown nonlinear systems with input disturbances. IEEE Trans. Neural Netw. Learn. Syst. 27(1), 165–177 (2016)

    Article  MathSciNet  Google Scholar 

  20. Zhao, B., Liu, D., Yang, X., Li, Y.: Observer-critic structure-based adaptive dynamic programming for decentralised tracking control of unknown large-scale nonlinear systems. Int. J. Syst. Sci. 48(9), 1978–1989 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  21. Zhao, B., Li, Y.: Model-free adaptive dynamic programming based near-optimal decentralized tracking Control of reconfigurable manipulators. Int. J. Control Autom. Syst. 16(2), 478–490 (2018)

    Article  Google Scholar 

  22. Zhang, H., Song, R., Wei, Q., Zhang, T.: Optimal tracking control for a class of nonlinear discrete-time systems with time delays based on heuristic dynamic programming. IEEE Trans. Neural Netw. 22(12), 1851–1862 (2011)

    Article  Google Scholar 

  23. Xie, C., Yang, G.: Approximate guaranteed cost fault-tolerant control of unknown nonlinear systems with time-varying actuator faults. Nonlinear Dyn. 83(1–2), 269–282 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  24. Zhao, B., Liu, D., Li, Y.: Online fault compensation control based on policy iteration algorithm for a class of affine non-linear systems with actuator failures. IET Control Theory Appl. 10(15), 1816–1823 (2016)

    Article  MathSciNet  Google Scholar 

  25. Fu, Y., Fu, J., Chai, T.: Robust adaptive dynamic programming of two-player zero-sum games for continuous-time linear systems. IEEE Trans. Neural Netw. Learn. Syst. 26(12), 3314–3319 (2015)

    Article  MathSciNet  Google Scholar 

  26. Wang, D., Mu, C., He, H., Liu, D.: Event-driven adaptive robust control of nonlinear systems with uncertainties through NDP strategy. IEEE Trans. Syst. Man Cybern. Syst. 47(7), 1358–1370 (2017)

    Article  Google Scholar 

  27. Sun, L., Huo, W., Jiao, Z.: Adaptive backstepping control of spacecraft rendezvous and proximity operations with input saturation and full-state constraint. IEEE Trans. Industr. Electron. 64(1), 480–492 (2017)

    Article  Google Scholar 

  28. Wang, F., Hua, C., Zong, Q.: Attitude control of reusable launch vehicle in reentry phase with input constraint via robust adaptive backstepping control. Int. J. Aadpt. Control. 29(10), 1308–1327 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  29. Mendoza, M., Zavala-Rło, A., Santibnez, V., Reyes, F.: Output-feedback proportional–integral-derivative-type control with simple tuning for the global regulation of robot manipulators with input constraints. IET Control Theory Appl. 9(14), 2097–2106 (2015)

    Article  MathSciNet  Google Scholar 

  30. Chen, X., Jia, Y., Matsuno, F.: Tracking control for differential-drive mobile robots with diamond-shaped input constraints. IEEE Trans. Control Syst. Technol. 22(5), 1999–2006 (2014)

    Article  Google Scholar 

  31. Kiefer, T., Graichen, K., Kugi, A.: Trajectory tracking of a 3-DOF laboratory helicopter under input and state constraints. IEEE Trans. Control Syst. Technol. 18(4), 944–952 (2010)

    Article  Google Scholar 

  32. Zhai, D., Xia, Y.: Adaptive control for teleoperation system with varying time delays and input saturation constraints. IEEE Trans. Industr. Electron. 63(11), 6921–6929 (2016)

    Article  Google Scholar 

  33. Zhao, F., Ge, S., Tu, F., Qin, Y., Dong, M.: Adaptive neural network control for active suspension system with actuator saturation. IET Control Theory Appl. 10(14), 1696–1705 (2016)

    Article  MathSciNet  Google Scholar 

  34. Sun, J., Liu, C.: Finite-horizon differential games for missile–target interception system using adaptive dynamic programming with input constraints. Int. J. Syst. Sci. 49(2), 1–20 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  35. Liu, Y., Tong, S., Li, D., Gao, Y.: Fuzzy adaptive control with state observer for a class of nonlinear discrete-time systems with input constraint. IEEE Trans. Fuzzy Syst. 24(5), 1147–1158 (2016)

    Article  Google Scholar 

  36. Chen, M., Ge, S., Ren, B.: Adaptive tracking control of uncertain MIMO nonlinear systems with input constraints. Automatica 47(3), 452–465 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  37. Song, G., Li, T., Hu, K., Zheng, B.: Observer-based quantized control of nonlinear systems with input saturation. Nonlinear Dyn. 86(2), 1157–1169 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  38. He, P., Jagannathan, S.: Reinforcement learning-based output feedback control of nonlinear systems with input constraints. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 35(1), 150–154 (2005)

    Article  Google Scholar 

  39. Abu-Khalaf, M., Lewis, F.L., Huang, J.: Neurodynamic programming and zero-sum games for constrained control systems. IEEE Trans. Neural Netw. 19(7), 1243–1252 (2008)

    Article  Google Scholar 

  40. Heydari, A., Balakrishnan, S.N.: Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics. IEEE Trans. Neural Netw. Learn. Syst. 24(1), 145–157 (2013)

    Article  Google Scholar 

  41. Zhang, H., Qin, C., Luo, Y.: Neural-network-based constrained optimal control scheme for discrete-time switched nonlinear system using dual heuristic programming. IEEE Trans. Autom. Sci. Eng. 11(3), 839–849 (2014)

    Article  Google Scholar 

  42. Dong, L., Zhong, X., Sun, C., He, H.: Event-triggered adaptive dynamic programming for continuous-time systems with control constraints. IEEE Trans. Neural Netw. Learn. 28(8), 1941–1952 (2017)

    Article  MathSciNet  Google Scholar 

  43. Modares, H., Lewis, F.L., Naghibi-Sistani, M.B.: Adaptive optimal control of unknown constrained-input systems using policy iteration and neural networks. IEEE Trans. Neural Netw. Learn. Syst. 24(10), 1513–1525 (2013)

    Article  Google Scholar 

  44. Xu, H., Zhao, Q., Jagannathan, S.: Finite-horizon near-optimal output feedback neural network control of quantized nonlinear discrete-time systems with input constraint. IEEE Trans. Neural Netw. Learn. Syst. 26(8), 1776–1788 (2015)

    Article  MathSciNet  Google Scholar 

  45. Song, R., Zhang, H., Luo, Y., Wei, Q.: Optimal control laws for time-delay systems with saturating actuators based on heuristic dynamic programming. Neurocomputing 73(16), 3020–3027 (2010)

    Article  Google Scholar 

  46. Zhang, H., Luo, Y., Liu, D.: Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans. Neural Netw. 20(9), 1490–1503 (2009)

    Article  Google Scholar 

  47. Liu, D., Yang, X., Wang, D., Wei, Q.: Reinforcement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints. IEEE Trans. Cybern. 45(7), 1372–1385 (2015)

    Article  Google Scholar 

  48. Liu, D., Wang, D., Yang, X.: An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs. Inform Sci. 220, 331–342 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  49. Yang, X., Liu, D., Ma, H., Xu, Y.: Online approximate solution of HJI equation for unknown constrained-input nonlinear continuous-time systems. Inform Sci. 328, 435–454 (2016)

    Article  MATH  Google Scholar 

  50. Yang, X., Liu, D., Luo, B., Li, C.: Data-based robust adaptive control for a class of unknown nonlinear constrained-input systems via integral reinforcement learning. Inform Sci. 369, 731–747 (2016)

    Article  Google Scholar 

  51. Pomprapa, A., Leonhardt, S., Misgeld, B.J.E.: Optimal learning control of oxygen saturation using a policy iteration algorithm and a proof-of-concept in an interconnecting three-tank system. Control Eng. Pract. 59, 194–203 (2017)

    Article  Google Scholar 

  52. Yang, X., Liu, D., Wang, D.: Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints. Int. J. Contr. 87(3), 553–566 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  53. Song, R., Lewis, F.L., Wei, Q., Zhang, H.: Off-policy actor-critic structure for optimal control of unknown systems with disturbances. IEEE Trans. Cybern. 46(5), 1041–1050 (2016)

    Article  Google Scholar 

  54. Wu, H.N., Li, M.M., Guo, L.: Finite-horizon approximate optimal guaranteed cost control of uncertain nonlinear systems with application to mars entry guidance. IEEE Trans. Neural Netw. Learn. Syst. 26(7), 1456–1467 (2015)

    Article  MathSciNet  Google Scholar 

  55. Terra, M.H., Cerri, J.P., Ishihara, J.Y.: Optimal robust linear quadratic regulator for systems subject to uncertainties. IEEE Trans. Automat. Control 59(9), 2586–2591 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  56. Cui, X., Zhang, H., Luo, Y., Jiang, H.: Adaptive dynamic programming for \(H_\infty \) tracking design of uncertain nonlinear systems with disturbances and input constraints. Int. J. Adapt. Control. 31(11), 1567–1583 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  57. Zhao, B., Liu, D., Li, Y.: Observer based adaptive dynamic programming for fault tolerant control of a class of nonlinear systems. Inform Sci. 384, 21–33 (2017)

    Article  Google Scholar 

  58. Liu, D., Wei, Q.: Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 25(3), 621–634 (2014)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grants 61603387, 61533017, 61374051 and 61502494, and in part by the Early Career Development Award of SKLMCCS under Grant 20180201.

Author information

Authors and Affiliations

  1. The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Bo Zhao

  2. Research Center for Brain-inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Lihao Jia

  3. Department of Control Science and Engineering, Changchun University of Technology, Changchun, 130012, China

    Hongbing Xia & Yuanchun Li

Authors
  1. Bo Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lihao Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongbing Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuanchun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Zhao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, B., Jia, L., Xia, H. et al. Adaptive dynamic programming-based stabilization of nonlinear systems with unknown actuator saturation. Nonlinear Dyn 93, 2089–2103 (2018). https://doi.org/10.1007/s11071-018-4309-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-018-4309-8

Keywords

Search

Navigation