Event-Triggered Nonlinear \(H_{\infty }\) Control Design via an Improved Critic Learning Strategy

  • Ding WangEmail author
  • Chaoxu Mu
Part of the Studies in Systems, Decision and Control book series (SSDC, volume 167)


In this chapter, we aim at improving the critic learning criterion to cope with the event-based nonlinear \(H_{\infty }\) state feedback control design. First of all, the \(H_{\infty }\) control problem is regarded as a two-player zero-sum game and the adaptive critic mechanism is used to achieve the minimax optimization under event-based environment. Then, based on an improved updating rule, the event-based optimal control law and the time-based worst-case disturbance law are obtained approximately by training a single critic neural network. The initial stabilizing control is no longer required during the implementation process of the new algorithm. Next, the closed-loop system is formulated as an impulsive model and its stability issue is handled by incorporating the improved learning criterion. The infamous Zeno behavior of the present event-based design is also avoided through theoretical analysis on the lower bound of the minimal inter-sample time. Finally, the applications to an aircraft dynamics and a robot arm plant are carried out to verify the efficient performance of the present novel design method.


  1. 1.
    Bian, T., Jiang, Y., Jiang, Z.P.: Decentralized adaptive optimal control of large-scale systems with application to power systems. IEEE Trans. Industr. Electron. 62(4), 2439–2447 (2015)CrossRefGoogle Scholar
  2. 2.
    Cheng, L., Wang, Y., Ren, W., Hou, Z.G., Tan, M.: Containment control of multi-agent systems with dynamic leaders based on a \(PI^n\)-type approach. IEEE Trans. Cybern. 46(12), 3004–3017 (2016)Google Scholar
  3. 3.
    Dierks T., Jagannathan, S.: Optimal control of affine nonlinear continuous-time systems. In: Proceedings of the American Control Conference, pp. 1568–1573 (2010)Google Scholar
  4. 4.
    Dierks, T., Thumati, B.T., Jagannathan, S.: Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence. Neural Netw. 22(5–6), 851–860 (2009)CrossRefGoogle Scholar
  5. 5.
    Gao, W., Jiang, Z.P.: Adaptive dynamic programming and adaptive optimal output regulation of linear systems. IEEE Trans. Autom. Control 61(12), 4164–4169 (2016)MathSciNetCrossRefGoogle Scholar
  6. 6.
    He, W., Yin, Z., Sun, C.: Adaptive neural network control of a marine vessel with constraints using the asymmetric barrier Lyapunov function. IEEE Trans. Cybern. 47(7), 1641–1651 (2017)CrossRefGoogle Scholar
  7. 7.
    Heydari, A., Balakrishnan, S.N.: Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics. IEEE Trans. Neural Netw. Learn. Syst. 24(1), 145–157 (2013)CrossRefGoogle Scholar
  8. 8.
    Jiang, Y., Jiang, Z.P.: Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Trans. Autom. Control 60(11), 2917–2929 (2015)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Khalil, H.K.: Nonlinear Systems, 3rd edn. Prentice-Hall, New Jersey (2002)zbMATHGoogle Scholar
  10. 10.
    Kim, Y.H., Lewis, F.L., Abdallah, C.T.: A dynamic recurrent neural-network-based adaptive observer for a class of nonlinear systems. Automatica 33(8), 1539–1543 (1997)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Li, H., Liu, D.: Optimal control for discrete-time affine non-linear systems using general value iteration. IET Control Theory Appl. 6(18), 2725–2736 (2012)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Liu, D., Li, H., Wang, D.: Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm. Neurocomputing 110, 92–100 (2013)CrossRefGoogle Scholar
  13. 13.
    Liu, D., Li, H., Wang, D.: Online synchronous approximate optimal learning algorithm for multiplayer nonzero-sum games with unknown dynamics. IEEE Trans. Syst. Man Cybern. Syst. 44(8), 1015–1027 (2014)CrossRefGoogle Scholar
  14. 14.
    Liu, D., Wang, D., Wang, F.Y., Li, H., Yang, X.: Neural-network-based online HJB solution for optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems. IEEE Trans. Cybern. 44(12), 2834–2847 (2014)CrossRefGoogle Scholar
  15. 15.
    Liu, Y.J., Tong, S., Chen, C.L.P., Li, D.J.: Neural controller design-based adaptive control for nonlinear MIMO systems with unknown hysteresis inputs. IEEE Trans. Cybern. 46(1), 9–19 (2016)CrossRefGoogle Scholar
  16. 16.
    Luo, B., Wu, H.N.: Computationally efficient simultaneous policy update algorithm for nonlinear \(H_{\infty }\) state feedback control with Galerkin’s method. Int. J. Robust Nonlinear Control 23(9), 991–1012 (2013)Google Scholar
  17. 17.
    Modares, H., Lewis, F.L.: Linear quadratic tracking control of partially-unknown continuous-time systems using reinforcement learning. IEEE Trans. Autom. Control 59(11), 3051–3056 (2014)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Mu, C., Sun, C., Song, A., Yu, H.: Iterative GDHP-based approximate optimal tracking control for a class of discrete-time nonlinear systems. Neurocomputing 214, 775–784 (2016)CrossRefGoogle Scholar
  19. 19.
    Mu, C., Ni, Z., Sun, C., He, H.: Air-breathing hypersonic vehicle tracking control based on adaptive dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 584–598 (2017)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Sahoo, A., Xu, H., Jagannathan, S.: Neural network-based event-triggered state feedback control of nonlinear continuous-time systems. IEEE Trans. Neural Netw. Learn. Syst. 27(3), 497–509 (2016)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Vamvoudakis, K.G., Lewis, F.L.: Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5), 878–888 (2010)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Vamvoudakis, K.G., Mojoodi, A., Ferraz, H.: Event-triggered optimal tracking control of nonlinear systems. Int. J. Robust Nonlinear Control 27(4), 598–619 (2017)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Wang, D., Liu, D., Wei, Q., Zhao, D., Jin, N.: Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica 48(8), 1825–1832 (2012)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Wang, D., Liu, D., Li, H.: Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems. IEEE Trans. Autom. Sci. Eng. 11(2), 627–632 (2014)CrossRefGoogle Scholar
  25. 25.
    Wang, D., Liu, D., Zhang, Q., Zhao, D.: Data-based adaptive critic designs for nonlinear robust optimal control with uncertain dynamics. IEEE Trans. Syst. Man Cybern. Syst. 46(11), 1544–1555 (2016)CrossRefGoogle Scholar
  26. 26.
    Wang, D., Li, C., Liu, D., Mu, C.: Data-based robust optimal control of continuous-time affine nonlinear systems with matched uncertainties. Inf. Sci. 366, 121–133 (2016)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Wang, D., He, H., Liu, D.: Improving the critic learning for event-based nonlinear \(H_{\infty }\) control design. IEEE Trans. Cybern. 47(10), 3417–3428 (2017)CrossRefGoogle Scholar
  28. 28.
    Wang, D., Mu, C., He, H., Liu, D.: Event-driven adaptive robust control of nonlinear systems with uncertainties through NDP strategy. IEEE Trans. Syst. Man Cybern. Syst. 47(7), 1358–1370 (2017)CrossRefGoogle Scholar
  29. 29.
    Wang, D., Mu, C., Liu, D.: Adaptive critic designs for solving event-based \(H_{\infty }\) control problems. In: Proceedings of American Control Conference, Seattle, WA, USA, May 2017, pp. 2435–2440 (2017)Google Scholar
  30. 30.
    Werbos, P.J.: Approximate dynamic programming for real-time control and neural modeling. Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, pp. 493–526 (1992)Google Scholar
  31. 31.
    Yang, X., Liu, D., Ma, H., Xu, Y.: Online approximate solution of HJI equation for unknown constrained-input nonlinear continuous-time systems. Inf. Sci. 328, 435–454 (2016)CrossRefGoogle Scholar
  32. 32.
    Zhang, H., Qin, C., Jiang, B., Luo, Y.: Online adaptive policy learning algorithm for \(H_{\infty }\) state feedback control of unknown affine nonlinear discrete-time systems. IEEE Trans. Cybern. 44(12), 2706–2718 (2014)CrossRefGoogle Scholar
  33. 33.
    Zhang, H., Jiang, H., Luo, Y., Xiao, G.: Data-driven optimal consensus control for discrete-time multi-agent systems with unknown dynamics using reinforcement learning method. IEEE Trans. Industr. Electron. 64(5), 4091–4100 (2017)CrossRefGoogle Scholar
  34. 34.
    Zhang, Q., Zhao, D., Zhu, Y.: Event-triggered \(H_{\infty }\) control for continuous-time nonlinear system via concurrent learning. IEEE Trans. Syst. Man Cybern. Syst. 47(7), 1071–1081 (2017)CrossRefGoogle Scholar
  35. 35.
    Zhao, Q., Xu, H., Jagannathan, S.: Near optimal output feedback control of nonlinear discrete-time systems based on reinforcement neural network learning. IEEE/CAA J. Autom. Sin. 1(4), 372–384 (2014)CrossRefGoogle Scholar
  36. 36.
    Zhong, X., He, H.: An event-triggered ADP control approach for continuous-time system with unknown internal states. IEEE Trans. Cybern. 47(3), 683–694 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.The State Key Laboratory of Management and Control for Complex SystemsInstitute of Automation, Chinese Academy of SciencesBeijingChina
  2. 2.School of Electrical and Information EngineeringTianjin UniversityTianjinChina

Personalised recommendations