Skip to main content
SpringerLink
Log in
Book cover

Adaptive Critic Control with Robust Stabilization for Uncertain Nonlinear Systems pp 173–197Cite as

An Improved Adaptive Optimal Regulation Framework with Robust Control Synthesis

  • Chapter
  • First Online:

  • 635 Accesses

Part of the book series: Studies in Systems, Decision and Control ((SSDC,volume 167))

Abstract

In this chapter, we focus on developing adaptive optimal regulators for a class of continuous-time nonlinear dynamical systems through an improved neural learning mechanism. The main objective lies in that establishing an additional stabilizing term to reinforce the traditional training process of the critic neural network, so that to reduce the requirement with respect to the initial stabilizing control, and therefore, bring in an obvious convenience to the adaptive-critic-based learning control implementation. It is exhibited that by employing the novel updating rule, the adaptive optimal control law can be obtained with an excellent approximation property. The closed-loop system is constructed and its stability issue is handled by considering the improved learning criterion. After that, we apply the adaptation-oriented approximate optimal control strategy to perform robust stabilization when including complex nonlinearity and uncertainty. By considering the dynamical uncertainties, it is proven that the developed near-optimal control law can achieve uniform ultimate boundedness of the closed-loop state vector, thereby guaranteeing a certain extent of robustness for the uncertain nonlinear plant. Simulation for a classical nonlinear system and experiment on an overhead crane are conducted to verify the efficient performance of the present design methods, especially the major role that the stabilizing term performed.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Bian, T., Jiang, Y., Jiang, Z.P.: Decentralized adaptive optimal control of large-scale systems with application to power systems. IEEE Trans. Industr. Electron. 62(4), 2439–2447 (2015)

    Article  Google Scholar 

  2. Chen, M., Tao, G.: Adaptive fault-tolerant control of uncertain nonlinear large-scale systems with unknown dead zone. IEEE Trans. Cybern. 46(8), 1851–1862 (2016)

    Article  Google Scholar 

  3. Dierks, T., Jagannathan, S.: Optimal control of affine nonlinear continuous-time systems. In: Proceedings of the American Control Conference, pp. 1568–1573 (2010)

    Google Scholar 

  4. Dierks, T., Thumati, B.T., Jagannathan, S.: Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence. Neural Netw. 22(5–6), 851–860 (2009)

    Article  Google Scholar 

  5. Gao, W., Jiang, Z.P.: Adaptive dynamic programming and adaptive optimal output regulation of linear systems. IEEE Trans. Autom. Control 61(12), 4164–4169 (2016)

    Article  MathSciNet  Google Scholar 

  6. He, W., Dong, Y., Sun, C.: Adaptive neural impedance control of a robotic manipulator with input saturation. IEEE Trans. Syst. Man Cybern. Syst. 46(3), 334–344 (2016)

    Article  Google Scholar 

  7. Heydari, A., Balakrishnan, S.N.: Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics. IEEE Trans. Neural Netw. Learn. Syst. 24(1), 145–157 (2013)

    Article  Google Scholar 

  8. Jiang, Y., Jiang, Z.P.: Robust adaptive dynamic programming and feedback stabilization of nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 25(5), 882–893 (2014)

    Article  Google Scholar 

  9. Jiang, Y., Jiang, Z.P.: Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Trans. Autom. Control 60(11), 2917–2929 (2015)

    Article  MathSciNet  Google Scholar 

  10. Lewis, F.L., Liu, D.: Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. Wiley, New Jersey (2013)

    Google Scholar 

  11. Lewis, F.L., Jagannathan, S., Yesildirek, A.: Neural Network Control of Robot Manipulators and Nonlinear Systems. Taylor & Francis, London (1998)

    Google Scholar 

  12. Li, C., Gao, J., Yi, J., Zhang, G.: Analysis and design of functionally weighted single-input-rule-modules connected fuzzy inference systems. IEEE Trans. Fuzzy Syst. 26(1), 56–71 (2018)

    Article  Google Scholar 

  13. Liu, D., Wei, Q., Wang, D., Yang, X., Li, H.: Adaptive Dynamic Programming with Applications in Optimal Control. Springer, London (2017)

    Book  Google Scholar 

  14. Liu, Y.J., Tong, S., Chen, C.L.P., Li, D.J.: Neural controller design-based adaptive control for nonlinear MIMO systems with unknown hysteresis inputs. IEEE Trans. Cybern. 46(1), 9–19 (2016)

    Article  Google Scholar 

  15. Luo, B., Wu, H.N., Huang, T., Liu, D.: Reinforcement learning solution for HJB equation arising in constrained optimal control problem. Neural Netw. 71, 150–158 (2015)

    Article  Google Scholar 

  16. Modares, H., Lewis, F.L.: Linear quadratic tracking control of partially-unknown continuous-time systems using reinforcement learning. IEEE Trans. Autom. Control 59(11), 3051–3056 (2014)

    Article  MathSciNet  Google Scholar 

  17. Mu, C., Wang, D.: Neural-network-based adaptive guaranteed cost control of nonlinear dynamical systems with matched uncertainties. Neurocomputing 245, 46–54 (2017)

    Article  Google Scholar 

  18. Mu, C., Sun, C., Song, A., Yu, H.: Iterative GDHP-based approximate optimal tracking control for a class of discrete-time nonlinear systems. Neurocomputing 214, 775–784 (2016)

    Article  Google Scholar 

  19. Mu, C., Ni, Z., Sun, C., He, H.: Air-breathing hypersonic vehicle tracking control based on adaptive dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 584–598 (2017)

    Article  MathSciNet  Google Scholar 

  20. Mu, C., Wang, D., He, H.: Novel iterative neural dynamic programming for data-based approximate optimal control design. Automatica 81, 240–252 (2017)

    Article  MathSciNet  Google Scholar 

  21. Na, J., Herrmann, G.: Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuoustime unknown nonlinear systems. IEEE/CAA J. Autom. Sin. 1(4), 412–422 (2014)

    Article  Google Scholar 

  22. Nodland, D., Zargarzadeh, H., Jagannathan, S.: Neural network-based optimal adaptive output feedback control of a helicopter UAV. IEEE Trans. Neural Netw. Learn. Syst. 24(7), 1061–1073 (2013)

    Article  Google Scholar 

  23. Qin, C., Zhang, H., Wang, Y., Luo, Y.: Neural network-based online \(H_{\infty }\) control for discrete-time affine nonlinear system using adaptive dynamic programming. Neurocomputing 198, 91–99 (2016)

    Article  Google Scholar 

  24. Song, R., Wei, Q., Sun, Q.: Nearly finite-horizon optimal control for a class of nonaffine time-delay nonlinear systems based on adaptive dynamic programming. Neurocomputing 156, 166–175 (2015)

    Article  Google Scholar 

  25. Vamvoudakis, K.G., Lewis, F.L.: Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5), 878–888 (2010)

    Article  MathSciNet  Google Scholar 

  26. Vamvoudakis, K.G., Antsaklis, P.J., Dixon, W.E., Hespanha, J.P., Lewis, F.L., Modares, H., Kiumarsi, B.: Autonomy and machine intelligence in complex systems: a tutorial. In: Proceedings of the 2015 American Control Conference, pp. 5062–5079 (2015)

    Google Scholar 

  27. Vamvoudakis, K.G., Modares, H., Kiumarsi, B., Lewis, F.L.: Game theory-based control system algorithms with real-time reinforcement learning: how to solve multiplayer games online. IEEE Control Syst. Mag. 37(1), 33–52 (2017)

    Article  MathSciNet  Google Scholar 

  28. Wang, D.: Adaptation-oriented near-optimal control and robust synthesis of an overhead crane system. In: Proceedings of 24th International Conference on Neural Information Processing, Part VI, Guangzhou, China, November 2017, pp. 42–50 (2017)

    Google Scholar 

  29. Wang, D., Mu, C.: Developing nonlinear adaptive optimal regulators through an improved neural learning mechanism. Sci. China Inf. Sci. 60(5), 058201:1–058201:3 (2017)

    Google Scholar 

  30. Wang, D., Mu, C.: A novel neural optimal control framework with nonlinear dynamics: closed-loop stability and simulation verification. Neurocomputing 266, 353–360 (2017)

    Article  Google Scholar 

  31. Wang, D., Liu, D., Zhang, Q., Zhao, D.: Data-based adaptive critic designs for nonlinear robust optimal control with uncertain dynamics. IEEE Trans. Syst. Man Cybern. Syst. 46(11), 1544–1555 (2016)

    Article  Google Scholar 

  32. Wang, D., He, H., Liu, D.: Adaptive critic nonlinear robust control: a survey. IEEE Trans. Cybern. 47(10), 3429–3451 (2017)

    Article  Google Scholar 

  33. Wang, D., He, H., Mu, C., Liu, D.: Intelligent critic control with disturbance attenuation for affine dynamics including an application to a micro-grid system. IEEE Trans. Industr. Electron. 64(6), 4935–4944 (2017)

    Article  Google Scholar 

  34. Wang, D., Mu, C., Liu, D.: Data-driven nonlinear near-optimal regulation based on iterative neural dynamic programming. Acta Autom. Sin. 43(3), 366–375 (2017)

    MATH  Google Scholar 

  35. Wang, T., Tong, S.: Observer-based output-feedback asynchronous control for switched fuzzy systems. IEEE Trans. Cybern. 47(9), 2579–2591 (2016)

    Article  Google Scholar 

  36. Wang, Y., Cheng, L., Hou, Z.G., Yu, J., Tan, M.: Optimal formation of multi-robot systems based on a recurrent neural network. IEEE Trans. Neural Netw. Learn. Syst. 27(2), 322–333 (2016)

    Article  MathSciNet  Google Scholar 

  37. Werbos, P.J.: Approximate dynamic programming for real-time control and neural modeling. Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, pp. 493–526 (1992)

    Google Scholar 

  38. Xu, B.: Robust adaptive neural control of flexible hypersonic flight vehicle with dead-zone input nonlinearity. Nonlinear Dyn. 80(3), 1509–1520 (2015)

    Article  MathSciNet  Google Scholar 

  39. Yang, X., Liu, D., Wei, Q., Wang, D.: Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming. Neurocomputing 198, 80–90 (2016)

    Article  Google Scholar 

  40. Zhang, H., Cui, L., Luo, Y.: Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network ADP. IEEE Trans. Cybern. 43(1), 206–216 (2013)

    Article  Google Scholar 

  41. Zhang, H., Zhang, X., Luo, Y., Yang, J.: An overview of research on adaptive dynamic programming. Acta Autom. Sin. 39(4), 303–311 (2013)

    Article  Google Scholar 

  42. Zhang, Q., Zhao, D., Wang, D.: Event-based robust control for uncertain nonlinear systems using adaptive dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 29(1), 37–50 (2018)

    Article  MathSciNet  Google Scholar 

  43. Zhao, D., Xia, Z., Wang, D.: Model-free optimal control for affine nonlinear systems with convergence analysis. IEEE Trans. Autom. Sci. Eng. 12(4), 1461–1468 (2015)

    Article  Google Scholar 

  44. Zhao, Q., Xu, H., Jagannathan, S.: Near optimal output feedback control of nonlinear discrete-time systems based on reinforcement neural network learning. IEEE/CAA J. Autom. Sin. 1(4), 372–384 (2014)

    Article  Google Scholar 

  45. Zhong, X., He, H., Zhang, H., Wang, Z.: A neural network based online learning and control approach for Markov jump systems. Neurocomputing 149, 116–123 (2015)

    Article  Google Scholar 

  46. Zhong, X., Ni, Z., He, H.: A theoretical foundation of goal representation heuristic dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 27(12), 2513–2525 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Ding Wang

  2. School of Electrical and Information Engineering, Tianjin University, Tianjin, China

    Chaoxu Mu

Authors
  1. Ding Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chaoxu Mu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ding Wang .

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Wang, D., Mu, C. (2019). An Improved Adaptive Optimal Regulation Framework with Robust Control Synthesis. In: Adaptive Critic Control with Robust Stabilization for Uncertain Nonlinear Systems. Studies in Systems, Decision and Control, vol 167. Springer, Singapore. https://doi.org/10.1007/978-981-13-1253-3_6

Download citation

Publish with us

Policies and ethics

Search

Navigation