Skip to main content
SpringerLink
Log in

Robust and Optimal Guaranteed Cost Control of Continuous-Time Nonlinear Systems

  • Chapter
  • First Online:
Book cover Adaptive Dynamic Programming with Applications in Optimal Control

Part of the book series: Advances in Industrial Control ((AIC))

  • 3888 Accesses

Abstract

In this chapter, the robust control and optimal guaranteed cost control of continuous-time uncertain nonlinear systems are studied using adaptive dynamic programming (ADP) methods. First, a novel strategy is established to design the robust controller for a class of nonlinear systems with uncertainties based on online policy iteration algorithm. By properly choosing a cost function that reflects the uncertainties, states, and controls, the robust control problem is transformed into an optimal control problem, which is solved under the framework of ADP. Then, the infinite horizon optimal guaranteed cost control of uncertain nonlinear systems is investigated. A critic neural network is constructed to facilitate the solution of the modified Hamilton–Jacobi–Bellman equation corresponding to the nominal system. An additional stabilizing term is introduced to ensure stability, which reinforces the updating process of the weight vector and reduces the requirement of an initial stabilizing control. The uniform ultimate boundedness of the closed-loop system is analyzed by using the Lyapunov’s direct approach. Simulation examples are provided to verify the effectiveness of the present control approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5):779–791

    Article  MathSciNet  MATH  Google Scholar 

  2. Adhyaru DM, Kar IN, Gopal M (2009) Fixed final time optimal control approach for bounded robust controller design using Hamilton-Jacobi-Bellman solution. IET Control Theory Appl 3(9):1183–1195

    Article  MathSciNet  Google Scholar 

  3. Adhyaru DM, Kar IN, Gopal M (2011) Bounded robust control of nonlinear systems using neural network-based HJB solution. Neural Comput Appl 20(1):91–103

    Article  Google Scholar 

  4. Beard RW, Saridis GN, Wen JT (1997) Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation. Automatica 33(12):2159–2177

    Article  MathSciNet  MATH  Google Scholar 

  5. Bhasin S, Kamalapurkar R, Johnson M, Vamvoudakis KG, Lewis FL, Dixon WE (2013) A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems. Automatica 49(1):82–92

    Article  MathSciNet  MATH  Google Scholar 

  6. Chang SSL, Peng TKC (1972) Adaptive guaranteed cost control of systems with uncertain parameters. IEEE Trans Autom Control 17(4):474–483

    Article  MathSciNet  MATH  Google Scholar 

  7. Dierks T, Jagannathan S (2010) Optimal control of affine nonlinear continuous-time systems. In: Proceedings of the American Control Conference, pp 1568–1573

    Google Scholar 

  8. Dierks T, Jagannathan S (2012) Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update. IEEE Trans Neural Netw Learn Syst 23(7):1118–1129

    Article  Google Scholar 

  9. Haddad WM, Chellaboina V (2008) Nonlinear dynamical systems and control: a Lyapunov-based approach. Princeton University Press, Princeton

    MATH  Google Scholar 

  10. Haddad WM, Chellaboina V, Fausz JL (1998) Robust nonlinear feedback control for uncertain linear systems with nonquadratic performance criteria. Syst Control Lett 33(5):327–338

    Article  MathSciNet  MATH  Google Scholar 

  11. Haddad WM, Chellaboina V, Fausz JL, Leonessa A (2000) Optimal non-linear robust control for nonlinear uncertain systems. Int J Control 73(4):329–342

    Article  MathSciNet  MATH  Google Scholar 

  12. Heydari A, Balakrishnan SN (2013) Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics. IEEE Trans Neural Netw Learn Syst 24(1):145–157

    Article  Google Scholar 

  13. LaSalle JP, Lefschetz S (1967) Stability by Liapunov’s direct method with applications. Academic Press, New York

    Google Scholar 

  14. Lewis FL, Vrabie D (2009) Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst Mag 9(3):32–50

    Article  MathSciNet  Google Scholar 

  15. Lewis FL, Jagannathan S, Yesildirek A (1999) Neural network control of robot manipulators and nonlinear systems. Taylor & Francis, London

    Google Scholar 

  16. Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst Mag 32(6):76–105

    Article  MathSciNet  Google Scholar 

  17. Li H, Liu D (2012) Optimal control for discrete-time affine nonlinear systems using general value iteration. IET Control Theory Appl 6(18):2725–2736

    Article  MathSciNet  Google Scholar 

  18. Liang J, Venayagamoorthy GK, Harley RG (2012) Wide-area measurement based dynamic stochastic optimal power flow control for smart grids with high variability and uncertainty. IEEE Trans Smart Grid 3(1):59–69

    Article  Google Scholar 

  19. Lin F, Brand RD, Sun J (1992) Robust control of nonlinear systems: compensating for uncertainty. Int J Control 56(6):1453–1459

    Article  MathSciNet  MATH  Google Scholar 

  20. Liu D, Wang D, Zhao D, Wei Q, Jin N (2012) Neural-network-based optimal control for a class of unknown discrete-time nonlinear systems using globalized dual heuristic programming. IEEE Trans Autom Sci Eng 9(3):628–634

    Article  Google Scholar 

  21. Liu D, Yang X, Li H (2013) Adaptive optimal control for a class of continuous-time affine nonlinear systems with unknown internal dynamics. Neural Comput Appl 23:1843–1850

    Article  Google Scholar 

  22. Liu D, Wang D, Yang X (2013) An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs. Inf Sci 220:331–342

    Article  MathSciNet  MATH  Google Scholar 

  23. Liu D, Huang Y, Wang D, Wei Q (2013) Neural-network-observer-based optimal control for unknown nonlinear systems using adaptive dynamic programming. Int J Control 86(9):1554–1566

    Article  MathSciNet  MATH  Google Scholar 

  24. Liu D, Li H, Wang D (2013) Data-based self-learning optimal control: Research progress and prospects. Acta Autom Sinica 39(11):1858–1870

    Article  MathSciNet  MATH  Google Scholar 

  25. Liu D, Li H, Wang D (2013) Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm. Neurocomputing 110:92–100

    Article  Google Scholar 

  26. Liu D, Li H, Wang D (2014) Online synchronous approximate optimal learning algorithm for multi-player non-zero-sum games with unknown dynamics. IEEE Trans Syst Man Cybern Syst 44(8):1015–1027

    Article  Google Scholar 

  27. Liu D, Wang D, Li H (2014) Decentralized stabilization for a class of continuous-time nonlinear interconnected systems using online learning optimal control approach. IEEE Trans Neural Netw Learn Syst 25(2):418–428

    Article  MathSciNet  Google Scholar 

  28. Liu D, Wang D, Wang FY, Li H, Yang X (2014) Neural-network-based online HJB solution for optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems. IEEE Trans Cybern 44(12):2834–2847

    Article  Google Scholar 

  29. Mehraeen S, Jagannathan S (2011) Decentralized optimal control of a class of interconnected nonlinear discrete-time systems by using online Hamilton-Jacobi-Bellman formulation. IEEE Trans Neural Netw 22(11):1757–1769

    Article  Google Scholar 

  30. Michel AN, Hou L, Liu D (2015) Stability of dynamical systems: On the role of monotonic and non-monotonic Lyapunov functions. Birkhäuser, Boston

    Book  MATH  Google Scholar 

  31. Modares H, Lewis FL, Naghibi-Sistani MB (2013) Adaptive optimal control of unknown constrained-input systems using policy iteration and neural networks. IEEE Trans Neural Netw Learn Syst 24(10):1513–1525

    Article  Google Scholar 

  32. Murray JJ, Cox CJ, Lendaris GG, Saeks R (2002) Adaptive dynamic programming. IEEE Trans Syst Man Cybern Part C Appl Rev 32(2):140–153

    Article  Google Scholar 

  33. Nevistic V, Primbs JA (1996) Constrained nonlinear optimal control: a converse HJB approach. Technical Memorandum No. CIT-CDS, California Institute of Technology, Pasadena, CA, pp 96–021

    Google Scholar 

  34. Ni Z, He H, Wen J (2013) Adaptive learning in tracking control based on the dual critic network design. IEEE Trans Neural Netw Learn Syst 24(6):913–928

    Article  Google Scholar 

  35. Ni Z, He H, Wen J, Xu X (2013) Goal representation heuristic dynamic programming on maze navigation. IEEE Trans Neural Netw Learn Syst 24(12):2038–2050

    Article  Google Scholar 

  36. Nodland D, Zargarzadeh H, Jagannathan S (2013) Neural network-based optimal adaptive output feedback control of a helicopter UAV. IEEE Trans Neural Netw Learn Syst 24(7):1061–1073

    Article  Google Scholar 

  37. Prokhorov DV, Wunsch DC (1997) Adaptive critic designs. IEEE Trans Neural Netw 8(5):997–1007

    Article  Google Scholar 

  38. Rudin W (1976) Principles of mathematical analysis. McGraw-Hill, New York

    MATH  Google Scholar 

  39. Si J, Wang YT (2001) On-line learning control by association and reinforcement. IEEE Trans Neural Netw 12(2):264–276

    Article  MathSciNet  Google Scholar 

  40. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge

    Google Scholar 

  41. Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888

    Article  MathSciNet  MATH  Google Scholar 

  42. Vrabie D, Lewis FL (2009) Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Netw 22(3):237–246

    Article  MATH  Google Scholar 

  43. Wang D, Liu D (2013) Neuro-optimal control for a class of unknown nonlinear dynamic systems using SN-DHP technique. Neurocomputing 121:218–225

    Article  Google Scholar 

  44. Wang D, Liu D, Wei Q (2012) Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach. Neurocomputing 78(1):14–22

    Article  Google Scholar 

  45. Wang D, Liu D, Wei Q, Zhao D, Jin N (2012) Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica 48(8):1825–1832

    Article  MathSciNet  MATH  Google Scholar 

  46. Wang D, Liu D, Zhao D, Huang Y, Zhang D (2013) A neural-network-based iterative GDHP approach for solving a class of nonlinear optimal control problems with control constraints. Neural Comput Appl 22(2):219–227

    Article  Google Scholar 

  47. Wang D, Liu D, Li H (2014) Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems. IEEE Trans Autom Sci Eng 11(2):627–632

    Article  Google Scholar 

  48. Wang D, Liu D, Li H, Ma H (2014) Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming. Inf Sci 282:167–179

    Article  MathSciNet  Google Scholar 

  49. Wang FY, Zhang H, Liu D (2009) Adaptive dynamic programming: an introduction. IEEE Comput Intell Mag 4(2):39–47

    Article  Google Scholar 

  50. Wang FY, Jin N, Liu D, Wei Q (2011) Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with \(\varepsilon \)-error bound. IEEE Trans Neural Netw 22(1):24–36

    Article  Google Scholar 

  51. Werbos PJ (1977) Advanced forecasting methods for global crisis warning and models of intelligence. General Syst Yearb 22:25–38

    Google Scholar 

  52. Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA (eds) Handbook of intelligent control: neural, fuzzy, and adaptive approaches (Chapter 13). Van Nostrand Reinhold, New York

    Google Scholar 

  53. Werbos PJ (2008) ADP: The key direction for future research in intelligent control and understanding brain intelligence. IEEE Trans Syst Man Cybern Part B Cybern 38(4):898–900

    Article  Google Scholar 

  54. Werbos PJ (2009) Intelligence in the brain: a theory of how it works and how to build it. Neural Netw 22(3):200–212

    Article  Google Scholar 

  55. Wu HN, Luo B (2012) Neural network based online simultaneous policy update algorithm for solving the HJI equation in nonlinear \(H_{\infty }\) control. IEEE Trans Neural Netw Learn Syst 23(12):1884–1895

    Article  MathSciNet  Google Scholar 

  56. Xu H, Jagannathan S, Lewis FL (2012) Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses. Automatica 48(6):1017–1030

    Article  MathSciNet  MATH  Google Scholar 

  57. Xu X, Lian C, Zuo L, He H (2014) Kernel-based approximate dynamic programming for real-time online learning control: an experimental study. IEEE Trans Control Syst Technol 22(1):146–156

    Article  Google Scholar 

  58. Yu L, Chu J (1999) An LMI approach to guaranteed cost control of linear uncertain time-delay systems. Automatica 35(6):1155–1159

    Article  MathSciNet  MATH  Google Scholar 

  59. Yu L, Han QL, Sun MX (2005) Optimal guaranteed cost control of linear uncertain systems with input constraints. Int J Control Autom Syst 3(3):397–402

    Google Scholar 

  60. Zhang H, Cui L, Luo Y (2013) Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network ADP. IEEE Trans Cybern 43(1):206–216

    Article  Google Scholar 

  61. Zhang H, Zhang X, Luo Y, Yang J (2013) An overview of research on adaptive dynamic programming. Acta Autom Sinica 39(4):303–311

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Derong Liu, Qinglai Wei & Ding Wang

  2. Tianjin University, Tianjin, China

    Xiong Yang

  3. Tencent Inc., Shenzhen, China

    Hongliang Li

Authors
  1. Derong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qinglai Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ding Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hongliang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Derong Liu .

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Liu, D., Wei, Q., Wang, D., Yang, X., Li, H. (2017). Robust and Optimal Guaranteed Cost Control of Continuous-Time Nonlinear Systems. In: Adaptive Dynamic Programming with Applications in Optimal Control. Advances in Industrial Control. Springer, Cham. https://doi.org/10.1007/978-3-319-50815-3_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-50815-3_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-50813-9

  • Online ISBN: 978-3-319-50815-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation