Skip to main content
SpringerLink
Log in

A Survey of Near-Optimal Control of Nonlinear Systems

  • Chapter
  • First Online:
Deep Reinforcement Learning with Guaranteed Performance

Part of the book series: Studies in Systems, Decision and Control ((SSDC,volume 265))

Abstract

For nonlinear dynamical systems, the optimal control problem generally requires solving a partial differential equation called the Hamilton–Jacobi–Bellman equation, of which the analytical solution generally cannot be obtained. However, the demand for optimal control keeps increasing, with the goal to save energy, reduce transient time, minimize error accumulation, etc. Consequently, methods were reported to approximately solve the problem leading to the so-called near-optimal control although their technical details differ. This research direction has experienced great progress in recent years but a timely review of them is still missing. This chapter serves as a brief survey for existing methods in this research direction.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Lewis, F.L., Vrabie, D., Syrmos, V.L.: Optimal Control. Wiley, New York (2012)

    Book  MATH  Google Scholar 

  2. Zhang, H., Li, L., Xu, J., Fu, M.: Linear quadratic regulation and stabilization of discrete-time systems with delay and multiplicative noise. IEEE Trans. Autom. Control 60(10), 2599–2613 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  3. Grieder, P., Borrelli, F., Torrisi, F., Morari, M.: Computation of the constrainedin infinite time linear quadratic regulator. Automtica 40, 701–708 (2004)

    Article  MATH  Google Scholar 

  4. Modares, H., Lewis, F.L., Jiang, Z.: Optimal output-feedback control of unknown continuous-time linear systems using off-policy reinforcement learning. IEEE Trans. Autom. Control 46(11), 2401–2410 (2016)

    Google Scholar 

  5. Anderson, B.D.O., Moore, J.B.: Optimal Control: Linear Quadratic Methods. Courier Corporation (2007)

    Google Scholar 

  6. Zhang, H., Duan, G., Xie, L.: Linear quadratic regulation for linear time-varying systems with multiple input delays. Automatica 42, 1465–1476 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  7. Modares, H., Lewis, F.L.: Linear quadratic tracking control of partially-unknown continuous-time systems using reinforcement learning. IEEE Trans. Autom. Control 59(11), 3051–3056 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  8. Zhou, J., Wen, C., Zhang, Y.: Adaptive output control of nonlinear systems with uncertain dead-zone nonlinearity. IEEE Trans. Autom. Control 51(3), 504–511 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  9. Zhang, Y., Li, S.: Time-scale expansion-based approximated optimal control for underactuated systems using projection neural networks. IEEE Trans. Syst. Man, Cybern., Syst. in press. https://doi.org/10.1109/TSMC.2017.2703140

    Article  Google Scholar 

  10. Liang, X., Fang, Y., Sun, N., Lin, H.: Nonlinear hierarchical control for unmanned quadrotor transportation systems. IEEE Trans. Ind. Electron. 65(4), 3395–3405 (2018)

    Article  Google Scholar 

  11. Xiao, B., Yin, S.: Velocity-free fault-tolerant and uncertainty attenuation control for a class of nonlinear systems. IEEE Trans. Autom. Control 63(7), 4400–4411 (2016)

    Google Scholar 

  12. Sassano, M., Astolf, A.: Dynamic approximate solutions of the HJ inequality and of the HJB equation for input-affine nonlinear systems. IEEE Trans. Autom. Control 57(10), 2490–2503 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  13. Zhang, Y., Li, S., Jiang, X.: Near-optimal control without solving HJB equations and its applications. IEEE Trans. Ind. Electron. 65(9), 7173–7184 (2018)

    Article  Google Scholar 

  14. Bertsekas, D.P.: Dynamic programming and suboptimal control: a survey from ADP to MPC. Eur. J. Control 11(4), 310–334 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  15. Mayne, D.Q.: Model predictive control: recent developments and future promise. Automatica 50(12), 2967–2986 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  16. Lee, J.H.: Model predictive control: review of the three decades of development. Int. J. Control Autom. Syst. 9(3), 415–424 (2011)

    Article  Google Scholar 

  17. Mesbah, A.: Stochastic model predictive control with active uncertainty learning: a survey on dual control. Annu. Rev. Control 45, 107–117 (2018)

    Article  MathSciNet  Google Scholar 

  18. Vazquez, S., Leon, J.I., Franquelo, L.G., Rodriguez, J., Young, H.A., Marquez, A., Zanchetta, P.: Model predictive control: a review of its applications in power electronics. IEEE Ind. Electron. Mag. 8(1), 16–31 (2014)

    Article  Google Scholar 

  19. Kiumarsi, B., Vamvoudakis, K.G., Modares, H., Lewis, F.L.: Optimal and autonomous control using reinforcement learning: a survey. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2042–2062 (2018)

    Article  MathSciNet  Google Scholar 

  20. Khan, S.G., Herrmann, G., Lewis, F.L., Pipe, T., Melhuish, C.: Reinforcement learning and optimal adaptive control: an overview and implementation examples. Annu. Rev. Control 36, 42–59 (2012)

    Article  Google Scholar 

  21. Wang, D., He, H., Liu, D.: Adaptive critic nonlinear robust control: a survey. IEEE Trans. Cybern. 47(10), 3429–3451 (2017)

    Article  Google Scholar 

  22. Kang, W., Wilcox, L.C.: Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations. Optimal Control Appl. 68(2), 289–315 (2017)

    MathSciNet  MATH  Google Scholar 

  23. Chen, W., Ballanceb, D.J., Gawthrop, P.J.: Optimal control of nonlinear systems: a predictive control approach. Automatica 39(4), 633–641 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  24. Liu, Y., Tong, S.: Optimal control-based adaptive NN design for a class of nonlinear discrete-time block-triangular systems. IEEE Trans. Cybern. 46(11), 2670–2680 (2016)

    Article  Google Scholar 

  25. Wang, D., Liu, D., Li, H., Ma, H.: Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming. Infom. Sci. 282, 167–179 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  26. Zhang, H., Qin, C., Luo, Y.: Neural-network-based constrained optimal control scheme for discrete-time switched nonlinear system using dual heuristic programming. IEEE Trans. Autom. Sic. Eng. 11(3), 839–849 (2014)

    Article  Google Scholar 

  27. Park, J., Sandberg, I.W.: Universal approximation using radial-basis-function networks. Neural comput. 3(2), 246–257 (1991)

    Article  Google Scholar 

  28. Vrabie, D., Lewis, F.: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Netw. 22(3), 237–246 (2009)

    Article  MATH  Google Scholar 

  29. Zhang, Y., Li, S., Liu, X.: Neural network-based model-free adaptive near-optimal tracking control for a class of nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. in press. https://doi.org/10.1109/TNNLS.2018.2828114

    Article  Google Scholar 

  30. Luo, B., Liu, D., Huang, T., Wang, D.: Model-free optimal tracking control via critic-only Q-learning. IEEE Trans. Neural Netw. Learn. Syst. 27(10), 2134–2144 (2016)

    Article  MathSciNet  Google Scholar 

  31. Tassa, Y., Erez, T.: Least squares solutions of the HJB equation with neural-network value-function approximators. IEEE Trans. Neural Netw. 18(4), 1–10 (2007)

    Article  Google Scholar 

  32. Dierks, T., Jagannathan, S.: Optimal control of affine nonlinear continuous-time systems. In: Proceedings of the ACC, pp. 1568–1573 (2010)

    Google Scholar 

  33. Wang, F., Zhang, H., Liu, D.: Adaptive dynamic programming: an introduction. IEEE Comput. Mag. 4(2), 39–47 (2009)

    Article  Google Scholar 

  34. Al-Tamimi, A., Lewis, F.L., Abu-Khalaf, M.: Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof. IEEE Trans. Syst., Man, Cybern. B, Cybern. 38(4), 943–949 (2008)

    Article  Google Scholar 

  35. Liu, D., Wei, Q.: Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 25(3), 621–634 (2014)

    Article  Google Scholar 

  36. Zhang, H., Song, R., Wei, Q., Zhang, T.: Optimal tracking control for a class of nonlinear discrete-time systems with time delays based on heuristic dynamic programming. IEEE Trans. Neural Netw. 22(12), 1851–1862 (2011)

    Article  Google Scholar 

  37. Zhang, H., Wei, Q., Luo, Y.: A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm. IEEE Trans. Syst., Man, Cybern. B, Cybern. 38(4), 937–942 (2008)

    Article  Google Scholar 

  38. Yang, L., Si, J., Tsakalis, K.S., Rodriguez, A.A.: Direct heuristic dynamic programming for nonlinear tracking control with filtered tracking error. IEEE Trans. Syst., Man, Cybern. B, Cybern. 39(6), 1617–1622 (2009)

    Article  Google Scholar 

  39. Dong, L., Zhong, X., Sun, C., He, H.: Adaptive event-triggered control based on heuristic dynamic programming for nonlinear discrete-time systems. IEEE Trans. Neural Netw. Learn. Syst. 28(7), 1594–1605 (2017)

    Article  MathSciNet  Google Scholar 

  40. Luo, B., Liu, D., Huang, T., Liu, J.: Output tracking control based on adaptive dynamic programming with multistep policy evaluation. IEEE Trans. Syst., Man., Cybern., Syst. to be published. https://doi.org/10.1109/TSMC.2017.2771516

    Article  Google Scholar 

  41. He, H., Ni, Z., Fu, J.: A three-network architecture for on-line learning and optimization based on adaptive dynamic programming. Neurocomputing 78(1), 3–13 (2012)

    Article  Google Scholar 

  42. Ni, Z., He, H., Wen, J.: Adaptive learning in tracking control based on the dual critic network design. IEEE Trans. Neural Netw. Learn. Syst. 24(6), 913–928 (2013)

    Article  Google Scholar 

  43. Ni, Z., He, H., Zhao, D., Xu, X., Prokhorov, D.V.: GrDHP: a general utility function representation for dual heuristic dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 26(3), 614–627 (2015)

    Article  MathSciNet  Google Scholar 

  44. Ni, Z., He, H., Zhong, X., Prokhorov, D.V.: Model-free dual heuristic dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 26(8), 1834–1839 (2015)

    Article  MathSciNet  Google Scholar 

  45. Wang, B., Zhao, D., Alippi, C., Liu, D.: Dual heuristic dynamic programming for nonlinear discrete-time uncertain systems with state delay. Neurocomputing 134, 222–229 (2014)

    Article  Google Scholar 

  46. Lin, W., Yang, P.: Adaptive critic motion control design of autonomous wheeled mobile robot by dual heuristic programming. Automatica 44(11), 2716–2723 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  47. Lian, C., Xu, X., Chen, H., He, H.: Near-optimal tracking control of mobile robots via receding-horizon dual heuristic programming. IEEE Trans. Cybern. 46(11), 2484–2496 (2016)

    Article  Google Scholar 

  48. Fairbank, M., Alonso, E., Prokhorov, D.: An equivalence between adaptive dynamic programming with a critic and backpropagation through time. IEEE Trans. Neural Netw. Learn. Syst. 24(12), 2088–2100 (2013)

    Article  Google Scholar 

  49. Zhong, X., Ni, Z., He, H.: Gr-GDHP: a new architecture for globalized dual heuristic dynamic programming. IEEE Trans. Cybern. 47(10), 3318–3330 (2017)

    Article  Google Scholar 

  50. Yen, G.G., DeLima, P.G.: Improving the performance of globalized dual heuristic programming for fault tolerant control through an online learning supervisor. IEEE Trans. Autom. Sci. Eng. 2(2), 121–131 (2005)

    Article  Google Scholar 

  51. Wang, D., Liu, D., Wei, Q., Zhao, D., Jin, N.: Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica 48(8), 1825–1832 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  52. Fairbank, M., Alonso, E., Prokhorov, D.: Simple and fast calculation of the second-order gradients for globalized dual heuristic dynamic programming in neural networks. IEEE Trans. Neural Netw. Learn. Syst. 23(10), 1671–1676 (2012)

    Article  Google Scholar 

  53. Liu, D., Wang, D., Zhao, D., Wei, Q., Jin, N.: Neural-network-based optimal control for a class of unknown discrete-time nonlinear systems using globalized dual heuristic programming. IEEE Trans. Autom. Sci. Eng. 9(3), 628–634 (2012)

    Article  Google Scholar 

  54. Zhao, D., Xia, Z., Wang, D.: Model-free optimal control for affine nonlinear systems with convergence analysis. IEEE Trans. Autom. Sci. Eng. 12(4), 1461–1468 (2015)

    Article  Google Scholar 

  55. Si, J., Wang, Y.: On-line learning control by association and reinforcement. IEEE Trans. Neural Netw. 12(2), 264–276 (2001)

    Article  Google Scholar 

  56. Wei, Q., Zhang, H., Dai, J.: Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions. Neurocomputing 72(7), 1839–1848 (2009)

    Article  Google Scholar 

  57. Dierks, T., Thumati, B.T., Jagannathan, S.: Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence. Neural Netw. 22(5), 851–860 (2009)

    Article  MATH  Google Scholar 

  58. Wei, Q., Liu, D.: An iterative \(\epsilon \)-optimal control scheme for a class of discrete-time nonlinear systems with unfixed initial state. Neural Netw. 32, 236–244 (2012)

    Article  MATH  Google Scholar 

  59. Dierks, T., Jagannathan, S.: Online optimal control of nonlinear discrete-time systems using approximate dynamic programming. J. Control Theory Appl. 9(3), 361–369 (2011)

    Article  MathSciNet  Google Scholar 

  60. Zhao, Q., Xu, H., Jagannathan, S.: Neural network-based finite-horizon optimal control of uncertain affine nonlinear discrete-time systems. IEEE Trans. Neural Netw. Learn. Syst. 26(3), 486–499 (2015)

    Article  MathSciNet  Google Scholar 

  61. Wang, D., He, H., Mu, C., Liu, D.: Intelligent critic control with disturbance attenuation for affine dynamics including an application to a microgrid system. IEEE Trans. Ind. Electron. 64(6), 4935–4944 (2017)

    Article  Google Scholar 

  62. Zhang, H., Cui, L., Zhang, X., Luo, Y.: Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method. IEEE Trans. Neural Netw. 22(12), 2226–2236 (2011)

    Article  Google Scholar 

  63. Wei, Q., Liu, D.: Adaptive dynamic programming for optimal tracking control of unknown nonlinear systems with application to coal gasification. IEEE Trans. Autom. Sci. Eng. 11(4), 1020–1036 (2014)

    Article  Google Scholar 

  64. Zhong, X., He, H.: An event-triggered adp control approach for continuous-time system with unknown internal states. IEEE Trans. Cybern. 47(3), 683–694 (2017)

    Article  Google Scholar 

  65. Yang, X., He, H.: Adaptive critic designs for event-triggered robust control of nonlinear systems with unknown dynamics. IEEE Trans. Cybern. to be published. https://doi.org/10.1109/TCYB.2018.2823199

    Article  Google Scholar 

  66. Wang, D., Mu, C., Liu, D., Ma, H.: On mixed data and event driven design for adaptive-critic-based nonlinear \(H_\infty \) control. IEEE Trans. Neural Netw. Learn. Syst. 29(4), 993–1005 (2018)

    Article  Google Scholar 

  67. Liu, W., Venayagamoorthy, G.K., Wunsch II, D.C.: A heuristic-dynamic-programming-based power system stabilizer for a turbogenerator in a single-machine power system. IEEE Trans. Ind. Appl. 41(5), 1377–1385 (2005)

    Article  Google Scholar 

  68. Jiang, Y., Jiang, Z.: Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Trans. Autom. Control 60(11), 2917–2929 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  69. Mu, C., Ni, Z., Sun, C., He, H.: Air-breathing hypersonic vehicle tracking control based on adaptive dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 584–598 (2017)

    Article  MathSciNet  Google Scholar 

  70. Wen, Y., Si, J., Gao, X., Huang, S., Huang, H.: A new powered lower limb prosthesis control framework based on adaptive dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 28(9), 2215–2220 (2017)

    Google Scholar 

  71. Xu, X., Lian, C., Zuo, L., He, H.: Kernel-based approximate dynamic programming for real-time online learning control: an experimental study. IEEE Trans. Control Syst. Technol. 22(1), 146–156 (2014)

    Article  Google Scholar 

  72. Venayagamoorthy, G.K., Harley, R.G., Wunsch, D.C.: Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogenerator. IEEE Trans. Neural Netw. 13(3), 764–773 (2002)

    Article  Google Scholar 

  73. Beard, R.W., Saridis, G.N., Wen, J.T.: Approximate solutions to the time-invariant Hamilton-Jacobi-Bellman equation. J. Opt. Theory Appl. 96(3), 589–626 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  74. Chen, Y., Edgar, T., Manousiouthakis, V.: On infinite-time nonlinear quadratic optimal control. Syst. Control Lett. 51(3), 259–268 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  75. Song, C., Ye, J., Liu, D., Kang, Q.: Generalized receding horizon control of fuzzy systems based on numerical optimization algorithm. IEEE Trans. Fuzzy Syst. 17(6), 1336–1352 (2009)

    Article  Google Scholar 

  76. Fakharian, A., Beheshti, M.T.H., Davari, A.: Solving the Hamilton-Jacobi-Bellman equation using Adomian decomposition method. Int. J. Comp. Math. 87(12), 2769–2785 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  77. Nik, H.S., Effati, S., Shirazian, M.: An approximate-analytical solution for the Hamilton-Jacobi-Bellman equation via homotopy perturbation method. App. Math. Model. 36(11), 5614–5623 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  78. Ganjefar, S., Rezaei, S.: Modified homotopy perturbation method for optimal control problems using the Padé approximant. App. Math. Model. 40(15), 7062–7081 (2016)

    Article  Google Scholar 

  79. Zhu, J.: A feedback optimal control by Hamilton-Jacobi-Bellman equation. Eur. J. Control 37, 70–74 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  80. Govindarajan, N., de Visser, C.C., Krishnakumar, K.: A sparse collocation method for solving time-dependent HJB equations using multivariate B-splines. Automatica 50(9), 2234–2244 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  81. Sun, B., Guo, B.: Convergence of an upwind finite-difference scheme for Hamilton-Jacobi-Bellman equation in optimal control. IEEE Trans. Autom. Control 60(11), 3012–3017 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  82. Chen, X., Chen, X.: An iterative method for optimal feedback control and generalized HJB equation. IEEE/CAA J. Autom. Sinica 5(5), 999–1006 (2018)

    Article  MathSciNet  Google Scholar 

  83. Michailidis, I., Baldi, S., Kosmatopoulos, E.B., Ioannou, P.A.: Adaptive optimal control for large-scale nonlinear systems. IEEE Trans. Autom. Control 62(11), 5567–5577 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  84. Kirches, C., Bock, H.G., Schlöer, J.P., Sager, S.: Block-structured quadratic programming for the direct multiple shooting method for optimal control. Opt. Methods Soft. 26(2), 239–257 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  85. Schäfer, A., Kühl, P., Diehl, M., Schlöder, J., Bock, H.G.: Fast reduced multiple shooting methods for nonlinear model predictive control. Chem. Eng. Process. 46, 1200–1214 (2007)

    Article  Google Scholar 

  86. Gerdts, M.: Direct shooting method for the numerical solution of higher-index DAE optimal control problems. J. Opt. Theory Appl. 117(2), 267–294 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  87. Hannemann-Tamás, R., Marquardt, W.: How to verify optimal controls computed by direct shooting methods? - A tutorial. J. Process Control 22, 494–507 (2012)

    Article  Google Scholar 

  88. Garg, D., Hager, W.W., Rao, A.V.: Pseudospectral methods for solving infinite-horizon optimal control problems. Automatica 47(4), 829–837 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  89. Wang, X., Peng, H., Zhang, S., Chen, B., Zhong, W.: A symplectic pseudospectral method for nonlinear optimal control problems with inequality constraints. ISA Trans. 68, 335–352 (2017)

    Article  Google Scholar 

  90. Garg, D., Patterson, M., Hager, W.W., Rao, A.V., Benson, D.A., Huntington, G.T.: A unified framework for the numerical solution of optimal control problems using pseudospectral methods. Automatica 46(11), 1843–1851 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  91. Darby, C.L., Hager, W.W., Rao, A.V.: An hp-adaptive pseudospectral method for solving optimal control problems. Optim. Control Appl. Meth. 32(4), 476–502 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  92. Ruths, J., Zlotnik, A., Li, J.: Convergence of a pseudospectral method for optimal control of complex dynamical systems. In: Proceedings of the CDC-ECC, pp. 5553–5558. Orlando (2011)

    Google Scholar 

  93. Xu, S., Li, S.E., Deng, K., Li, S., Cheng, B.: A unified pseudospectral computational framework for optimal control of road vehicles. IEEE/ASME Trans. Mechatron. 20(4), 1499–1510 (2015)

    Article  Google Scholar 

  94. Fahroo, F., Ross, I.M.: Advances in pseudospectral methods for optimal control. Proceeding of the AIAA Guidance. Navigation, and Control Conference and Exhibition, pp. 18–21. Honolulu, Hawaii (2008)

    Google Scholar 

  95. Peng, H., Wang, X., Li, M., Chen, B.: An hp symplectic pseudospectral method for nonlinear optimal control. Commun. Nonlinear Sci. Numer. Simulat. 42, 623–644 (2017)

    Article  MathSciNet  Google Scholar 

  96. Tabrizidooz, H.R., Marzban, H.R., Pourbabaee, M., Hedayati, M.: A composite pseudospectral method for optimal controlproblems with piecewise smooth solutions. J. Frankl. Inst. 354(5), 2393–2414 (2017)

    Article  MATH  Google Scholar 

  97. Ge, X., Yi, Z., Chen, L.: Optimal control of attitude for coupled-rigid-body spacecraft via Chebyshev-Gauss pseudospectral method. Appl. Math. Mech. -Eng. Ed. 38(9), 1257–1272 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  98. Genest, R., Ringwood, J.V.: Receding horizon pseudospectral control for energy maximization with application to wave energy devices. IEEE Trans. Control Syst. Technol. 25(1), 29–38 (2017)

    Article  Google Scholar 

  99. Merigaud, A., Ringwood, J.V.: Improving the computational performance of nonlinear pseudospectral control of wave energy converters. IEEE Trans. Sustain. Energy 9(3), 1419–1426 (2018)

    Article  Google Scholar 

  100. Mayne, D.Q., Rawlings, J.B., Rao, C.V., Scokaert, P.O.M.: Constrained model predictive control: stability and optimality. Automatica 36(6), 789–814 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  101. Gros, S.: A distributed algorithm for NMPC-based wind farm control. In: Proceedings of the CDS, pp. 4844–4849. Los Angeles, California (2014)

    Google Scholar 

  102. Cannon, M.: Efficient nonlinear model predictive control algorithms. Annu. Rev. Control 28(2), 229–237 (2004)

    Article  Google Scholar 

  103. Ohtsuka, T.: A continuation/GMRES method for fast computation of nonlinear receding horizon control. Automatica 40(4), 563–574 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  104. Wan, Z., Kothare, M.V.: Efficient scheduled stabilizing output feedback model predictive control for constrained nonlinear systems. IEEE Trans. Autom. Control 49(7), 1172–1177 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  105. Xu, F., Chen, H., Gong, X., Mei, Q.: Fast nonlinear model predictive control on fpga using particle swarm optimization. IEEE Trans. Ind. Electron. 63(1), 310–321 (2016)

    Article  Google Scholar 

  106. Mozaffari, A., Azad, N.L., Hedrick, J.K.: A nonlinear model predictive controller with multiagent online optimizer for automotive cold-start hydrocarbon emission reduction. IEEE Trans. Veh. Technol. 65(6), 4548–4563 (2016)

    Article  Google Scholar 

  107. Li, Z., Deng, J., Lu, R., Xu, Y., Bai, J., Su, C.: Trajectory-tracking control of mobile robot systems incorporating neural-dynamic optimized model predictive approach. IEEE Trans. Syst., Man, Cybern., Syst. 46(6), 740–749 (2016)

    Article  Google Scholar 

  108. Pan, Y., Wang, J.: Model predictive control of unknown nonlinear dynamical systems based on recurrent neural networks. IEEE Trans. Ind. Electron. 59(8), 3089–3101 (2012)

    Article  MathSciNet  Google Scholar 

  109. Yan, Z., Wang, J.: Model predictive control of nonlinear systems with unmodeled dynamics based on feedforward and recurrent neural networks. IEEE Trans. Ind. Inform. 8(4), 746–756 (2012)

    Article  Google Scholar 

  110. Yan, Z., Wang, J.: Robust model predictive control of nonlinear systems with unmodeled dynamics and bounded uncertainties based on neural networks. IEEE Trans. Neural Netw. Learn. Syst. 25(3), 457–469 (2014)

    Article  MathSciNet  Google Scholar 

  111. Yan, Z., Wang, J.: Nonlinear model predictive control based on collective neurodynamic optimization. IEEE Trans. Neural Netw. Learn. Syst. 26(4), 840–850 (2015)

    Article  MathSciNet  Google Scholar 

  112. Cheng, L., Liu, W., Hou, Z., Yu, J., Tan, M.: Neural-network-based nonlinear model predictive control for piezoelectric actuators. IEEE Trans. Ind. Electron. 62(12), 7717–7727 (2015)

    Article  Google Scholar 

  113. Bayer, F., Burger, M., Allgöwer, F.: Discrete-time incremental ISS: a framework for robust NMPC. In: Proceedings of the ECC, Zürich, pp. 2068–2073. Switzerland (2013)

    Google Scholar 

  114. Garcia, G.A., Keshmiri, S.S., Stastny, T.: Robust and adaptive nonlinear model predictive controller for unsteady and highly nonlinear unmanned aircraft. IEEE Trans. Control Syst. Technol. 23(4), 1620–1627 (2015)

    Article  Google Scholar 

  115. Pin, G., Raimondo, D.M., Magni, L., Parisini, T.: Robust model predictive control of nonlinear systems with bounded and state-dependent uncertainties. IEEE Trans. Autom. Control 54(7), 1681–1687 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  116. Li, H., Shi, Y.: Robust distributed model predictive control of constrained continuous-time nonlinear systems: a robustness constraint approach. IEEE Trans. Autom. Control 59(6), 1673–1678 (2014)

    Article  Google Scholar 

  117. Liu, C., Li, H., Gao, J., Xu, D.: Robust self-triggered min-max model predictive control for discrete-time nonlinear systems. Automatica 89, 333–339 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  118. Rubagotti, M., Raimondo, D.M., Ferrara, A., Magni, L.: Robust model predictive control with integral sliding mode in continuous-time sampled-data nonlinear systems. IEEE Trans. Autom. Control 56(3), 556–570 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  119. Kang, Y., Hedrick, J.K.: Linear tracking for a fixed-wing UAV using nonlinear model predictive control. IEEE Trans. Control Syst. Technol. 17(5), 1202–1210 (2009)

    Article  Google Scholar 

  120. Kim, S., Oh, H., Suk, J., Tsourdos, A.: Coordinated trajectory planning for efficient communication relay using multiple UAVs. Control Eng. Prac. 29, 42–49 (2014)

    Article  Google Scholar 

  121. Hosen, M.A., Hussain, M.A., Mjalli, F.S.: Control of polystyrene batch reactors using neural network based model predictive control (NNMPC): an experimental investigation. Control Eng. Prac. 19, 454–467 (2011)

    Article  Google Scholar 

  122. Vukov, M., Loock, W.V., Houska, B., Ferreau, H.J., Swevers, J., Diehl, M.: Experimental validation of nonlinear MPC on an overhead crane using automatic code generation. In: Proceedings of the ACC, pp. 6264–6269. Fairmont Queen Elizabeth, Montréal, Canada (2012)

    Google Scholar 

  123. Mills, A., Wills, A., Ninness, B.: Nonlinear model predictive control of an inverted pendulum. In: Proceedings of the ACC, pp. 2335–2340. Hyatt Regency Riverfront, St. Louis (2009)

    Google Scholar 

  124. Zhu, Q., Onori, S., Prucka, R.: An economic nonlinear model predictive control strategy for SI engines: model-based design and real-time experimental validation. IEEE Trans. Control Syst. Technol., to be published, https://doi.org/10.1109/TCST.2017.2769660

    Article  Google Scholar 

  125. Xie, Y., Ghaemi, R., Sun, J., Freudenberg, J.S.: Model predictive control for a full bridge DC/DC converter. IEEE Trans. Control Syst. Technol. 20(1), 164–172 (2012)

    Article  Google Scholar 

  126. Kayacan, E., Saeys, W., Ramon, H., Belta, C., Peschel, J.: Experimental validation of linear and nonlinear MPC on an articulated unmanned ground vehicle. IEEE/ASME Trans. Mechatron. to be published. https://doi.org/10.1109/TMECH.2018.2854877

    Article  Google Scholar 

  127. Chen, W.: Closed-form nonlinear MPC for multivariable nonlinear systems with different relative degree. In: Proceedings of the ACC, pp. 4887–4892. Denver (2003)

    Google Scholar 

  128. Merabet, A., Arioui, H., Ouhrouche, M.: Cascaded predictive controller design for speed control and load torque rejection of induction motor. In: Proceedings of the ACC, pp. 1139–1144. Seattle (2008)

    Google Scholar 

  129. Wang, X., Xie, Y., Shuai, D.: Three-phase four-leg active power filter based on nonlinear optimal predictive control. In: Proceedings of the CCC, pp. 217–222. Kunming, Yunnan, China (2008)

    Google Scholar 

  130. Dong, Z., Zheng, G., Liu, G.: Networked nonlinear model predictive control of the ball and beam system. In: Proceedings of the CCC, pp. 469–473. Kunming, Yunnan, China (2008)

    Google Scholar 

  131. Errouissi, R., Ouhrouche, M.: Nonlinear predictive controller for a permanent magnet synchronous motor drive. Math. Comput. Simul. 81, 394–406 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  132. Errouissi, R., Ouhrouche, M., Chen, W., Trzynadlowski, A.M.: Robust nonlinear predictive controller for permanent-magnet synchronous motors with an optimized cost function. IEEE Trans. Ind. Electron. 59(7), 2849–2858 (2012)

    Article  Google Scholar 

  133. Yang, J., Zhao, Z., Li, S., Zheng, W.X.: Composite predictive flight control for airbreathing hypersonic vehicles. Int. J. Control 87(9), 1970–1984 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  134. Liu, C., Chen, W., Andrews, J.: Tracking control of small-scale helicopters using explicit nonlinear MPC augmented with disturbance observers. Control Eng. Prac. 20(3), 258–268 (2012)

    Article  Google Scholar 

  135. Yang, J., Zheng, W.X.: Offset-free nonlinear MPC for mismatched disturbance attenuation with application to a static var compensator. IEEE Trans. Circuits Syst. II, Exp. Briefs 61(1), 49–53 (2014)

    Article  Google Scholar 

  136. Ouari, K., Rekioua, T., Ouhrouche, M.: Real time simulation of nonlinear generalized predictive control for wind energy conversion system with nonlinear observer. ISA Trans. 53(1), 76–84 (2014)

    Article  Google Scholar 

  137. Errouissi, R., Muyeen, S.M., Al-Durra, A., Leng, S.: Experimental validation of a robust continuous nonlinear model predictive control based grid-interlinked photovoltaic inverter. IEEE Trans. Ind. Electron. 63(7), 4495–4505 (2016)

    Article  Google Scholar 

  138. Errouissi, R., Al-Durra, A., Muyeen, S.M.: Design and implementation of a nonlinear PI predictive controller for a grid-tied photovoltaic inverter. IEEE Trans. Ind. Electron. 64(2), 1241–1250 (2017)

    Article  Google Scholar 

  139. Errouissi, R., Ouhrouche, M., Chen, W., Trzynadlowski, A.M.: Robust cascaded nonlinear predictive control of a permanent magnet synchronous motor with antiwindup compensator. IEEE Trans. Ind. Electron. 59(8), 3078–3088 (2012)

    Article  Google Scholar 

  140. Zhang, Y., Li, S., Liu, X.: Adaptive near-optimal control of uncertain systems with application to underactuated surface vessels. IEEE Trans. Control Syst. Technol. 26(4), 1204–1218 (2018)

    Article  Google Scholar 

  141. Zhang, Y., Li, S., Luo, X., Shang, M.: A dynamic neural controller for adaptive optimal control of permanent magnet DC motors. In: Proceedings of the IJCNN, pp. 839–844. Anchorage, USA (2017)

    Google Scholar 

  142. Zhang, Y., Li, S.: Predictive suboptimal consensus of multiagent systems with nonlinear dynamics. IEEE Trans. Syst., Man, Cybern., Syst. 47(7), 1701–1711 (2017)

    Article  MathSciNet  Google Scholar 

  143. Zhang, Y., Li, S.: Adaptive near-optimal consensus of high-order nonlinear multi-agent systems with heterogeneity. Automatica 85, 426–432 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  144. Movric, K.H., Lewis, F.L.: Cooperative optimal control for multi-agent systems on directed graph topologies. IEEE Trans. Autom. Control 59(3), 769–774 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  145. Liu, Y., Gao, Y., Tong, S., Li, Y.: Fuzzy approximation-based adaptive backstepping optimal control for a class of nonlinear discrete-time systems with dead-zone. IEEE Trans. Fuzzy Syst. 24(1), 16–28 (2016)

    Article  Google Scholar 

  146. Chen, S., Wang, J., Yao, M., Kim, Y.: Improved optimal sliding mode control for a non-linear vehicle active suspension system. J. Sound Vib. 395, 1–25 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Cyber Security, Jinan University, Guangzhou, China

    Yinyan Zhang

  2. School of Information Science and Engineering, Lanzhou University, Lanzhou, China

    Shuai Li

  3. Guangdong Institute of Intelligent Manufacturing, Guangdong Academy of Science, Guangzhou, China

    Xuefeng Zhou

Authors
  1. Yinyan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuefeng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuai Li .

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zhang, Y., Li, S., Zhou, X. (2020). A Survey of Near-Optimal Control of Nonlinear Systems. In: Deep Reinforcement Learning with Guaranteed Performance. Studies in Systems, Decision and Control, vol 265. Springer, Cham. https://doi.org/10.1007/978-3-030-33384-3_1

Download citation

Publish with us

Policies and ethics

Search

Navigation