Abstract
In this chapter, a time-scale expansion-based scheme is presented for approximately solving the optimal control problem of continuous-time underactuated nonlinear systems subject to input constraints and system dynamics. By time-scale Taylor approximation of the original performance index, the optimal control problem is relaxed into an approximated optimal control problem. Based on the system dynamics, the problem is further reformulated as a quadratic program, which is solved by a projection neural network. Theoretical analysis on the closed-loop system synthesized by the controlled system and the projection neural network is conducted, which reveals that, under certain conditions, the closed-loop system possesses exponential stability and the original performance index converges to zero as time tends to infinity. In addition, two illustrative examples, which are based on a flexible joint manipulator and an underactuacted ship, are provided to validate the theoretical results and demonstrate the efficacy and superiority of the presented control scheme.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Liu, Y.-J., Tong, S.: Optimal control-based adaptive NN design for a class of nonlinear discrete-time block-triangular systems. IEEE Trans. Cybern. 46(11), 2670–2680 (2016)
Liu, Y.-J., Gao, Y., Tong, S., Li, Y.: Fuzzy approximation-based adaptive backstepping optimal control for a class of nonlinear discrete-time systems with dead-zone. IEEE Trans. Fuzzy Syst. 24(1), 16–28 (2016)
Lewis, F.L., Syrmos, V.L.: Optimal Control. Wiley, New York (1995)
Chen, Z., Jagannathan, S.: Generalized Hamilton–Jacobi–Bellman formulation-based neural network control of affine nonlinear discrete-time systems. IEEE Trans. Neural Netw. 19(1), 90–106 (2008)
Mayne, D.Q.: Model predictive control: recent developments and future promise. Automatica 50(12), 2967–2986 (2014)
Chen, W.H., Ballance, D.J., Gawthrop, P.J.: Optimal control of nonlinear systems: a predictive control approach. Automatica 39(4), 633–641 (2003)
Hedjar, R., Toumi, R., Boucher, P., Dumur, D.: Finite horizon nonlinear predictive control by the Taylor approximation application to robot tracking trajectory. Int. J. Appl. Math. Comput. Sci. 15(4), 527–540 (2005)
Mayne, D.Q., Michalska, H.: Receding horizon control of nonlinear systems. IEEE Trans. Autom. Contro 35(7), 814–824 (1990)
Han, D., Shi, L.: Guaranteed cost control of affine nonlinear systems via partition of unity method. Automatica 49(2), 660–666 (2013)
Lian, C., Xu, X., Chen, H., He, H.: Near-optimal tracking control of mobile robots via receding-horizon dual heuristic programming. IEEE Trans. Cybern. 46(11), 2484–2496 (2016)
Li, H., Shi, Y., Yan, W.: On neighbor information utilization in distributed receding horizon control for consensus-seeking. IEEE Trans. Cybern. 46(9), 2019–2027 (2016)
Yang, Q., Jagannathan, S.: Reinforcement learning controller design for affine nonlinear discrete-time systems using online approximators. IEEE Trans. Syst. Man Cybern. B Cybern. 42(2), 377–390 (2012)
Du, H., Zhang, N.: Fuzzy control for nonlinear uncertain electrohydraulic active suspensions with input constraint. IEEE Trans. Fuzzy Syst. 17(2), 343–356 (2009)
Liu, Y.-J., Gao, Y., Tong, S., Philip Chen, C.L.: A unified approach to adaptive neural control for nonlinear discrete-time systems with nonlinear dead-zone input. IEEE Trans. Neural Netw. Learn. Syst. 27(1), 39–150 (2016)
Ma, Y., Borrelli, F., Hencey, B., Coffey, B., Bengea, S., Haves, P.: Model predictive control for the operation of building cooling systems. IEEE Trans. Control Syst. Technol. 20(3), 796–803 (2012)
Han, H.G., Wu, X.L., Qiao, J.F.: Real-time model predictive control using a self-organizing neural network. IEEE Trans. Neural Netw. Learn. Syst. 24(9), 1425–1436 (2013)
Lu, C.H.: Design and application of stable predictive controller using recurrent wavelet neural networks. IEEE Trans. Ind. Electron. 56(9), 3733–3742 (2009)
Peng, H., Nakano, K., Shioya, H.: Nonlinear predictive control using neural nets-based local linearization ARX model-stability and industrial application. IEEE Trans. Control Syst. Techno. 15(1), 130–143 (2017)
Li, Z., Xiao, H., Yang, C., Zhao, Y.: Model predictive control of nonholonomic chained systems using general projection neural networks optimization. IEEE Trans. Syst. Man Cybern. Syst. 45(10), 1313–1321 (2015)
Lu, P.: Constrained tracking control of nonlinear systems. Automatica 27(5), 305–314 (1996)
Li, X., Rakkiyappan, R., Velmurugan, G.: Dissipativity analysis of memristor-based complex-valued neural networks with time-varying delays. Inform. Sci. 294, 645–665 (2015)
Zhang, Y., Ge, S.S., Lee, T.H.: A unified quadratic-programming-based dynamical system approach to joint torque optimization of physically constrained redundant manipulators. IEEE Trans. Syst. Man Cybern. B 34(5), 2126–2132 (2004)
Li, X., Regan, D.O., Akca, H.: Global exponential stabilization of impulsive neural networks with unbounded continuously distributed delays. IMA J. Appl. Math. 80(1), 85–99 (2015)
Jin, L., Zhang, Y., Li, S., Zhang, Y.: Modified ZNN for time-varying quadratic programming with inherent tolerance to noises and its application to kinematic redundancy resolution of robot manipulators. IEEE Trans. Ind. Electron. 63(11), 6978–6988 (2016)
Xiao, L.: A nonlinearly-activated neurodynamic model and its finite-time solution to equality-constrained quadratic optimization with nonstationary coefficients. Appl. Soft Comput. 40, 252–259 (2016)
Qin, S., Xue, X.: A two-layer recurrent neural network for nonsmooth convex optimization problems. IEEE Trans. Neural Netw. Learn. Syst. 26(6), 1149–1160 (2015)
Zhang, Z., Zhang, Y.: Variable joint-velocity limits of redundant robot manipulators handled by quadratic programming. IEEE/ASME Trans. Mechatronics 18(2), 674–686 (2013)
Li, X., Song, S.: Impulsive control for existence, uniqueness and global stability of periodic solutions of recurrent neural networks with discrete and continuously distributed delays. IEEE Trans. Neural Netw. 24, 868–877 (2013)
Li, S., Chen, S., Liu, B., Li, Y., Liang, Y.: Decentralized kinematic control of a class of collaborative redundant manipulators via recurrent neural networks. Neurocomputing 91, 1–10 (2012)
Li, S., Cui, H., Li, Y., Liu, B., Lou, Y.: Decentralized control of collaborative redundant manipulators with partial command coverage via locally connected recurrent neural networks. Neural Comput. Appl. 23(3), 1051–1060 (2013)
Li, S., He, J., Li, Y., Rafique, M.U.: Distributed recurrent neural networks for cooperative control of manipulators: A game-theoretic perspective. IEEE Trans. Neural Netw. Learn. Syst. 28(2), 415–426 (2017)
Jin, L., Li, S., La, H.M., Luo, X.: Manipulability optimization of redundant manipulators using dynamic neural networks. IEEE Trans. Ind. Electron. 64(6), 4710–4720 (2017)
Li, Y., Li, S., Hannaford, B.: A novel recurrent neural network for improving redundant manipulator motion planning completeness. ICRA, 2956–2961 (2018)
Zhang, Y., Li, S.: A neural controller for image-based visual servoing of manipulators with physical constraints. IEEE Trans. Neural Netw. Learning Syst. 29(11), 5419–5429 (2018)
Li, S., Zhou, M., Luo, X.: Modified primal-dual neural networks for motion control of redundant manipulators with dynamic rejection of harmonic noises. IEEE Trans. Neural Netw. Learning Syst. 29(10), 4791–4801 (2018)
Li, S., Wang, H., Rafique, M.U.: A novel recurrent neural network for manipulator control with improved noise tolerance. IEEE Trans. Neural Netw. Learning Syst. 29(5), 1908–1918 (2018)
Jin, L., Li, S., Luo, X., Li, Y., Qin, B.: Neural dynamics for cooperative control of redundant robot manipulators. IEEE Trans. Ind. Inform. 14(9), 3812–3821 (2018)
Li, J., Zhang, Y., Li, S., Mao, M.: New discretization-formula-based zeroing dynamics for real-time tracking control of serial and parallel manipulators. IEEE Trans. Ind. Inform. 14(8), 3416–3425 (2018)
Chen, D., Zhang, Y., Li, S.: Tracking control of robot manipulators with unknown models: A jacobian-matrix-adaption method. IEEE Trans. Ind. Inform. 14(7), 3044–3053 (2018)
Zhang, Y., Li, S., Gui, J., Luo, X.: Velocity-level control with compliance to acceleration-level constraints: A novel scheme for manipulator redundancy resolution. IEEE Trans. Ind. Inform. 14(3), 921–930 (2018)
Xiao, L., Liao, B., Li, S., Zhang, Z., Ding, L., Jin, L.: Design and analysis of FTZNN applied to the real-time solution of a nonstationary Lyapunov equation and tracking control of a wheeled mobile manipulator. IEEE Trans. Ind. Inform. 14(1), 98–105 (2018)
Zhang, Y., Chen, S., Li, S., Zhang, Z.: Adaptive projection neural network for kinematic control of redundant manipulators with unknown physical parameters. IEEE Trans. Ind. Electron. 65(6), 4909–4920 (2018)
Zhang, Z., Lin, Y., Li, S., Li, Y., Yu, Z., Luo, Y.: Tricriteria optimization-coordination motion of dual-redundant-robot manipulators for complex path planning. IEEE Trans. Control Syst. Technol. 26(4), 1345–1357 (2018)
Jin, L., Li, S., Hu, B., Yi, C.: Dynamic neural networks aided distributed cooperative control of manipulators capable of different performance indices. Neurocomputing 291, 50–58 (2018)
Jin, L., Li, S., Yu, J., He, J.: Robot manipulator control using neural networks: a survey. Neurocomputing 285, 23–34 (2018)
Chen, D., Zhang, Y., Li, S.: Zeroing neural-dynamics approach and its robust and rapid solution for parallel robot manipulators against superposition of multiple disturbances. Neurocomputing 275, 845–858 (2018)
Li, S., Shao, Z., Guan, Y.: A dynamic neural network approach for efficient control of manipulators. IEEE Trans. Syst. Man Cybern. Syst. 49(5), 932–941 (2019)
Zhang, Y., Li, S., Zhou, X.: Recurrent-neural-network-based velocity-level redundancy resolution for manipulators subject to a joint acceleration limit. IEEE Trans. Ind. Electron. 66(5), 3573–3582 (2019)
Zhang, Z., Chen, S., Li, S.: Compatible convex-nonconvex constrained QP-based dual neural networks for motion planning of redundant robot manipulators. IEEE Trans. Control Syst. Technol. 27(3), 1250–1258 (2019)
Xu, Z., Li, S., Zhou, X., Yan, W., Cheng, T., Huang, D.: Dynamic neural networks based kinematic control for redundant manipulators with model uncertainties. Neurocomputing 329, 255–266 (2019)
Li, S., Zhang, Y., Jin, L.: Kinematic control of redundant manipulators using neural networks. IEEE Trans. Neural Netw. Learn. Syst. (In Press)
Zhang, Y., Wang, J., Xia, Y.: A dual neural network for redundancy resolution of kinematically redundant manipulators subject to joint limits and joint velocity limits. IEEE Trans. Neural Netw. 14(3), 658–667 (2003)
Zhang, Z., Li, Z., Zhang, Y., Luo, Y., Li, Y.: Neural-dynamic-method-based dual-arm CMG scheme with time-varying constraints applied to humanoid robots. IEEE Trans. Neural Netw. Learn. Syst. 26(12), 3251–3262 (2015)
Zhang, Y., Li, W., Zhang, Z.: Physical-limits-constrained minimum velocity norm coordinating scheme for wheeled mobile redundant manipulators. Robotica 33(6), 1325–1350 (2015)
Mohammed, A.M., Li, S.: Dynamic neural networks for kinematic redundancy resolution of parallel stewart platforms. IEEE Trans. Cybern. 46(7), 1538–1550 (2016)
Jin, L., Zhang, Y.: G2-type SRMPC scheme for synchronous manipulation of two redundant robot Arms. IEEE Trans. Cybern. 45(2), 153–164 (2015)
Jin, L., Li, S., Xiao, L., Lu, R., Liao, B.: Cooperative motion generation in a distributed network of redundant robot manipulators with noises. IEEE Trans. Syst. Man Cybern. Syst. (In Press). https://doi.org/10.1109/TSMC.2017.2693400
Li, S., Shao, Z., Guan, Y.: A dynamic neural network approach for efficient control of manipulators. IEEE Trans. Syst. Man Cybern. Syst. (In Press). https://doi.org/10.1109/TSMC.2017.2690460
Jin, L., Li, S.: Distributed task allocation of multiple robots: a control perspective. IEEE Trans. Syst. Man Cybern. Syst. (In Press). https://doi.org/10.1109/TSMC.2016.2627579
Chen, C.L.P., Wen, G.-X., Liu, Y.-J., Wang, F.-Y.: Adaptive consensus control for a class of nonlinear multiagent time-delay systems using neural networks. IEEE Trans. Neural Netw. Learn. Syst. 25(6), 1217–1226 (2014)
Wang, H.Q., Liu, X.P., Liu, K.F.: Robust adaptive neural tracking control for a class of stochastic nonlinear interconnected systems. IEEE Trans. Neural Netw. Learn. Syst. 27(3), 510–523 (2016)
Chen, C.L.P., Liu, Y.-J., Wen, G.-X.: Fuzzy neural network-based adaptive control for a class of uncertain nonlinear stochastic systems. IEEE Trans. Cybern. 44(5), 583–593 (2014)
Liu, Y.-J., Tang, L., Tong, S., Chen, C.L.P.: Adaptive NN controller design for a class of nonlinear MIMO discrete-time systems. IEEE Trans. Neural Netw. Learn. Syst. 26(5), 1007–1018 (2015)
Liu, Y.-J., Tang, L., Tong, S., Chen, C.L.P., Li, D.-J.: Reinforcement learning design-based adaptive tracking control with less learning parameters for nonlinear discrete-time MIMO systems. IEEE Trans. Neural Netw. Learn. Syst. 26(1), 165–176 (2015)
Li, T., Duan, S., Liu, J., Wang, L., Huang, T.: A spintronic memristor-based neural network with radial basis function for robotic manipulator control implementation. IEEE Trans. Syst. Man Cybern. Syst. 46(4), 582–588 (2016)
Khalil, H.K.: Nonlinear Systems, 3rd edn. Prentice-Hall, New Jersey (2002)
Gao, Y., Wang, H., Liu, Y.-J.: Adaptive fuzzy control with minimal leaning parameters for electric induction motors. Neurocomputing 156, 143–150 (2015)
Wang, H., Chen, B., Liu, X., Liu, K., Lin, C.: Robust adaptive fuzzy tracking control for pure-feedback stochastic nonlinear systems with input constraints. IEEE Trans. Cybern. 43(6), 2093–2104 (2013)
Isidori, A.: Nonlinear Control Systems: An Introduction, 3rd edn. Springer, New York (1995)
Xia, Y.: Further results on global convergence and stability of globally projected dynamical systems. J. Optim. Theory Appl. 122(3), 627–649 (2004)
Wang, H., Shi, P., Li, H., Zhou, Q.: Adaptive neural tracking control for a class of nonlinear systems with dynamic uncertainties. IEEE Trans. Cybern. (In Press)
Chen, C.L.P., Wen, G.-X., Liu, Y.-J., Liu, Z.: Observer-based adaptive backstepping consensus tracking control for high-order nonlinear semi-strict-feedback multiagent systems. IEEE Trans. Cybern. 46(7), 1591–1601 (2016)
Liu, Y.-J., Tong, S.: Barrier Lyapunov functions-based adaptive control for a class of nonlinear pure-feedback systems with full state constraints. Automatica 64, 70–75 (2016)
He, W., Ge, W., Li, Y., Liu, Y.-J., Yang, C., Sun, C.: Model identification and control design for a humanoid robot. IEEE Trans. Syst. Man Cybern. Syst. 47(1), 45–57 (2017)
Wang, H., Liu, X., Liu, K.: Adaptive fuzzy tracking control for a class of pure-feedback stochastic nonlinear systems with non-lower triangular structure. Fuzzy Set Syst. 302, 101–120 (2016)
Chien, T., Chen, C., Huang, C.: Feedback linearization control and its application to mimo cancer immunotherapy. IEEE Trans. Control Syst. Technol. 18(4), 953–961 (2010)
Zhang, Y., Li, S.: Predictive suboptimal consensus of multiagent systems with nonlinear dynamics. IEEE Trans. Syst. Man Cybern. Syst. (In Press). https://doi.org/10.1109/TSMC.2017.2668440
Allen, L.J.S.: An Introduction to Mathematical Biology. Pearson Education, New Jersey (2007)
Groves, K., Serrani, A.: Modeling and nonlinear control of a single-link flexible joint manipulator (2004)
Jiang, Z.: Global tracking control of underactuated ships by Lyapunov’s direct method. Automatica 38(2), 301–309 (2002)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Zhang, Y., Li, S., Zhou, X. (2020). Near-Optimal Control with Input Saturation. In: Deep Reinforcement Learning with Guaranteed Performance. Studies in Systems, Decision and Control, vol 265. Springer, Cham. https://doi.org/10.1007/978-3-030-33384-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-33384-3_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33383-6
Online ISBN: 978-3-030-33384-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)