Abstract
Using the neural-network-based iterative adaptive dynamic programming (ADP) algorithm, an optimal control scheme for a class of unknown discrete-time nonlinear systems with discount factor in the cost function is proposed in this paper. The optimal controller is designed with convergence analysis in terms of cost function and control law. In order to implement the algorithm via globalized dual heuristic programming (GDHP) technique, a neural network is constructed first to identify the unknown nonlinear system, and then two other neural networks are used to approximate the cost function and the control law, respectively. An example is provided to verify the effectiveness of the present approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Jagannathan, S.: Neural Network Control of Nonlinear Discrete-time Systems. CRC Press, Boca Raton (2006)
Yu, W.: Recent Advances in Intelligent Control Systems. Springer, London (2009)
Werbos, P.J.: Approximate Dynamic Programming for Real-time Control and Neural Modeling. In: White, D.A., Sofge, D.A. (eds.) Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approach, ch. 13. Van Nostrand Reinhold, New York (1992)
Werbos, P.J.: Advanced Forecasting Methods for Global Crisis Warning and Models of Intelligence. General Systems Yearbook 22, 25–38 (1977)
Canelon, J.I., Shieh, L.S., Karayiannis, N.B.: A New Approach for Neural Control of Nonlinear Discrete Dynamic Systems. Information Sciences 174(3-4), 177–196 (2005)
Prokhorov, D.V., Wunsch, D.C.: Adaptive Critic Designs. IEEE Transactions on Neural Networks 8(5), 997–1007 (1997)
Si, J., Wang, Y.T.: On-line Learning Control by Association and Reinforcement. IEEE Transactions on Neural Networks 12(2), 264–276 (2001)
Murray, J.J., Cox, C.J., Lendaris, G.G., Saeks, R.: Adaptive Dynamic Programming. IEEE Transactions on Systems, Man, Cybernatics–Part C: Applications and Reviews 32(2), 140–153 (2002)
Wang, F.Y., Zhang, H., Liu, D.: Adaptive Dynamic Programming: an Introduction. IEEE Computational Intelligence Magazine 4(2), 39–47 (2009)
Lewis, F.L., Vrabie, D.: Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control. IEEE Circuits and Systems Magazine 9(3), 32–50 (2009)
Liu, D., Xiong, X., Zhang, Y.: Action-dependent Adaptive Critic Designs. In: Proceedings of the International Joint Conference on Neural Networks, pp. 990–995 (2001)
Liu, D., Zhang, Y., Zhang, H.: A Self-learning Call Admission Control Scheme for CDMA Cellular Networks. IEEE Transactions on Neural Networks 16(5), 1219–1228 (2005)
Liu, D., Jin, N.: ε-adaptive Dynamic Programming for Discrete-time Systems. In: Proceedings of the International Joint Conference on Neural Networks, pp. 1417–1424 (2008)
Yen, G.G., Delima, P.G.: Improving the Performance of Globalized Dual Heuristic Programming for Fault Tolerant Control Through an Online Learning Supervisor. IEEE Transactions an Automation Science and Engineering 2(2), 121–131 (2005)
Abu-Khalaf, M., Lewis, F.L.: Nearly Optimal Control Laws for Nonlinear Systems with Saturating Actuators Using a Neural Network HJB Approach. Automatica 41(5), 779–791 (2005)
Al-Tamimi, A., Lewis, F.L., Abu-Khalaf, M.: Discrete-time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof. IEEE Transactions on Systems, Man, Cybernatics–Part B: Cybernatics 38(4), 943–949 (2008)
Luo, Y., Zhang, H.: Approximate Optimal Control for a Class of Nonlinear Discrete-time Systems with Saturating Actuators. Progress in Natural Science 18(8), 1023–1029 (2008)
Wei, Q., Zhang, H., Liu, D., Zhao, Y.: An Optimal Control Scheme for a Class of Discrete-time Nonlinear Systems with Time Delays Using Adaptive Dynamic Programming. Acta Automatica Sinica 36(1), 121–129 (2010)
Dierks, T., Thumati, B.T., Jagannathan, S.: Optimal Control of Unknown Affine Nonlinear Discrete-time Systems Using Offline-trained Neural Networks with Proof of Convergence. Neural Networks 22(5-6), 851–860 (2009)
Vrabie, D., Pastravanu, O., Abu-Khalaf, M., Lewis, F.L.: Adaptive Optimal Control for Continuous-time Linear Systems Based on Policy Iteration. Automatica 45(2), 477–484 (2009)
Sun, Z., Chen, X., He, Z.: Adaptive Critic Designs for Energy Minimization of Portable Video Communication Devices. IEEE Transactions on Circuits and Systems for Video Technology 20(1), 27–37 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, D., Liu, D. (2011). Optimal Control for a Class of Unknown Nonlinear Systems via the Iterative GDHP Algorithm. In: Liu, D., Zhang, H., Polycarpou, M., Alippi, C., He, H. (eds) Advances in Neural Networks – ISNN 2011. ISNN 2011. Lecture Notes in Computer Science, vol 6676. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21090-7_72
Download citation
DOI: https://doi.org/10.1007/978-3-642-21090-7_72
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21089-1
Online ISBN: 978-3-642-21090-7
eBook Packages: Computer ScienceComputer Science (R0)