Abstract
There has been great interest recently in “universal model-free controllers” that do not need a mathematical model of the controlled plant, but mimic the functions of biological processes to learn about the systems they are controlling online, so that performance improves automatically. Neural network (NN) control has had two major thrusts: approximate dynamic programming, which uses NN to approximately solve the optimal control problem, and NN in closed-loop feedback control.
Neural Feedback Control
The objective is to design NN feedback controllers that cause a system to follow, or track, a prescribed trajectory or path. Consider the dynamics of an n-link robot manipulator
with \(q(t) \in \mathbb{R}^{n}\) the joint variable vector, M(q) an inertia matrix, V m a centripetal/coriolis matrix, G(q) a gravity vector, and \(F(\cdot )\)representing...
Keywords
- Adaptive Controller
- Bellman Equation
- Policy Iteration
- Neural Network Controller
- Approximate Dynamic Programming
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Bibliography
Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5):779–791
Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern Part B 38(4):943–949
Lewis FL, Liu D (2012) Reinforcement learning and approximate dynamic programming for feedback control. IEEE Press computational intelligence series. Wiley-Blackwell, Oxford
Lewis FL, Vamvoudakis KG (2011) Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data. IEEE Trans Syst Man Cybern Part B 41(1):14–25
Lewis FL, Jagannathan S, Yesildirek A (1999) Neural network control of robot manipulators and nonlinear systems. Taylor and Francis, London
Lewis FL, Campos J, Selmic R (2002) Neuro-fuzzy control of industrial systems with actuator nonlinearities. Society of Industrial and Applied Mathematics Press, Philadelphia
Lewis FL, Vrabie D, Syrmos VL (2012a) Optimal control. Wiley, New York
Lewis FL, Vrabie D, Vamvoudakis KG (2012b) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst Mag 32(6):76–105
Slotine JJE, Li W (1987) On the adaptive control of robot manipulators. Int J Robot Res 6(3):49–59
Sutton RS, Barto AG (1998) Reinforcement learning – an introduction. MIT, Cambridge
Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888
Vamvoudakis KG, Lewis FL (2011) Multi-player non zero sum games: online adaptive learning solution of coupled Hamilton-Jacobi equations. Automatica 47(8):1556–1569
Vamvoudakis KG, Lewis FL (2012) Online solution of nonlinear two-player zero-sum games using synchronous policy iteration. Int J Robust Nonlinear Control 22(13):1460–1483
Vamvoudakis KG, Lewis FL, Hudas GR (2012a) Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality. Automatica 48(8):1598–1611
Vamvoudakis KG, Lewis FL, Johnson M, Dixon WE (2012b) Online learning algorithm for Stackelberg games in problems with hierarchy. In: Proceedings of the 51st IEEE conference on decision and control, Maui pp 1883–1889
Vamvoudakis KG, Vrabie D, Lewis FL (2013) Online adaptive algorithm for optimal control with integral reinforcement learning. Int J Robust Nonlinear Control, Wiley. doi: 10.1002/rnc.3018
Vrabie D, Pastravanu O, Lewis FL, Abu-Khalaf M (2009) Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica 45(2):477–484
Vrabie D, Vamvoudakis KG, Lewis FL (2012) Optimal adaptive control and differential games by reinforcement learning principles. Control engineering series. IET Press, London
Werbos PJ (1989) Neural networks for control and system identification. In: Proceedings of the IEEE conference on decision and control, Tampa
Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA (eds) Handbook of intelligent control. Van Nostrand Reinhold, New York
Acknowledgements
This material is based upon the work supported by NSF. Grant Number: ECCS-1128050, ARO. Grant Number: W91NF-05-1-0314, AFOSR. Grant Number: FA9550-09-1-0278.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag London
About this entry
Cite this entry
Lewis, F.L., Vamvoudakis, K.G. (2014). Neural Control and Approximate Dynamic Programming. In: Baillieul, J., Samad, T. (eds) Encyclopedia of Systems and Control. Springer, London. https://doi.org/10.1007/978-1-4471-5102-9_224-2
Download citation
DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-2
Received:
Accepted:
Published:
Publisher Name: Springer, London
Online ISBN: 978-1-4471-5102-9
eBook Packages: Springer Reference EngineeringReference Module Computer Science and Engineering
Publish with us
Chapter history
-
Latest
Neuro-Inspired Control- Published:
- 20 August 2020
DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-3
-
Neural Control and Approximate Dynamic Programming
- Published:
- 08 December 2014
DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-2
-
Original
Neural Control and Approximate Dynamic Programming- Published:
- 12 April 2014
DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-1