Abstract
There has been great interest recently in “universal model-free controllers” that do not need a mathematical model of the controlled plant but mimic the functions of biological processes to learn about the systems they are controlling online, so that performance improves automatically. Neural network (NN) control has had two major thrusts: approximate dynamic programming, which uses NN to approximately solve the optimal control problem, and NN in closed-loop feedback control.
Bibliography
Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5):779–791
Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern Part B 38(4):943–949
Alessandri A, Baglietto M, Parisini T, Zoppoli R (1999) A neural state estimator with bounded errors for nonlinear systems. IEEE Trans Autom Control 44(11):2028–2042
Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming, vol 5. Athena Scientific, Belmont
Ioannou PA, Fidan B (2006) Adaptive control tutorial. Advances in design and control. SIAM
Jiang Y, Jiang ZP (2017) Robust adaptive dynamic programming. Wiley
Kamalapurkar R, Walters PS, Rosenfeld JA, Dixon WE (2018) Reinforcement learning for optimal feedback control: a Lyapunov-based approach. Springer
Kiumarsi B, Vamvoudakis KG, Modares H, Lewis FL (2018) Optimal and autonomous control using reinforcement learning: a survey. IEEE Trans Neural Netw Learn Syst 29(6):2042–2062
Kosmatopoulos EB, Polycarpou MM, Christodoulou MA, Ioannou PA (1995) High-order neural network structures for identification of dynamical systems. IEEE Trans Neural Netw 6(2):422–431
Lewis FL, Liu D (2012) Reinforcement learning and approximate dynamic programming for feedback control. IEEE Press computational intelligence series
Lewis FL, Vamvoudakis KG (2011) Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data. IEEE Trans Syst Man Cybern Part B 41(1):14–25
Lewis FL, Jagannathan S, Yesildirek A (1999) Neural network control of robot manipulators and nonlinear systems. Taylor and Francis, London
Lewis FL, Campos J, Selmic R (2002) Neuro-fuzzy control of industrial systems with actuator nonlinearities. Society of Industrial and Applied Mathematics Press, Philadelphia
Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst Mag 32(6):76–105
Patre PM, MacKunis W, Kaiser K, Dixon WE (2008) Asymptotic tracking for uncertain dynamic systems via a multi layer neural network feedforward and RISE feedback control structure. IEEE Trans Autom Control 53(9):2180–2185
Slotine JJE, Li W (1987) On the adaptive control of robot manipulators. Int J Robot Res 6(3):49–59
Sutton RS, Barto AG (1998) Reinforcement learning an introduction. MIT Press, Cambridge, MA
Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888
Vamvoudakis KG, Lewis FL (2011a) Multi-player non zero sum games: online adaptive learning solution of coupled Hamilton-Jacobi equations. Automatica 47(8):1556–1569
Vamvoudakis KG, Lewis FL (2011b) Policy iteration algorithm for distributed networks and graphical games. In: Proceedings of 50th IEEE conference on decision and control, Orlando, pp 128–135
Vamvoudakis KG, Lewis FL (2012) Online solution of nonlinear two-player zero-sum games using synchronous policy iteration. Int J Robust Nonlinear Control 22(13):1460–1483
Vamvoudakis KG, Lewis FL, Hudas GR (2012) Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality. Automatica 48(8):1598–1611
Vamvoudakis KG, Vrabie D, Lewis FL (2014) Online adaptive algorithm for optimal control with integral reinforcement learning. Int J Robust Nonlinear Control 24(17):2686–2710
Vamvoudakis KG, Antsaklis PJ, Dixon WE, Hespanha JP, Lewis FL, Modares H, Kiumarsi B (2015) Autonomy and machine intelligence in complex systems: a tutorial. In: Proceedings of American control conference, Chicago, pp 5062–5079
Vamvoudakis KG, Modares H, Kiumarsi B, Lewis FL (2017) Game theory-based control system algorithms with real-time reinforcement learning: how to solve multiplayer games online. IEEE Control Syst Mag 37(1):33–52
Vamvoudakis KG, Lewis FL, Dixon WE (2019) Open-loop Stackelberg learning solution for hierarchical control problems. Int J Adapt Control Signal Process 33(2):285–299
Vrabie D, Pastravanu O, Lewis FL, Abu-Khalaf M (2009) Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica 45(2):477–484
Vrabie DL, Vamvoudakis KG, Lewis FL (2012) Optimal adaptive control and differential games by reinforcement learning principles. Control engineering series. IET Press
Werbos PJ (1989) Neural networks for control and system identification. In: Proceedings of IEEE conference decision and control, Orlando
Acknowledgements
This material is based upon the work supported by NSF CPS-1851588, ECCS-1839804, SATC-1801611, by Minerva Research Initiative N00014-18-1-2160, and by ONR N00014-17-1-2239, N00014-18-1-2221.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2020 Springer-Verlag London Ltd., part of Springer Nature
About this entry
Cite this entry
Lewis, F.L., Vamvoudakis, K.G. (2020). Neuro-Inspired Control. In: Baillieul, J., Samad, T. (eds) Encyclopedia of Systems and Control. Springer, London. https://doi.org/10.1007/978-1-4471-5102-9_224-3
Download citation
DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-3
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-4471-5102-9
Online ISBN: 978-1-4471-5102-9
eBook Packages: Springer Reference EngineeringReference Module Computer Science and Engineering
Publish with us
Chapter history
-
Latest
Neuro-Inspired Control- Published:
- 20 August 2020
DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-3
-
Neural Control and Approximate Dynamic Programming
- Published:
- 08 December 2014
DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-2
-
Original
Neural Control and Approximate Dynamic Programming- Published:
- 12 April 2014
DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-1