Skip to main content
SpringerLink
Log in

Neuro-Inspired Control

  • Living reference work entry
  • Latest version View entry history
  • First Online:
Encyclopedia of Systems and Control

  • 317 Accesses

Abstract

There has been great interest recently in “universal model-free controllers” that do not need a mathematical model of the controlled plant but mimic the functions of biological processes to learn about the systems they are controlling online, so that performance improves automatically. Neural network (NN) control has had two major thrusts: approximate dynamic programming, which uses NN to approximately solve the optimal control problem, and NN in closed-loop feedback control.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Bibliography

  • Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5):779–791

    Article  MathSciNet  Google Scholar 

  • Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern Part B 38(4):943–949

    Article  Google Scholar 

  • Alessandri A, Baglietto M, Parisini T, Zoppoli R (1999) A neural state estimator with bounded errors for nonlinear systems. IEEE Trans Autom Control 44(11):2028–2042

    Article  MathSciNet  Google Scholar 

  • Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming, vol 5. Athena Scientific, Belmont

    MATH  Google Scholar 

  • Ioannou PA, Fidan B (2006) Adaptive control tutorial. Advances in design and control. SIAM

    Book  Google Scholar 

  • Jiang Y, Jiang ZP (2017) Robust adaptive dynamic programming. Wiley

    Google Scholar 

  • Kamalapurkar R, Walters PS, Rosenfeld JA, Dixon WE (2018) Reinforcement learning for optimal feedback control: a Lyapunov-based approach. Springer

    Google Scholar 

  • Kiumarsi B, Vamvoudakis KG, Modares H, Lewis FL (2018) Optimal and autonomous control using reinforcement learning: a survey. IEEE Trans Neural Netw Learn Syst 29(6):2042–2062

    Google Scholar 

  • Kosmatopoulos EB, Polycarpou MM, Christodoulou MA, Ioannou PA (1995) High-order neural network structures for identification of dynamical systems. IEEE Trans Neural Netw 6(2):422–431

    Article  Google Scholar 

  • Lewis FL, Liu D (2012) Reinforcement learning and approximate dynamic programming for feedback control. IEEE Press computational intelligence series

    Google Scholar 

  • Lewis FL, Vamvoudakis KG (2011) Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data. IEEE Trans Syst Man Cybern Part B 41(1):14–25

    Article  Google Scholar 

  • Lewis FL, Jagannathan S, Yesildirek A (1999) Neural network control of robot manipulators and nonlinear systems. Taylor and Francis, London

    Google Scholar 

  • Lewis FL, Campos J, Selmic R (2002) Neuro-fuzzy control of industrial systems with actuator nonlinearities. Society of Industrial and Applied Mathematics Press, Philadelphia

    Book  Google Scholar 

  • Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst Mag 32(6):76–105

    Article  MathSciNet  Google Scholar 

  • Patre PM, MacKunis W, Kaiser K, Dixon WE (2008) Asymptotic tracking for uncertain dynamic systems via a multi layer neural network feedforward and RISE feedback control structure. IEEE Trans Autom Control 53(9):2180–2185

    Article  Google Scholar 

  • Slotine JJE, Li W (1987) On the adaptive control of robot manipulators. Int J Robot Res 6(3):49–59

    Article  Google Scholar 

  • Sutton RS, Barto AG (1998) Reinforcement learning an introduction. MIT Press, Cambridge, MA

    Book  Google Scholar 

  • Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888

    Article  MathSciNet  Google Scholar 

  • Vamvoudakis KG, Lewis FL (2011a) Multi-player non zero sum games: online adaptive learning solution of coupled Hamilton-Jacobi equations. Automatica 47(8):1556–1569

    Article  MathSciNet  Google Scholar 

  • Vamvoudakis KG, Lewis FL (2011b) Policy iteration algorithm for distributed networks and graphical games. In: Proceedings of 50th IEEE conference on decision and control, Orlando, pp 128–135

    Google Scholar 

  • Vamvoudakis KG, Lewis FL (2012) Online solution of nonlinear two-player zero-sum games using synchronous policy iteration. Int J Robust Nonlinear Control 22(13):1460–1483

    Article  MathSciNet  Google Scholar 

  • Vamvoudakis KG, Lewis FL, Hudas GR (2012) Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality. Automatica 48(8):1598–1611

    Article  MathSciNet  Google Scholar 

  • Vamvoudakis KG, Vrabie D, Lewis FL (2014) Online adaptive algorithm for optimal control with integral reinforcement learning. Int J Robust Nonlinear Control 24(17):2686–2710

    Article  MathSciNet  Google Scholar 

  • Vamvoudakis KG, Antsaklis PJ, Dixon WE, Hespanha JP, Lewis FL, Modares H, Kiumarsi B (2015) Autonomy and machine intelligence in complex systems: a tutorial. In: Proceedings of American control conference, Chicago, pp 5062–5079

    Google Scholar 

  • Vamvoudakis KG, Modares H, Kiumarsi B, Lewis FL (2017) Game theory-based control system algorithms with real-time reinforcement learning: how to solve multiplayer games online. IEEE Control Syst Mag 37(1):33–52

    Article  MathSciNet  Google Scholar 

  • Vamvoudakis KG, Lewis FL, Dixon WE (2019) Open-loop Stackelberg learning solution for hierarchical control problems. Int J Adapt Control Signal Process 33(2):285–299

    Article  MathSciNet  Google Scholar 

  • Vrabie D, Pastravanu O, Lewis FL, Abu-Khalaf M (2009) Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica 45(2):477–484

    Article  MathSciNet  Google Scholar 

  • Vrabie DL, Vamvoudakis KG, Lewis FL (2012) Optimal adaptive control and differential games by reinforcement learning principles. Control engineering series. IET Press

    Book  Google Scholar 

  • Werbos PJ (1989) Neural networks for control and system identification. In: Proceedings of IEEE conference decision and control, Orlando

    Book  Google Scholar 

Download references

Acknowledgements

This material is based upon the work supported by NSF CPS-1851588, ECCS-1839804, SATC-1801611, by Minerva Research Initiative N00014-18-1-2160, and by ONR N00014-17-1-2239, N00014-18-1-2221.

Author information

Authors and Affiliations

  1. University of Texas at Arlington, Automation & Robotics Research Institute, Fort Worth, TX, USA

    Frank L. Lewis

  2. The Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA, USA

    Kyriakos G. Vamvoudakis

Authors
  1. Frank L. Lewis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kyriakos G. Vamvoudakis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kyriakos G. Vamvoudakis .

Editor information

Editors and Affiliations

  1. Electrical and Computer Engineering, Boston University, Boston, MA, USA

    John Baillieul

  2. Automation and Control Solutions, Honeywell, Golden Valley, MN, USA

    Tariq Samad

Section Editor information

  1. South Kensington Campus, Electrical and Electronic Engineering, Imperial College, London, UK

    Thomas Parisini

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer-Verlag London Ltd., part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Lewis, F.L., Vamvoudakis, K.G. (2020). Neuro-Inspired Control. In: Baillieul, J., Samad, T. (eds) Encyclopedia of Systems and Control. Springer, London. https://doi.org/10.1007/978-1-4471-5102-9_224-3

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-3

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-5102-9

  • Online ISBN: 978-1-4471-5102-9

  • eBook Packages: Springer Reference EngineeringReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Chapter history

  1. Latest

    Neuro-Inspired Control
    Published:
    20 August 2020

    DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-3

  2. Neural Control and Approximate Dynamic Programming
    Published:
    08 December 2014

    DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-2

  3. Original

    Neural Control and Approximate Dynamic Programming
    Published:
    12 April 2014

    DOI: https://doi.org/10.1007/978-1-4471-5102-9_224-1

Search

Navigation