Advertisement

Adaptive Dynamic Programming for Control

Algorithms and Stability

  • Huaguang Zhang
  • Derong Liu
  • Yanhong Luo
  • Ding Wang

Part of the Communications and Control Engineering book series (CCE)

Table of contents

  1. Front Matter
    Pages I-XV
  2. Huaguang Zhang, Derong Liu, Yanhong Luo, Ding Wang
    Pages 1-25
  3. Huaguang Zhang, Derong Liu, Yanhong Luo, Ding Wang
    Pages 27-107
  4. Huaguang Zhang, Derong Liu, Yanhong Luo, Ding Wang
    Pages 109-160
  5. Huaguang Zhang, Derong Liu, Yanhong Luo, Ding Wang
    Pages 161-199
  6. Huaguang Zhang, Derong Liu, Yanhong Luo, Ding Wang
    Pages 201-221
  7. Huaguang Zhang, Derong Liu, Yanhong Luo, Ding Wang
    Pages 223-255
  8. Huaguang Zhang, Derong Liu, Yanhong Luo, Ding Wang
    Pages 257-307
  9. Huaguang Zhang, Derong Liu, Yanhong Luo, Ding Wang
    Pages 309-344
  10. Huaguang Zhang, Derong Liu, Yanhong Luo, Ding Wang
    Pages 345-393
  11. Huaguang Zhang, Derong Liu, Yanhong Luo, Ding Wang
    Pages 395-422
  12. Back Matter
    Pages 423-424

About this book

Introduction

There are many methods of stable controller design for nonlinear systems. In seeking to go beyond the minimum requirement of stability, Adaptive Dynamic Programming for Control approaches the challenging topic of optimal control for nonlinear systems using the tools of  adaptive dynamic programming (ADP). The range of systems treated is extensive; affine, switched, singularly perturbed and time-delay nonlinear systems are discussed as are the uses of neural networks and techniques of value and policy iteration. The text features three main aspects of ADP in which the methods proposed for stabilization and for tracking and games benefit from the incorporation of optimal control methods:
• infinite-horizon control for which the difficulty of solving partial differential Hamilton–Jacobi–Bellman equations directly is overcome, and  proof provided that the iterative value function updating sequence converges to the infimum of all the value functions obtained by admissible control law sequences;
• finite-horizon control, implemented in discrete-time nonlinear systems showing the reader how to obtain suboptimal control solutions within a fixed number of control steps and with results more easily applied in real systems than those usually gained from infinte-horizon control;
• nonlinear games for which  a pair of mixed optimal policies are derived for solving games both when the saddle point does not exist, and, when it does, avoiding the existence conditions of the saddle point.
Non-zero-sum games are studied in the context of a single network scheme in which policies are obtained guaranteeing system stability and minimizing the individual performance function yielding a Nash equilibrium.
In order to make the coverage suitable for the student as well as for the expert reader, Adaptive Dynamic Programming for Control:
• establishes the fundamental theory involved clearly with each chapter devoted to a clearly identifiable control paradigm;
• demonstrates convergence proofs of the ADP algorithms to deepen undertstanding of the derivation of stability and convergence with the iterative computational methods used; and
• shows how ADP methods can be put to use both in simulation and in real applications.
This text will be of considerable interest to researchers interested in optimal control and its applications in operations research, applied mathematics computational intelligence and engineering. Graduate students working in control and operations research will also find the ideas presented here to be a source of powerful methods for furthering their study.

The Communications and Control Engineering series reports major technological advances which have potential for great impact in the fields of communication and control. It reflects research in industrial and academic institutions around the world so that the readership can exploit new possibilities as they become available.

Keywords

Adaptive Dynamic Programming Finite-horizon Control Infinite-horizon Control Reinforcement Learning Zero-sum Game

Authors and affiliations

  • Huaguang Zhang
    • 1
  • Derong Liu
    • 2
  • Yanhong Luo
    • 3
  • Ding Wang
    • 4
  1. 1.College of Information Science Engin.Northeastern UniversityShenyangChina, People's Republic
  2. 2.Institute of Automation, Laboratory of Complex SystemsChinese Academy of SciencesBeijingChina, People's Republic
  3. 3.College of Information Science Engin.Northeastern UniversityShenyangChina, People's Republic
  4. 4.Institute of Automation, Laboratory of Complex SystemsChinese Academy of SciencesBeijingChina, People's Republic

Bibliographic information

  • DOI https://doi.org/10.1007/978-1-4471-4757-2
  • Copyright Information Springer-Verlag London 2013
  • Publisher Name Springer, London
  • eBook Packages Engineering
  • Print ISBN 978-1-4471-4756-5
  • Online ISBN 978-1-4471-4757-2
  • Series Print ISSN 0178-5354
  • Buy this book on publisher's site
Industry Sectors
Pharma
Materials & Steel
Automotive
Biotechnology
Electronics
IT & Software
Telecommunications
Energy, Utilities & Environment
Aerospace
Oil, Gas & Geosciences
Engineering