Advertisement

A Critical Review of Learning Control Research

  • K. S. Fu

Abstract

In designing an optimal control system, if the a priori information required is unknown or incompletely known, one possible approach is to design a controller which is capable of estimating the unknown information during its operation and determining the optimal control action on the basis of the estimated information. If the estimated information gradually approaches the true information as time proceeds, then the controller designed will approach the optimal controller; and, consequently, the performance of the control system is gradually improved. Because of the gradual improvement of performance due to the improvement of the estimated unknown information, this class of control systems has been called learning control systems. Design techniques proposed for learning control systems include: (1) trainable controllers using pattern classifiers, (2) reinforcement learning algorithms, (3) Bayesian estimation, (4) stochastic approximation, and (5) stochastic automata models. A survey of these techniques can be found in [1]. A general formulation using stochastic approximation has been treated extensively in [2, 3]. Practical applications include spacecraft control systems, the control of valve actuators, power systems, and production processes. In addition, several nonlinear learning algorithms have recently been proposed.

Keywords

Robot System Stochastic Approximation Reinforcement Learning Algorithm Intelligent Control System Optimal Control Policy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    K. S. Fu, “Learning Control Systems—Review and Outlook,” IEEE Trans. on Automatic Control, Vol. AC-15, No. 2, April 1970.Google Scholar
  2. 2.
    Ya. Z. Tsypkin, “Adaptation and Learning in Automatic Systems,” (English Translation), Academic Press, 1971.Google Scholar
  3. 3.
    Ya. Z. Tsypkin, “Learning Control Systems,” Automatika i Telemekhanika, No. 4, April 1970.Google Scholar
  4. 4.
    L. R. Cockrell and K. S. Fu, “On Search Techniques in Switching Environments,” Proc. IEEE 9th Symposium on Adaptive Processes, Dec. 7–9, 1970, Austin, Texas.Google Scholar
  5. 5.
    L. A. Zadeh, “Fuzzy Sets and Systems,” Symposium on System Theory, Polytechnic Institute of Brooklyn, April 20–22, 1965.Google Scholar
  6. 6.
    W. G. Wee and K. S. Fu, “A Formulation of Fuzzy Automata and Its Application as a Model of Learning Systems,” IEEE Trans. on Systems Science and Cybernetics, Vol. SSC-5, July 1969.Google Scholar
  7. 7.
    K. Asai and S. Katajima, “Learning Control Using Fuzzy Automata,” US-Japan Seminar on Learning Processes in Control Systems, Nagoya, Japan, August 1970.Google Scholar
  8. 8.
    L. R. Young, “On Adaptive Manual Control,” IEEE Trans. on Man-Machine Systems, Vol. MMS-10, No. 4, Dec. 1969.Google Scholar
  9. 9.
    D. W. Gilstad and K. S. Fu, “A Two-Dimensional, Pattern Recognizing, Adaptive Model of a Human Controller,” Proc. 6th Annual Conference on Manual Control, April 1970.Google Scholar
  10. 10.
    A. E. Preyss and J. L. Meiry, “Stochastic Modeling of Human Learning behavior,” USC-NASA Conference on Manual Control, NASA SP-144, March 1967.Google Scholar
  11. 11.
    R. E. Thomas and J. T. Tou, “Evaluation of Heuristics by Human Operators in Control Systems,” IEEE Trans. on Systems Science and Cybernetics, Vol. SSC-4, No. 1, March 1968.Google Scholar
  12. 12.
    D. Michie and R. A. Chambers, “Boxes: an Experiment in Adaptive Control,” Machine Intelligence, Vol. 2, edited by E. Dale and D. Michie, 1967.Google Scholar
  13. 13.
    A. G. Ivakhnenko, “Heuristic Self-Organization in Problems of Engineering Cybernetics,” 4th IFAC Congress, Warsaw, Poland, June 16–21, 1969.Google Scholar
  14. 14.
    K. Nakamura and M. Oda, “Heuristics and Learning Control,” US-Japan Seminar on Learning Process in Control Systems, Nagoya, Japan, August 1970.Google Scholar
  15. 15.
    J. E. Doran, “Planning and Generalization in an Automaton/ Environment System,” Machine Intelligence, Vol. 4, edited by B. Meltzer and D. Michie, 1969.Google Scholar
  16. 16.
    T. B. Sheridan and W. R. Ferrell, “Human Control of Remote Computer Manipulators,” Proc. International Joint Conference on Artificial Intelligence, May 1969.Google Scholar
  17. 17.
    D. E. Whitney, “State Space Models of Remote Manipulation Tasks,” Proc. International Joint Conference on Artificial Intelligence, May 1969.Google Scholar
  18. 18.
    A. Freedy, F. Hull and J. Lyman, “The Application of A Theoretical Learning Model to Remote Manipulator Control,” Proc. 6th Annual Conference on Manual Control, April 1970.Google Scholar
  19. 19.
    N. J. Nilsson, “A Mobil Automaton: An Application of Artificial Intelligence Techniques,” Proc. International Joint Conference on Artificial Intelligence, May 1969.Google Scholar
  20. 20.
    M. L. Minsky and S. A. Papert, “Research on Intelligent Automata,” Status Report II, Project MAC, M.I.T., Sept. 1967.Google Scholar
  21. 21.
    K. Pingle, J. A. Singer and W. M. Wichman, “Computer Control of a Mechanical Arm Through Visual Input,” Proc. IFIP Congress, Edinburgh, Vol. II, 1968.Google Scholar
  22. 22.
    P. H. Winston, “Learning Structural Descriptions from Examples,” Rept. MAC TR-76, Project MAC, M.I.T., Sept. 1970.Google Scholar
  23. 23.
    W. G. Keckler and R. E. Larson, “Control of a Robot in a Partially Unknown Environment,” Automatica, IFAC Journal, Vol. 6, May 1970.Google Scholar

Copyright information

© Plenum Press, New York 1971

Authors and Affiliations

  • K. S. Fu
    • 1
  1. 1.Purdue UniversityLafayetteUSA

Personalised recommendations