Skip to main content
SpringerLink
Log in

Learning Control Systems

  • Chapter
Book cover Advances in Information Systems Science

Abstract

In designing an optimal control system, if all the a priori information about the controlled process (plant-environment) is known and can be described deterministically, the optimal controller is usually designed by deterministic optimization techniques. If all or a part of the a priori information can only be described statistically—for example, in terms of probability distribution or density functions—then stochastic or statistical design techniques will be used. However, if the a priori information required is unknown or incompletely known, in general an optimal design cannot be achieved. Two different approaches have been taken to solve this class of problems. One approach is to design a controller based only upon the amount of information available. In that case the unknown information is either ignored or is assumed to take on some known values chosen according to the designer’s best guess. The second approach is to design a controller which is capable of estimating the unknown information during its operation and of determining an optimal control action on the basis of the estimated information. In the first case a rather conservative design criterion (for example, the minimax criterion) is often used; the systems designed are in general inefficient and suboptimal. In the second case, if the estimated information gradually approaches the true information as time proceeds, then the controller thus designed will approach to the optimal controller.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. N. J. Nilsson, “Learning Machines,“ McGraw-Hill Book Co., New York (1965)

    Google Scholar 

  2. C. K. Chow, An Optimum Character Recognition System Using Decision Functions, IRE Trans. on Electronic Computers, EC-6, 247–254 (Dec. 1957)

    Google Scholar 

  3. K. S. Fu, A Sequential Decision Model for Optimum Recognition, in “Biological Prototypes and Synthetic Systems,“ Vol. I, Plenum Press, New York (1962)

    Google Scholar 

  4. Y. T. Chien and K. S. Fu, A Modified Sequential Recognition Machine Using Time-Varying Stopping Boundaries, IEEE Trans. on Information Theory IT-12, 206–214 (April 1966)

    Article  Google Scholar 

  5. K. S. Fu, Y. T. Chien, and G. P. Cardillo, A Dynamic Programming Approach to Sequential Pattern Recognition, IEEE Trans. on Electronic Computers, Dec. 1967, 790–803

    Google Scholar 

  6. F. W. Smith, Contact Control by Adaptive Pattern-Recognition Techniques, Tech. Rept. No. 6762–2, Stanford Electronics Laboratories, Stanford, California (April 1964)

    Google Scholar 

  7. B. Widrow and F. W. Smith, Pattern Recognizing Control Systems, in: “Computer and Information Sciences,“ (J. T. Tou and R. H. Wilcox, eds.), Spartan Books, Washington, D. C. (1964)

    Google Scholar 

  8. R. Bush and F. Mosteller, “Stochastic Models for Learning,“ John Wiley and Sons, New York (1955)

    Google Scholar 

  9. M. L. Tsetlin, On the Behavior of Finite Automata in Random Environments, Avtomatika i Telemekhanika 22 (10), 1345–1354 (1961)

    Google Scholar 

  10. V. I. Varshavskii and I. P. Vorontsova, On the Behavior of Stochastic Automata with Variable Structure, Avtomatika i Telemekhanika 24 (3), 353–360 (1963)

    Google Scholar 

  11. K. S. Fu and R. W. McLaren, An Application of Stochastic Automata to the Synthesis of Learning Systems, Technical Report TR-EE-65-17, School of Electrical Engineering, Purdue University (Sept. 1965)

    Google Scholar 

  12. G. J. McMurtry and K. S. Fu, A Variable Structure Automaton Used as a Multi-Modal Searching Technique, IEEE Trans. on Automatic Control AC-11, 379–387 (July 1966)

    Article  Google Scholar 

  13. K. S. Fu and Z. J. Nikolic, On Some Reinforcement Techniques and their Relations With Stochastic Approximation, IEEE Trans. on Automatic Control AC-11 (Oct. 1966)

    Google Scholar 

  14. M. D. Waltz and K. S. Fu, A Heuristic Approach to Reinforcement Learning Control Systems, IEEE Trans. on Automatic Control AC-10, 390–398 (Oct. 1965)

    Article  Google Scholar 

  15. J. M. Mendel, Survey of Learning Control Systems for Space Vehicle Applications, Preprints, 1966 JACC (Aug. 1966)

    Google Scholar 

  16. H. Freeman, On the Digital Computer Classification of Geometric Line Patterns, in “Proc. National Electronics Conference,” Vol. 18 (Oct. 1962)

    Google Scholar 

  17. J. T. Tou, “Modern Control Theory,” McGraw-Hill Book Co., New York (1964)

    Google Scholar 

  18. J. C. Hsu and W. E. Meserve, Decision-Making in Adaptive Control Systems, IRE Trans. on Automatic Control Jan. 1962, 24–32

    Google Scholar 

  19. N. Ula and M. Kim, An Empirical Bayes Approach to Adaptive Control, J. Franklin Institute, 280 (3) (Sept. 1965)

    Google Scholar 

  20. Y. Sawaragi, Y. Sunahara, and T. Nakamizo, “Statistical Decision Theory in Adaptive Control Systems,” Academic Press, New York (1967)

    Google Scholar 

  21. K. S. Fu, A Class of Learning Control Systems Using Statistical Decision Functions, in “Proceedings of the Second IFAC (Teddington) Symposium on the Theory of Self-Adaptive Control Systems, Sept. 1965,“ Plenum Press, New York (1967)

    Google Scholar 

  22. N. Abramson and D. Braverman, Learning to Recognize Patterns in a Random Environment, IRE Trans. on Information Theory IT-8, 58–63 (Sept. 1962)

    Article  Google Scholar 

  23. D. G. Keehn, A Note on Learning for Gaussian Properties, IEEE Trans. on Information Theory IT-11, 126–132 (Jan. 1965)

    Article  Google Scholar 

  24. S. C. Fralick, Learning to Recognize a Pattern Without a Teacher, IEEE Trans. on Information Theory IT-13, 57–64 (Jan. 1967)

    Article  Google Scholar 

  25. H. Teicher, Identifiability of Finite Mixtures, Ann. Math. Stat. 34, 1265–1269 (Dec. 1963)

    Article  Google Scholar 

  26. Y. T. Chien and K. S. Fu, On Bayesian Learning and Stochastic Approximation, IEEE Trans. on System Science and Cybernetics June 1967, 28–38

    Google Scholar 

  27. Z. J. Nikolic and K. S. Fu, On the Estimation and Decomposition of Mixture Using Stochastic Approximation, in: “Proc. 1967 SWIEEECO, April 1967.”

    Google Scholar 

  28. Z. J. Nikolic and K. S. Fu, An Algorithm for Learning Without External Supervision and Its Application to Learning Control Systems, IEEE Trans. on Automatic Control, AC-11, 414–422 (July 1966)

    Article  Google Scholar 

  29. Ya. Z. Tsypkin, Adaptation, Learning and Self-Learning in Control Systems, paper presented at Third IFAC Congress, London (June 20–25, 1966)

    Google Scholar 

  30. K. S. Fu and Z. J. Nikolic, A Study of Learning Systems Operating in Unknown Stationary Environments, Technical Report TR-EE-66-20, School of Electrical Engineering, Purdue University (Nov. 1966)

    Google Scholar 

  31. M. A. Aizerman, E. M. Braverman, and L. I. Rozonoer, Theoretical Foundations of the Potential Function Method in Pattern Recognition, Avtomatika i Telemekhanika 25, 917–936 (1964)

    Google Scholar 

  32. M. A. Aizerman, E. M. Braverman, and L. I. Rozonoer, The Probability Problem of Pattern Recognition Learning and the Method of Potential Functions, Avtomatika i Telemekhanika 25, 1307–1323 (1964)

    Google Scholar 

  33. M. A. Aizerman. E. M. Braverman, and L. I. Rozonoer, The Method of Potential Functions for the Problem of Restoring the Characteristic of a Function Converter from Randomly Observed Points, Avtomatika i Telemekhanika 25, 1705–2213 (1964)

    Google Scholar 

  34. E. M. Braverman and E. S. Pyatnitskii, Estimation of the Rate of Convergence of Algorithms Based on the Potential Function Method, Avtomatika i Telemakhanika, 27, 95–112 (1966)

    Google Scholar 

  35. G. J. Simmons, Iterative Storage of Multidimensional Functions in Discrete Distributed Memories, in “Computer and Information Sciences - II” (J. T. Tou, ed.), pp. 261–280, Academic Press, New York (1967)

    Google Scholar 

  36. N. Aronszajn, Theory of Reproducing Kernels, Trans. Am. Math. Soc. 68, 337–404 (1950)

    Article  Google Scholar 

  37. F. G. Tricomi, “Integral Equations,” John Wiley and Sons, New York (1957)

    Google Scholar 

  38. Z. J. Nikolic and K. S. Fu, A Mathematical Model of Learning in an Unknown Random Environment, in: “Proc. National Electronics Conference,“ Vol. 22, 607–612, NEC (1966)

    Google Scholar 

  39. H. Robbins and S. Monro, A Stochastic Approximation Method, Ann. Math. Stat. 22 (1), 400–407 (1951)

    Article  Google Scholar 

  40. J. A. Blum, Approximation Methods which Converge with Probability One, Ann. Math. Stat. 25 (2), 382–386 (1954)

    Article  Google Scholar 

  41. J. A. Blum, Multidimensional Stochastic Approximation Procedures, Ann. Math. Stat. 25 (4), 737–744 (1954)

    Article  Google Scholar 

  42. E. G. Gladyshev, On Stochastic Approximation, Teoriya Veroyatnostei i ee Primeneniya 10 (2) (1965)

    Google Scholar 

  43. M. Driml and N. Nedoma, Stochastic Approximations for Continuous Random Processes, in “Trans. Second Prague Conference on Information Theory, Statistical Decision Functions, and Random Processes, 1960

    Google Scholar 

  44. O. Hans and A. Spacek, Random Fixed Point Approximation by Differentiable Trajectories, in “Trans. Second Prague Conference on Information Theory, Statistical Decision Functions, and Random Processes (1960)

    Google Scholar 

  45. M. Driml and O. Hans, Continuous Stochastic Approximations, in “Trans. Second Prague Conference on Information Theory, Statistical Decision Functions, and Random Processes (1960)

    Google Scholar 

  46. J. Kiefer and J. Wolfowitz, Stochastic Estimation of the Maximum of a Regression Function, Ann. Math. Stat., 23 (3), 462–466 (1952)

    Article  Google Scholar 

  47. J. A. Blum, A Note on Stochastic Approximation, Proc. Am. Math. Soc. 9, 404–407 (1958)

    Article  Google Scholar 

  48. J. Sacks, Asymptotic Distribution of Stochastic Approximations, Ann. Math. Stat. 29 (2), 373–405 (1958)

    Article  Google Scholar 

  49. D. L. Sakrison, A Continuous Kiefer-Wolfowitz Procedure for Random Processes, Ann. Math. Stat. 35 (2), 590–599 (1964)

    Article  Google Scholar 

  50. A. Dvoretzky, On Stochastic Approximation, in “Proc. Third Berkeley Symposium on Mathematical Statistics and Probability, “Vol. 1 (1956)

    Google Scholar 

  51. K. B. Gray, Application of Stochastic Approximation to the Optimization of Random Circuits, in “Proc. Sixteenth Symposium on Applied Mathematics,“ Vol. XVI (1964)

    Google Scholar 

  52. H. Kesten, Accelerated Stochastic Approximation, Ann. Math. Stat. 29 (1), 41–59 (1958)

    Article  Google Scholar 

  53. V. Fabian, Stochastic Approximation Methods, Czech. Math. J. 10 (1), 123–159 (1960)

    Google Scholar 

  54. D. Burkholder, On A Class of Stochastic Approximation Processes, Ann. Math. Stat. 27 (4), 1044–1059 (1956)

    Article  Google Scholar 

  55. H. D. Block, Estimates of Error for Two Modifications of the Robbins-Monro Stochastic Approximation Process, Ann. Math. Stat. 28 (4) (1957)

    Article  Google Scholar 

  56. J. H. Venter, An Extension of the Robbins-Monro Procedure, Ann. Math. Stat. 38 (2), 181–190 (1967)

    Article  Google Scholar 

  57. V. Fabian, Stochastic Approximation of Minima with Improved Asymptotic Speed, Ann. Math. Stat. 38 (2) 191–200 (1967)

    Article  Google Scholar 

  58. K. S. Fu, Z. J. Nikolic, Y. T. Chien, and W. G. Wee, On the Stochastic Approximation and Related Learning Techniques, Technical Report TR-EE-66-6, School of Electrical Engineering, Purdue University (April 1966)

    Google Scholar 

  59. K. S. Fu, Learning Control Systems, in “Computer and Information Sciences (J. T. Tou and R. H. Wilcox, eds.), Spartan Books, Washington, D. C. (1964)

    Google Scholar 

  60. J. Sklansky, Learning Systems for Automatic Control, IEEE Trans. on Automatic Control AC-11 (Jan. 1966)

    Google Scholar 

  61. J. T. Tou and J. D. Hill, Steps Toward Learning Control, Preprints, 1966 JACC (Aug. 1966)

    Google Scholar 

  62. K. S. Fu, Stochastic Automata as Models of Learning Systems, in “Computer and Information Sciences-II,” (J. T. Tou, ed.), Academic Press, New York (1967)

    Google Scholar 

  63. R. W. McLaren, A Stochastic Automaton Model for Synthesis of Learning Systems, IEEE Trans. on System Science and Cybernetics SSC-2, 109–114 (Dec. 1966)

    Google Scholar 

  64. G. D. Bruce and K. S. Fu, A Model for Finite-State Probabilistic Systems, in “Proc. of the First Allerton Conference on Circuit and System Theory,“ University of Michigan Press (1963)

    Google Scholar 

  65. J. W. Carlyle, Equivalent Stochastic Sequential Machines, Technical Report, Series 60, Issue 415, Electronics Research Laboratory, University of California, Berkeley (1961)

    Google Scholar 

  66. J. W. Carlyle, Reduced Forms for Stochastic Sequential Machines, j. Math. Analysis and Application I, 167–175 (1963)

    Article  Google Scholar 

  67. G. C. Bacon, The Decomposition of Stochastic Automata, Information and Control I, 320–339 (1964)

    Article  Google Scholar 

  68. G. C. Bacon, Minimal-State Stochastic Finite-State Systems, IEEE Trans. on Circuit Theory CT-11, 307–308 (1964)

    Google Scholar 

  69. M. O. Rabin, Probabilistic Automata, Information and Control 6, 230–245 (1963)

    Article  Google Scholar 

  70. A. Paz, Some Aspects of Probabilistic Automata, Information and Control 9 (1), 26–60 (1966)

    Article  Google Scholar 

  71. G. N. Tsertsvadze, Stochastic Automata and the Problem of Constructing Reliable Automata of Unreliable Elements, Avtomatika i Telemekhanika 25 (1964)

    Google Scholar 

  72. H. E. Maurer, An Approach to the Design of Reliable Radiation Hardened Integrated Logic and Sequential Circuits, Ph. D. Thesis, Purdue University (Aug. 1965)

    Google Scholar 

  73. G. N. Tsertsvadze, Certain Properties of Stochastic Automata and Methods, for Synthesizing Them, Avtomatika i Telemekhanika 24, 353–360 (1963)

    Google Scholar 

  74. M. L. Tsetlin and V. Ju. Krylov, On the Automata Games, Avtomatika i Telemekhanika 24 (1963)

    Google Scholar 

  75. M. L. Tsetlin, S. L. Ginzburg, and V. Ju. Krylov, On an Example of a Many-Automata Games, Avtomatika i Telemekhanika 26 (1965)

    Google Scholar 

  76. R. L. Kashyap, Optimization of Stochastic Finite State Systems, paper presented of the Fifth Symposium on Discrete Adaptive Processes, Chicago, Illinois, (Oc. 3–5, 1966)

    Google Scholar 

  77. K. S. Fu and G. J. McMurtry, A Study of Stochastic Automata as Models of Adaptive and Learning Controllers, Rept. TR-EE65–17, Purdue University (June 1965)

    Google Scholar 

  78. B. Chandrasekaran and D. W. C. Shen, On Expediency and Convergence in Variable Structure Automata, paper presented at the Fifth Symposium on Discrete Adaptive Process, Chicago, Illinois (Oct. 3–5, 1966)

    Google Scholar 

  79. K. S. Fu, On Learning Techniques in Engineering Cybernetic Systems, in: “Proc. Vth International Congress on Cybernetics, Sept. 11–14, 1967, Namur, Belgium.”

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Purdue University, Lafayette, Indiana, USA

    K. S. Fu

Authors
  1. K. S. Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. College of Engineering, University of Florida, Gainesville, Florida, USA

    Julius T. Tou

Rights and permissions

Reprints and permissions

Copyright information

© 1969 Plenum Press

About this chapter

Cite this chapter

Fu, K.S. (1969). Learning Control Systems. In: Tou, J.T. (eds) Advances in Information Systems Science. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-9050-7_5

Download citation

  • DOI: https://doi.org/10.1007/978-1-4615-9050-7_5

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4615-9052-1

  • Online ISBN: 978-1-4615-9050-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation