Recent Developments in Learning Automata

  • Kumpati S. Narendra
  • Richard M. WheelerJr.

Abstract

The paper surveys the major developments in learning automata theory and applications in the last two decades. Since a survey article on the subject appeared in 1977, the emphasis is on subsequent developments. Of particular importance is some recent work on the use of many automata interacting in a decentralized manner. This framework provides a conceptual focus and an analytical basis for future research on modeling and control of complex systems. Applications of the theory are also reviewed, with special attention devoted to the problem of traffic routing in telecommunication networks.

Keywords

Markov Chain Random Environment Learn Automaton Finite Markov Chain Decentralize Manner 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    K.S. Narendra and M.A.L. Thathachar, “Learning Automata - A Survey,”IEEE Transactions on Systems, Man and Cybernetics, vol. SMC-4, pp. 323 – 334, 1974.Google Scholar
  2. [2]
    K.S. Narendra and S. Lakshmivarahan, “Learning Automata - A Critique,”Journal of Cybernetics and Information Science, vol. 1, pp. 53 – 65, 1977.Google Scholar
  3. [3]
    R.R. Bush and F. Mosteller, Stochastic Models for Learning, Wiley, New York, 1958.Google Scholar
  4. [4]
    R.C. Atkinson, G.H. Bower and E.J. Crothers, An Introduction to Mathematical Learning Theory, Wiley, New York, 1965.Google Scholar
  5. [5]
    M. Iosifescu and R. Theodorescu, Random Processes and Learning, Springer, New York, 1969.MATHCrossRefGoogle Scholar
  6. [6]
    M.F. Norman, Markov Processes and Learning Models, Academic, New York, 1972.MATHGoogle Scholar
  7. [7]
    M.L. Tsetlin, “On the Behavior of Finite Automata in Random Media,”Automation and Remote Control, vol. 22, pp. 1345 – 1354, 1961.MathSciNetGoogle Scholar
  8. [8]
    V.I. Varshayskii and I.P. Vorontsova, “On the Behavior of Stochastic Automata with Variable Structure,”Automation and Remote Control, vol. 24, pp. 327 – 333, 1963.MathSciNetGoogle Scholar
  9. [9]
    “Special Issue on Learning Automata,”J. of Cybernetics and Information Science, vol. 1, No. 2–4, 1977.Google Scholar
  10. [10]
    S. Lakshmivarahan, Learning Algorithms Theory and Applications, New York: Springer-Verlag, 1981.MATHCrossRefGoogle Scholar
  11. [11]
    M.L. Tsetlin, Automaton Theory and Modeling of Biological Systems, New York: Academic, 1973.Google Scholar
  12. [12]
    A.G. Barto, P. Anandan and C.W. Anderson, “Cooperativity in Networks of Pattern Recognizing Stochastic Learning Automata,” (this volume).Google Scholar
  13. [13]
    J.H. Holland, Adaptation in Natural and Artificial Systems, University of Michigan Press, Ann Arbor, 1975.Google Scholar
  14. [14]
    S.W. Wilson, “Knowledge Growth in an Artificial Animal,” (this volume).Google Scholar
  15. [15]
    D.E. Goldberg, “Controlling Dynamic Systems with Genetic Algorithms and Rule Learning,” (this volume).Google Scholar
  16. [16]
    L.G. Mason, “Self-Optimizing Allocation Systems,” Ph. D. dissertation, University of Saskatchewan, Saskatoon, Sask., Canada, 1972.Google Scholar
  17. [17]
    K.S. Narendra, L.G. Mason and S.S. Tripathi, “Application of Learning Automata to Telephone Traffic Routing Problems,” Becton Center Tech. Report CT-69, Dept. of Engg. and Applied Science, Yale University, New Haven, Ct., Jan. 1974.Google Scholar
  18. [18]
    K.S. Narendra, A.E. Wright and L.G. Mason, “Application of Learning Automata to Telephone Traffic Routing and Control,”IEEE Transactions on Systems, Man and Cybernetics, vol. SMC-7, pp. 785 – 792, 1977.Google Scholar
  19. [19]
    P. Mars and M. S. Chrystall, “Real-Time Telephone Traffic Simulation using Learning Automata Routing,” S&IS Report No. 7909, Dept. of Engg. and Applied Science, Yale University, New Haven, CT, Nov. 1979.Google Scholar
  20. [20]
    K.S. Narendra and M. A. L. Thathachar, “On the Behavior of a Learning Automaton in a Changing Environment with Application to Telephone Traffic Routing,”IEEE Trans. on Systems, Man and Cybernetics, vol. SMC-10, pp. 262 – 269, 1980.Google Scholar
  21. [21]
    P. R. Srikantakumar and K.S. Narendra, “A Learning Model for Routing in Telephone Networks,”SIAM J. Control and Optimization, vol. 20, pp. 34 – 57, 1982.MathSciNetMATHCrossRefGoogle Scholar
  22. [22]
    O.V. Nedzelnitsky, Jr. and K.S. Narendra, “A Learning Approach to Routing in Data Communication Networks,” Tech. Report No. 8212, Center for Systems Science, Yale University, New Haven, Ct.Google Scholar
  23. [23]
    K.S. Narendra and P. Mars, “The Use of Learning Algorithms in Telephone Traffic Routing - A Methodology,”Automatica, vol. 19, pp. 495 – 502, 1983.MATHCrossRefGoogle Scholar
  24. [24]
    K.S. Narendra and R.M. Wheeler, Jr, “Routing in Communication Networks - A Case Study of Learning in Large Scale Systems,”Journal of Large Scale Systems, vol. 8, pp. 211 – 222, 1985.MATHGoogle Scholar
  25. [25]
    L.G. Mason and X.D. Gu, “Learning Automata Models for Adaptive Flow Control in Packet-Switching Networks,” (this volume).Google Scholar
  26. [26]
    A.A. Hashim, S. Amir and P. Mars, “Application of Learning Automata to Image Data Compression,” (this volume).Google Scholar
  27. [27]
    L.P. Devroye, “A Class of Optimal Performance Directed Probabilistic Automata,”IEEE Trans. on Systems, Man and Cybernetics, vol. SMC-6, pp. 777 – 784, 1976.Google Scholar
  28. [28]
    M.A.L. Thathachar and P.S. Sastry, “A New Approach to the Design of Reinforcement Schemes for Learning Automata,” Tech. Report EE/60, Dept. Elec. Engg., Indian Inst. Sci., Bangalore, India, Dec. 1983.Google Scholar
  29. [29]
    O.V. Nedzelnitsky, Jr., “The Application of Learning Methodology to Message Routing in Data Communication Networks,” Ph.D. Thesis, Dept. Elec. Engg., Yale University, Dec. 1983.Google Scholar
  30. [30]
    D.E. Koditschek and K.S. Narendra, “Fixed Structure Automata in a Multi-Teacher Environment,”IEEE Transactions on Systems, Man and Cybernetics, vol. SMC-7, pp. 616 – 624, 1977.Google Scholar
  31. [31]
    M. A. L. Thathachar and R. Bhakthavathsalam, “Learning Automaton Operating in Parallel Environments,”J. of Cybernetics and Informatin Science - Special Issue on Learning Automata, vol. 1, pp. 121 – 127, 1977.Google Scholar
  32. [32]
    N. Baba, “On the Learning Behaviors of Variable-Structure Stochastic Automaton in the General N-Teacher Environment,”IEEE Trans. Syst., Man and Cybernetics, vol. SMC-13, pp. 224 – 231, 1983.Google Scholar
  33. [33]
    R.M. Wheeler, Jr. and K.S. Narendra, “Learning Models for Decentralized Decision Making,”Automatica, vol. 21, pp. 479 – 484, 1985.MATHCrossRefGoogle Scholar
  34. [34]
    V.Y. Krylov and M.L. Tsetlin, “Games Between Automata,”Autom. and Remote Cont., vol. 24, pp. 975 – 987, 1963.Google Scholar
  35. [35]
    S. Lakshmivarahan and K.S. Narendra, “Learning Algorithms for Two-Person Zero-Sum Stochastic Games with Incomplete Information,”Mathematics of Operations Research, vol. 6, pp. 379 – 386, 1981.MathSciNetMATHCrossRefGoogle Scholar
  36. [36]
    S. Lakshmivarahan and K.S. Narendra, “Learning Algorithms for Two-Person Zero-Sum Stochastic Games with Incomplete Information: A Unified Approach,”SIAM J. Control and Optimization, vol. 20, pp. 541 – 552, 1982.MathSciNetMATHCrossRefGoogle Scholar
  37. [37]
    K.S. Narendra and R.M. Wheeler, Jr., “An N-Player Sequential Stochastic Game with Identical Payoffs,”IEEE Trans. on Syst., Man and Cybernetics, vol. 13, pp. 1154 – 1158, 1983.MathSciNetMATHCrossRefGoogle Scholar
  38. [38]
    K.R. Ramakrishnan, “Hierarchical Systems and Cooperative Games of Learning Automata,” Ph.D. Thesis, Dept. Elec. Engg., Indian Inst. Sci., Bangalore, India, July 1982.Google Scholar
  39. [39]
    R.M. Wheeler, Jr. and K.S. Narendra, “Decentralized Learning in Finite Markov Chains,” Tech. Report No. 8410, Center for Systems Science, Yale University, New Haven, Ct., Dec. 1984; To appear in IEEE Transactions on Automatic Control.Google Scholar
  40. [40]
    Y.M. El-Fattah, “Multi-Automaton Games: A Rationale for Expedient Collective Behavior,”Systems and Control Letters, vol. 1, pp. 332 – 339, 1982.MATHCrossRefGoogle Scholar
  41. [41]
    Y.M. El-Fattah, “Fairness and Mutual Profitability in Collective Behavior of Automata,”IEEE Trans. Syst. Man and Cybern., vol. SMC-13, pp. 236 – 241, 1983.Google Scholar
  42. [42]
    V.I. Varshayskii, “Collective Behavior and Control Problems,” in Machine Intelligence, vol. 3, D. Michie (ed), Edinburgh University Press, Edinburgh, 1968.Google Scholar
  43. [43]
    N. Baba and Y. Sawaragi, “On the Learning Behavior of Stochastic Automata Under a Nonstationary Random Environment,”IEEE Trans. Syst., Man and Cybern., vol. SMC-5, pp. 273 – 275, 1975.Google Scholar
  44. [44]
    M.A.L. Thathachar and K.R. Ramakrishnan, “A Hierarchical System of Learning Automata,”IEEE Trans. Syst., Man and Cybern., vol. SMC-11, pp. 236 – 242, 1981.Google Scholar
  45. [45]
    B.T. Mitchell and D.I. Kountanis, “A Reorganization Scheme for a Hierarchical System of Learning Automata,”IEEE Trans. Syst., Man and Cybern., vol. SMC-14, pp. 328 – 334, 1984.Google Scholar
  46. [46]
    R.E. Bellman, “A Markovian Decision Process,”J. Math. Mech., vol. 6, pp. 679 – 684, 1957.MathSciNetMATHGoogle Scholar
  47. [47]
    R.A. Howard, Dynamic Programming and Markov Processes, MIT Press, Cambridge, Ma., 1960.Google Scholar
  48. [48]
    V. Borkar and P. Varaiya, “Adaptive Control of Markov Chains, I: Finite Parameter Set,”IEEE Trans. Autom. Cont., vol. AC-24, pp. 953 – 958, 1979.Google Scholar
  49. [49]
    P.R. Kumar and W. Lin, “Optimal Adaptive Controllers for Unknown Markov Chains,”IEEE Trans. Autom. Cont., vol. AC-27, pp. 765 – 774, 1982.Google Scholar
  50. [50]
    Y.M. El-Fattah, “Stochastic Automata Modeling of Certain Problems of Collective Behavior,”IEEE Trans. Syst., Man and Cybern., vol. SMC-10, pp. 304 – 314, 1980.Google Scholar
  51. [51]
    A.G. Barto and P. Anandan, “Pattern Recognizing Stochastic Learning Automata,”IEEE Trans. Syst., Man and Cybern., vol. SMC-15, pp. 360 – 375, 1985.Google Scholar
  52. [52]
    P.R. Srikanta Kumar, “Application of Learning Theory to Communication Networks Control,” Proc. of the Third Yale Workshop on Applications of Adaptive Systems Theory, New Haven, Ct., pp. 135 – 141, 1983.Google Scholar
  53. [53]
    M.R. Meybodi and S. Lakshmivarahan, “A Learning Approach to Priority Assignment in a Two Class M/M/1 Queuing System with Unknown Parameters.” Proc. of the Third Yale Workshop on Applications of Adaptive Systems Theory, New Haven, Ct., pp. 106 – 109, 1983.Google Scholar
  54. [54]
    R.M. Glorioso, Engineering Intelligent Systems, Digital Press, Bedford, Ma., 1980.Google Scholar

Copyright information

© Springer Science+Business Media New York 1986

Authors and Affiliations

  • Kumpati S. Narendra
    • 1
  • Richard M. WheelerJr.
    • 1
  1. 1.Center for Systems ScienceYale UniversityUSA

Personalised recommendations