Cybernetics and Learning Automata

Part of the Springer Handbooks book series (SHB)


Stochastic learning automata are probabilistic finite state machines which have been used to model how biological systems can learn. The structure of such a machine can be fixed or can be changing with time. A learning automaton can also be implemented using action (choosing) probability updating rules which may or may not depend on estimates from the environment being investigated. This chapter presents an overview of the field of learning automata, perceived as a completely new paradigm for learning, and explains how it is related to the area of cybernetics.



air traffic management


asynchronous transfer mode


automatic teller machine


discrete estimator algorithm


discretized generalized pursuit algorithm


discrete pursuit algorithm


discrete TSE algorithm


fixed structure stochastic automaton


generalized pursuit algorithm




industrial protocol


integer programming


intellectual property


internet protocol


learning automata


random environment




random-number generator




stochastic estimator learning algorithm


total system error


variable structure stochastic automata


  1. 12.1.
    M.L. Tsetlin: On the behaviour of finite automata in random media, Autom. Remote Control 22, 1210–1219 (1962), Originally in Avtom. Telemekh. 22, 1345–1354 (1961), in RussianzbMATHGoogle Scholar
  2. 12.2.
    M.L. Tsetlin: Automaton Theory and Modeling of Biological Systems (Academic, New York 1973)Google Scholar
  3. 12.3.
    K.S. Narendra, M.A.L. Thathachar: Learning Automata (Prentice-Hall, Upper Saddle River 1989)Google Scholar
  4. 12.4.
    R.R. Bush, F. Mosteller: Stochastic Models for Learning (Wiley, New York 1958)Google Scholar
  5. 12.5.
    C.R. Atkinson, G.H. Bower, E.J. Crowthers: An Introduction to Mathematical Learning Theory (Wiley, New York 1965)zbMATHGoogle Scholar
  6. 12.6.
    V.I. Varshavskii, I.P. Vorontsova: On the behavior of stochastic automata with a variable structure, Autom. Remote Control 24, 327–333 (1963)MathSciNetGoogle Scholar
  7. 12.7.
    M.S. Obaidat, G.I. Papadimitriou, A.S. Pomportsis: Learning automata: theory, paradigms, and applications, IEEE Trans. Syst. Man Cybern. B 32, 706–709 (2002)CrossRefGoogle Scholar
  8. 12.8.
    S. Lakshmivarahan: Learning Algorithms Theory and Applications (Springer, New York 1981)zbMATHGoogle Scholar
  9. 12.9.
    K. Najim, A.S. Poznyak: Learning Automata: Theory and Applications (Pergamon, Oxford 1994)Google Scholar
  10. 12.10.
    A.S. Poznyak, K. Najim: Learning Automata and Stochastic Optimization (Springer, Berlin 1997)zbMATHGoogle Scholar
  11. 12.11.
    M.A.L.T. Thathachar, P.S. Sastry: Networks of Learning Automata: Techniques for Online Stochastic Optimization (Kluwer, Boston 2003)Google Scholar
  12. 12.12.
    S. Misra, B.J. Oommen: GPSPA: a new adaptive algorithm for maintaining shortest path routing trees in stochastic networks, Int. J. Commun. Syst. 17, 963–984 (2004)CrossRefGoogle Scholar
  13. 12.13.
    M.S. Obaidat, G.I. Papadimitriou, A.S. Pomportsis, H.S. Laskaridis: Learning automata-based bus arbitration for shared-medium ATM switches, IEEE Trans. Syst. Man Cybern. B 32, 815–820 (2002)CrossRefGoogle Scholar
  14. 12.14.
    B.J. Oommen, T.D. Roberts: Continuous learning automata solutions to the capacity assignment problem, IEEE Trans. Comput. C 49, 608–620 (2000)CrossRefGoogle Scholar
  15. 12.15.
    G.I. Papadimitriou, A.S. Pomportsis: Learning-automata-based TDMA protocols for broadcast communication systems with bursty traffic, IEEE Commun. Lett. 3(3), 107–109 (2000)CrossRefGoogle Scholar
  16. 12.16.
    A.F. Atlassis, N.H. Loukas, A.V. Vasilakos: The use of learning algorithms in atm networks call admission control problem: a methodology, Comput. Netw. 34, 341–353 (2000)CrossRefGoogle Scholar
  17. 12.17.
    A.F. Atlassis, A.V. Vasilakos: The use of reinforcement learning algorithms in traffic control of high speed networks. In: Advances in Computational Intelligence and Learning (Kluwer, Dordrecht 2002) pp. 353–369Google Scholar
  18. 12.18.
    A. Vasilakos, M.P. Saltouros, A.F. Atlassis, W. Pedrycz: Optimizing QoS routing in hierarchical ATM networks using computational intelligence techniques, IEEE Trans. Syst. Sci. Cybern. C 33, 297–312 (2003)CrossRefGoogle Scholar
  19. 12.19.
    F. Seredynski: Distributed scheduling using simple learning machines, Eur. J. Oper. Res. 107, 401–413 (1998)CrossRefzbMATHGoogle Scholar
  20. 12.20.
    J. Kabudian, M.R. Meybodi, M.M. Homayounpour: Applying continuous action reinforcement learning automata (CARLA) to global training of hidden Markov models, Proc. ITCCʼ04 (Las Vegas 2004) pp. 638–642Google Scholar
  21. 12.21.
    M.R. Meybodi, H. Beigy: New learning automata based algorithms for adaptation of backpropagation algorithm parameters, Int. J. Neural Syst. 12, 45–67 (2002)Google Scholar
  22. 12.22.
    C. Unsal, P. Kachroo, J.S. Bay: Simulation study of multiple intelligent vehicle control using stochastic learning automata, Trans. Soc. Comput. Simul. Int. 14, 193–210 (1997)Google Scholar
  23. 12.23.
    B.J. Oommen, E.V. de St. Croix: Graph partitioning using learning automata, IEEE Trans. Comput. C 45, 195–208 (1995)CrossRefGoogle Scholar
  24. 12.24.
    G. Santharam, P.S. Sastry, M.A.L. Thathachar: Continuous action set learning automata for stochastic optimization, J. Franklin Inst. 331(5), 607–628 (1994)CrossRefMathSciNetGoogle Scholar
  25. 12.25.
    B.J. Oommen, G. Raghunath, B. Kuipers: Parameter learning from stochastic teachers and stochastic compulsive liars, IEEE Trans. Syst. Man Cybern. B 36, 820–836 (2006)CrossRefGoogle Scholar
  26. 12.26.
    V. Krylov: On the stochastic automaton which is asymptotically optimal in random medium, Autom. Remote Control 24, 1114–1116 (1964)Google Scholar
  27. 12.27.
    V.I. Krinsky: An asymptotically optimal automaton with exponential convergence, Biofizika 9, 484–487 (1964)Google Scholar
  28. 12.28.
    M.F. Norman: On linear models with two absorbing barriers, J. Math. Psychol. 5, 225–241 (1968)CrossRefzbMATHMathSciNetGoogle Scholar
  29. 12.29.
    I.J. Shapiro, K.S. Narendra: Use of stochastic automata for parameter self-optimization with multi-modal performance criteria, IEEE Trans. Syst. Sci. Cybern. SSC-5, 352–360 (1969)CrossRefGoogle Scholar
  30. 12.30.
    M.A.L. Thathachar, B.J. Oommen: Discretized reward–inaction learning automata, J. Cybern. Inf. Sci. 2(1), 24–29 (1979)Google Scholar
  31. 12.31.
    J.K. Lanctôt, B.J. Oommen: Discretized estimator learning automata, IEEE Trans. Syst. Man Cybern. 22, 1473–1483 (1992)CrossRefGoogle Scholar
  32. 12.32.
    B.J. Oommen, J.P.R. Christensen: ϵ-optimal discretized linear reward–penalty learning automata, IEEE Trans. Syst. Man Cybern. B 18, 451–457 (1998)MathSciNetGoogle Scholar
  33. 12.33.
    B.J. Oommen, E.R. Hansen: The asymptotic optimality of discretized linear reward–inaction learning automata, IEEE Trans. Syst. Man Cybern. 14, 542–545 (1984)zbMATHMathSciNetGoogle Scholar
  34. 12.34.
    B.J. Oommen: Absorbing and ergodic discretized two action learning automata, IEEE Trans. Syst. Man Cybern. 16, 282–293 (1986)CrossRefzbMATHMathSciNetGoogle Scholar
  35. 12.35.
    P.S. Sastry: Systems of Learning Automata: Estimator Algorithms Applications. Ph.D. Thesis (Department of Electrical Engineering, Indian Institute of Science, Bangalore 1985)Google Scholar
  36. 12.36.
    M.A.L. Thathachar, P.S. Sastry: A new approach to designing reinforcement schemes for learning automata, Proc. IEEE Int. Conf. Cybern. Soc. (Bombay 1984)Google Scholar
  37. 12.37.
    M.A.L. Thathachar, P.S. Sastry: A class of rapidly converging algorithms for learning automata, IEEE Trans. Syst. Man Cybern. 15, 168–175 (1985)zbMATHMathSciNetGoogle Scholar
  38. 12.38.
    M.A.L. Thathachar, P.S. Sastry: Estimator algorithms for learning automata, Proc. Platin. Jubil. Conf. Syst. Signal Process. (Department of Electrical Engineering, Indian Institute of Science, Bangalore 1986)Google Scholar
  39. 12.39.
    M. Agache: Estimator Based Learning Algorithms. MSC Thesis (School of Computer Science, Carleton University, Ottawa 2000)Google Scholar
  40. 12.40.
    M. Agache, B.J. Oommen: Generalized pursuit learning schemes: new families of continuous and discretized learning automata, IEEE Trans. Syst. Man Cybern. B 32(2), 738–749 (2002)CrossRefGoogle Scholar
  41. 12.41.
    M.A.L. Thathachar, P.S. Sastry: Pursuit algorithm for learning automata. Unpublished paper that can be available from the authorsGoogle Scholar
  42. 12.42.
    A.V. Vasilakos, G. Papadimitriou: Ergodic discretize destimator learning automata with high accuracy and high adaptation rate for nonstationary environments, Neurocomputing 4, 181–196 (1992)CrossRefzbMATHGoogle Scholar
  43. 12.43.
    A.F. Atlasis, M.P. Saltouros, A.V. Vasilakos: On the use of a stochastic estimator learning algorithm to the ATM routing problem: a methodology, Proc. IEEE GLOBECOM (1998)Google Scholar
  44. 12.44.
    M.K. Hashem: Learning Automata-Based Intelligent Tutorial-Like Systems. Ph.D. Thesis (School of Computer Science, Carleton University, Ottawa 2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  1. 1.School of Computer ScienceCarleton UniversityOttawaCanada
  2. 2.School of Information TechnologyIndian Institute of TechnologyKharagpurIndia

Personalised recommendations