A mathematical framework for learning and adaption: (Generalized) random systems with complete connections

  • Ulrich Herkenrath
  • Radu Theodorescu


The aim of this paper is to show that the theory of (generalized) random systems with complete connections may serve as a mathematical framework for learning and adaption. Chapter 1 is of an introductory nature and gives a general description of the problems with which one is faced. In Chapter 2 the mathematical model and some results about it are explained. Chapter 3 deals with special learning and adaption models.

Key words and phrases

Random systems with complete connections random automata random environment expediency optimality two-armed bandit problem 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Arrow K., Hahn F.:General competitive analysis. Holden Day, San Francisco 1972.Google Scholar
  2. [2]
    Brock W. A., Mirman L. J.:Optimal economic growth and uncertainty: The discounted case. J. Econom. Theory 4 (1972), 479–513.CrossRefMathSciNetGoogle Scholar
  3. [3]
    Bush R. R., Mosteller F.:Stochastic models for learning. Wiley, New York 1955.MATHGoogle Scholar
  4. [4]
    Doeblin W., Fortet R.:Sur les chaines à liaisons complètes. Bull. Soc. Math. France 65 (1937), 132–148.MathSciNetGoogle Scholar
  5. [5]
    Fu K.S.:Learning control systems. Review and outlook. IEEE Trans. Automatic Control 15 (1970), 210–221.CrossRefMathSciNetGoogle Scholar
  6. [6]
    Green J. R., Majumdar M.:The nature of stochastic equilibria. Econometrica 43 (1975), 647–660.MATHCrossRefMathSciNetGoogle Scholar
  7. [7]
    Herkenranth U.:Der assoziierte Markov Prozess verallgemeinerter zufälliger Systeme mit vollständigen Bindungen. Ph. D. Thesis, University of Bonn, Dept. Applied Math., Bonn, 1976.Google Scholar
  8. [8]
    Herkenranth U.:The associated Markov process of generalized random systems with complete connections. Rev. Roumaine. Math. Pures Appl. (1979), 24, 243–254.MathSciNetGoogle Scholar
  9. [9]
    Herkenrath U.:Modelle für die Interaktion eines stochastischen Automaten mit einer zufälligen Umwelt. Z. Angew. Math. Mech. (1978) (to appear).Google Scholar
  10. [10]
    Herkenrath U., Theodorescu R.:General control systems. Information Sci. 14 (1978) 57–73.MATHCrossRefMathSciNetGoogle Scholar
  11. [11]
    Herkenranth U., Theodorescu R.:Expediency and optimality for general control systems. Coll. Internat. C.N.R.S., no 276, Theorie de l’information, Cachan, July 4–8, 1977. Editions du C.N.R.S., Paris, 1978, 449–455.Google Scholar
  12. [12]
    Herkenrath U., Theodorescu R.:On certain aspects of the two-armed bandit problem. Elektron. Informationsverarbeit. Kybermetik, 1978, 14, 527–535.MATHMathSciNetGoogle Scholar
  13. [13]
    Herkenrath U., Theodorescu R.:Ona stochastic approximation procedure applied to the bandit problem. Elektron, Informationsverarbeit. Kybernetik, 1979, 15, 301–307.MATHMathSciNetGoogle Scholar
  14. [14]
    Hinderer K.:Foundations of nonstationary dynamic programming with discrete time parameter. Springer, Berlin 1970.Google Scholar
  15. [15]
    Iosifescu M.:Random systems with complete connections with an arbitrary set of states. Rev. Roumaine Math. Pures Appl. 8 (1963), 611–645; 9 (1964), 91–92.MathSciNetGoogle Scholar
  16. [16]
    Iosifescu M., Theodorescu R.:On the Bush-Mosteller stochastic models for learning. Math. Psychology 2 (1965), 196–203.CrossRefGoogle Scholar
  17. [17]
    Iosifescu M., Theodorescu R.:Random processes and learning. Springer, New York 1969.MATHGoogle Scholar
  18. [18]
    LeCalvé G., Theodorescu R.:Systèmes aléatoires à liaisons complètes totalelement non-homogènes. C.R. Acad. Sci, Paris Ser. A 265 (1967), 347–349.MathSciNetGoogle Scholar
  19. [19]
    LeCalvé G., Theodorescu R.Systèmes aléatoires généralisés à liaisons complètes. Z. Wahrscheinlichkeitstheorie Verw. Gebiete 19 (1971), 19–28.CrossRefGoogle Scholar
  20. [20]
    Mirman L. J.:The steady state behavior of a class of one sector growth models with incertain technology. J. Econom. Theory 6 (1973), 219–242.CrossRefMathSciNetGoogle Scholar
  21. [21]
    Narendra K. S., Thathachar M.A.L.:Learning automata- a survey. IEEE Trans Systems, Man, Cybernet, 4 (1974), 323–334.MATHMathSciNetGoogle Scholar
  22. [22]
    Norman M. F.:Markov processes and learning models. Academic Press, New York 1972.MATHGoogle Scholar
  23. [23]
    Pruscha H.:Die Statistische Analyse von ergodischen Ketten mit vollständigen Bindungen. Ph. D. Thesis, University of Munich, Dept. Math, Munich 1974.Google Scholar
  24. [24]
    Onicescu O., Mihoc G.:Sur les chaines variables statistiques. Bull. Sci. Math. 59 (1935), 174–192.MATHGoogle Scholar
  25. [25]
    Pruscha H., Theodorescu R.:Functions of event variables of a random system with complete connections. J. Multivariate Anal 7 (1977), 336–362.MATHCrossRefMathSciNetGoogle Scholar
  26. [26]
    Pruscha H., Theodorescu R.:Recurrence and transience for discrete parameter stochastic processes. Trans. Eighth Prague Conference on Information Theory, Statistical Decision Functions, and Random processes, Prague, Aug. 28–Sept. 1. 1978, Academia, Prague, 1978, 97–105.Google Scholar
  27. [27]
    Pruscha H., Theodorescu R.:On a non-Markovian model with linear transition rule. Coll. Math., (in print).Google Scholar
  28. [28]
    Rhenius D.:Markoffsche Entscheidungsmodelle mit unvollständiger Information und Anwendungen in der Lerntheorie. Ph. D. Thesis, University of Hamburg, Dept. of Stochastics, Hamburg, 1971.Google Scholar
  29. [29]
    Rothschild M.:A two-armed bandit theory of market princing. J. Econom. Theory 9 (1974), 185–202.CrossRefMathSciNetGoogle Scholar
  30. [30]
    Simon J. C.:Computer oriented learning processes. Noordhoff, Leyden 1976.MATHGoogle Scholar

Copyright information

© Springer 1981

Authors and Affiliations

  • Ulrich Herkenrath
    • 1
  • Radu Theodorescu
    • 2
  1. 1.Institute of Applied MathematicsUniversity of BonnBonnFederal Republic of Germany
  2. 2.Department of MathematicsLaval UniversityQuebecCanada

Personalised recommendations