Skip to main content
SpringerLink
Log in
Book cover

Anticipatory Behavior in Adaptive Learning Systems pp 66–85Cite as

Towards a Four Factor Theory of Anticipatory Learning

  • Chapter

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2684))

Abstract

This paper takes an overtly anticipatory stance to the understanding of animat learning and behavior. It analyses four major animal learning theories and attempts to identify the anticipatory and predictive elements inherent to them, and to provide a new unifying approach based on the anticipatory nature of those elements based on five simple predictive ”rules”. These rules encapsulate all the principal properties of the four diverse theories (the four factors) and provide a simple framework for understanding how an individual animat may appear to operate according to different principles under varying circumstances. The paper then indicates how these anticipatory principles can be used to define a more detailed set of postulates for the Dynamic Expectancy Model of animat learning and behavior, and to construct its computer implementation SRS/E. Some of the issues discussed are illustrated with an example experimental procedure using SRS/E

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Agre, P.E.: Computational Research on Interaction and Agency. Artificial Intelligence 72, 1–52 (1995)

    Article  Google Scholar 

  2. Baird, L.C., Klopf, A.H.: Extensions to the Associative Control Process (ACP) Network: Hierarchies and Provable Optimality. In: 2nd Int. Conf. on Simulation of Adaptive Behavior (SAB-2), pp. 163–171 (1993)

    Google Scholar 

  3. Balkenius, C.: Natural Intelligence in Artificial Creatures, Lund University Cognitive Studies 37 (1995)

    Google Scholar 

  4. Balkenius, C., Morén, J.: Computational Models of Classical Conditioning: A Comparative Study. In: 5th Int. Conf. on Simulation of Adaptive Behavior (SAB-5), pp. 348–353 (1998)

    Google Scholar 

  5. Barto, A.G., Sutton, R.S.: Simulation of Anticipatory Responses in Classical Conditioning by a Neuron-like Adaptive Element. Behavioral Brain Research 4, 221–235 (1982)

    Article  Google Scholar 

  6. Bower, G.H., Hilgard, E.R.: Theories of Learning, 5th edn. Prentice Hall Inc., Englewood Cliffs (1981)

    Google Scholar 

  7. Brooks, R.A.: Intelligence Without Reason, MIT AI Laboratory, A.I. Memo No. 1293 (Prepared for Computers and Thought). In: IJCAI 1991 (April 1991) (preprint)

    Google Scholar 

  8. Bryson, J.: Hierarchy and Sequence vs. Full Parallelism in Action Selection. In: 6th Int. Conf. on Simulation of Adaptive Behavior (SAB-6), pp. 147–156 (2000)

    Google Scholar 

  9. Butz, M.V., Sigaud, O., Gerard, P.: Internal Models and Anticipations in Adaptive Learning Systems. In: Adaptive Behavior in Anticipatory Learning Systems 2002 Workshop (ABiALS 2002), p. 23 (2002)

    Google Scholar 

  10. Catania, A.C.: The Operant Behaviorism of B.F. Skinner. In: Catania, A.C., Harnad, S. (eds.) The Selection of Behavior, pp. 3–8. Cambridge University Press, Cambridge (1988)

    Google Scholar 

  11. Maes, P.: Behavior-based Artificial Intelligence. In: 2nd Int. Conf. on Simulation of Adaptive Behavior (SAB-2), pp. 2–10 (1993)

    Google Scholar 

  12. Mowrer, O.H.: Two-factor Learning Theory Reconsidered, with Special Reference to Secondary Reinforcement and the Concept of Habit. Psychological Review 63, 114–128 (1956)

    Article  Google Scholar 

  13. Rosenblatt, J.K., Payton, D.W.: A Fine-Grained Alternative to the Subsumption Architecture for Mobile Robot Control. In: IEEE/INNS Int. Joint Conf. on Neural Networks, vol. II, pp. 317–323 (1989)

    Google Scholar 

  14. Saksida, L.M., Raymond, S.M., Touretzky, D.S.: Shaping Robot Behavior Using Principles from Instrumental Conditioning. Robotics and Autonomous Systems 22-3/4, 231–249 (1997)

    Google Scholar 

  15. Schaffer, S.: Babbage.s Intelligence: Calculating Engines and the Factory System (1998), hosted at http://cci.wmin.ac.uk/schaffer/schaffer01.html

  16. Schmajuk, N.A.: Behavioral Dynamics of Escape and Avoidance: A Neural Network Approach. In: 3rd Int. Conf. on Simulation of Adaptive Behavior (SAB-3), pp. 118-127 (1994)

    Google Scholar 

  17. Shettleworth, S.J.: Reinforcement and the Organization of Behavior in Golden Hamsters: Hunger, Environment, and Food Reinforcement. Journal of Experimental Psychology: Animal Behavior Processes 104-1, 56–87 (1975)

    Google Scholar 

  18. Stolzmann, W., Butz, M.V., Hoffmann, J., Goldberg, D.E.: First Cognitive Capabilities in the Anticipatory Classifier System. In: 6th Int. Conf. on Simulation of Adaptive Behavior (SAB-6), pp. 287-296 (2000)

    Google Scholar 

  19. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    Google Scholar 

  20. Tani, J., Nolfi, S.: Learning to Perceive the World as Articulated: An Approach for Hierarchical Learning in Sensory-Motor Systems. In: 5th Int. Conf. on Simulation of Adaptive Behavior (SAB-5), pp. 270–279 (1998)

    Google Scholar 

  21. Thistlethwaite, D.: A Critical Review of Latent Learning and Related Experiments. Psychological Bulletin 48(2), 97–129 (1951)

    Article  Google Scholar 

  22. Thorndike, E.L.: Animal Intelligence: An Experimental Study of the Associative Processes in Animals. Psychol. Rev., Monogr. Suppl., 2-8 (1898)

    Google Scholar 

  23. Tolman, E.C.: Purposive Behavior in Animals and Men. The Century Co., New York (1932)

    Google Scholar 

  24. Tyrrell, T.: Computational Mechanisms for Action Selection, University of Edinburgh, Ph.D. thesis (1993)

    Google Scholar 

  25. Witkowski, M.: Dynamic Expectancy: An Approach to Behaviour Shaping Using a New Method of Reinforcement Learning. In: 6th Int. Symp. on Intelligent Robotic Systems, pp. 73–81 (1998)

    Google Scholar 

  26. Witkowski, M.: Integrating Unsupervised Learning, Motivation and Action Selection in an A-life Agent. In: Floreano, D., Mondada, F. (eds.) ECAL 1999. LNCS, vol. 1674, pp. 355–364. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  27. Witkowski, M.: The Role of Behavioral Extinction in Animat Action Selection. In: 6th Int. Conf. on Simulation of Adaptive Behavior (SAB-6), pp. 177–186 (2000)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Interactive and Intelligent Systems Section, Department of Electrical & Electronic Engineering, Imperial College, Exhibition Road, London, SW7 2BT, United Kingdom

    Mark Witkowski

Authors
  1. Mark Witkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Psychology, University of Würzburg, Röntgenring 11, 97070, Würzburg, Germany

    Martin V. Butz

  2. Animat Lab, University Paris VI, 104 Av du Président Kennedy, 75016, Paris, France

    Olivier Sigaud

  3. ADAge, LIPN, Univ. de Paris-Nord, 93 430, Villetaneuse, France

    Pierre Gérard

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Witkowski, M. (2003). Towards a Four Factor Theory of Anticipatory Learning. In: Butz, M.V., Sigaud, O., Gérard, P. (eds) Anticipatory Behavior in Adaptive Learning Systems. Lecture Notes in Computer Science(), vol 2684. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45002-3_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-45002-3_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-40429-3

  • Online ISBN: 978-3-540-45002-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation