Advertisement

Time Perception and Reinforcement Learning — a Neural Network Model of Animal Experiments

  • J. L. Shapiro
  • John Wearden

Abstract

Animal data reveals that animals can time intervals associated with rewards. The data shows a striking property — the data for different time intervals collapses into a single curve when the data is scaled by the time interval. This is called the scalar property of interval timing. Here a simple model of a neural clock is presented and shown to give rise to the scalar property. The model is an accumulator consisting of noisy, linear spiking neurons. It is analytically tractable and contains only three parameters. When coupled with simple reinforcement learning, it simulates experiments on estimation of time duration in animals.

Keywords

Scalar Property Internal Clock Computational Neuroscience Bradford Book Spatial Node 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    R. Church and H. Broadbent. Alternative representations of time, number, and rate. Cognition, 37:55–81, 1990.CrossRefGoogle Scholar
  2. [2]
    John E. Desmond. Temporally adaptive responses in neural models: The stimulus trace. In Michael Gabriel and John Moore, editors, Learning and Computational Neuroscience: Foundations of Adaptive Networks, A Bradford Book, pages 421–456. The MIT Press, 1990.Google Scholar
  3. [3]
    John Gibbon and Russell M. Church. Representation of time. Cognition, 37:23–54, 1990.CrossRefGoogle Scholar
  4. [4]
    Stephen Grossberg and John W. L. Merrill. A neural network model of adaptively timed reinforcement learning and hippocampal dynamics. Cognitive Brain Research, 1:3–38, 1992.CrossRefGoogle Scholar
  5. [5]
    S. C. Hinton and W. H. Meck. How time flies: Functional and neural mechansims of interval timing. In C. M. Bradshaw and E. Szadabi, editors, Time and Behaviour: Psychological and Neurobehavioural Analyses. Amsterdam: Elsevier Science, 1997.Google Scholar
  6. [6]
    J. W. Moore, J. E. Desmond, and N. E. Berthier. Adaptively timed conditioned responses and the cerebellum: A neural network approach. Biological Cybernetics, 62:17–28, 1989.CrossRefGoogle Scholar
  7. [7]
    John W. Moore, Neil D. Berthier, and Diana E. J. Blazis. Classical eye-blink conditioning: Brain systems and implementation of a computational model. In Michael Gabriel and John Moore, editors, Learning and Computational Neuroscience: Foundations of Adaptive Networks, A Bradford Book, pages 359–387. The MIT Press, 1990.Google Scholar
  8. [8]
    Jonathan Shapiro, John Wearden, and Rossano Barrone. Simple models of scalar timing. In preparation.Google Scholar
  9. [9]
    Richard S. Sutton and Andrew G. Barto. Time-derivative models of pavlovian reinforcement. In Michael Gabriel and John Moore, editors, Learning and Computational Neuroscience: Foundations of Adaptive Networks, A Bradford Book, pages 497–537. The MIT Press, 1990.Google Scholar
  10. [10]
    Richard S. Sutton and Andrew G. Barto. Reinforcment Learning: An Introduction. A Bradford Book. The MIT Press, 1998.Google Scholar

Copyright information

© Springer-Verlag Wien 2001

Authors and Affiliations

  • J. L. Shapiro
  • John Wearden
    • 1
  1. 1.Manchester UniversityManchesterUK

Personalised recommendations