Abstract
When neurons fire together they wire together. This is Donald Hebb’s famous postulate. However, Hebbian learning is inherently unstable because synaptic weights will self amplify themselves: the more a synapse is able to drive a postsynaptic cell the more the synaptic weight will grow. We present a new biologically realistic way how to stabilise synaptic weights by introducing a third factor which switches on or off learning so that self amplification is minimised. The third factor can be identified by the activity of dopaminergic neurons in VTA which fire when a reward has been encountered. This leads to a new interpretation of the dopamine signal which goes beyond the classical prediction error hypothesis. The model is tested by a real world task where a robot has to find “food disks” in an environment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hebb, D.O.: The organization of behavior: A neurophychological study. Wiley-Interscience, New York (1949)
Kosco, B.: Differential hebbian learning. In: Denker, J.S. (ed.) Neural Networks for computing. AIP conference proceedings, Snowbird, Utah, vol. 151, pp. 277–282. American Institute of Physics, New York (1986)
Porr, B., Wörgötter, F.: Isotropic Sequence Order learning. Neural Comp. 15, 831–864 (2003)
Bailey, C.H., Giustetto, M., Huang, Y.Y., Hawkins, R.D., Kandel, E.R.: Is heterosynaptic modulation essential for stabilizing Hebbian plasticity and memory? Nat. Rev. Neurosci. 1(1), 11–20 (2000)
Grossberg, S., Schmajuk, N.: Neural dynamics of adaptive timing and temporal discrimination during associative learning. Neural Networks 2, 79–102 (1989)
Verschure, P.F.M.J., Voegtlin, T., Douglas, R.J.: Environmentally mediated synergy between perception and behaviour in mobile robots. Nature 425, 620–624 (2003)
Porr, B., Wörgötter, F.: Isotropic sequence order learning in a closed loop behavioural system. Roy. Soc. Phil. Trans. Math., Phys. & Eng. Sciences 361(1811), 2225–2244 (2003)
Ziemke, T.: Are robots embodied? In: First international workshop on epigenetic robotics Modeling Cognitive Development in Robotic Systems, Lund, vol. 85 (2001)
Sutton, R.: Learning to predict by method of temporal differences. Machine Learning 3(1), 9–44 (1988)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2002th edn. Bradford Books, MIT Press (1998)
Markram, H., Lübke, J., Frotscher, M., Sakman, B.: Regulation of synaptic efficacy by coincidence of postsynaptic aps and epsps. Science 275, 213–215 (1997)
Saudargiene, A., Porr, B., Wörgötter, F.: How the shape of pre- and postsynaptic signals can influence stdp: A biophysical model. Neural Comp. 16, 595–626 (2004)
Centonze, D., Picconi, B., Gubellini, P., Bernardi, G., Calabresi, P.: Dopaminergic control of synaptic plasticity in the dorsal striatum. Eur. J. Neurosci. 13(6), 1071–1077 (2001)
Reynolds, J.N., Wickens, J.R.: Dopamine dependent plasticity of corticostriatal synapses. Neural Networks 15, 507–521 (2002)
Wörgötter, F., Porr, B.: Temporal sequence learning, prediction and control - a review of different models and their relation to biological mechanisms. Neural Comp. 17, 245–319 (2005)
Zhang, L.I., Tao, H.W., Holt, C.E., Harris, W.A., Poo, M.: A critical window for cooperation and competition among developing retinotectal synapses. Nature 395, 37–44 (1998)
Schultz, W.: Dopamine neurons and their role in reward mechanisms. Curr. Opin. Neurobiol. 7(2), 191–197 (1997)
Prescott, T.J., González, F.M.M., Gurney, K., Humpries, M.D., Redgrave, P.: A robot model of the basal ganglia: Behaviour and intrinsic processing. Neural Networks (in press, 2006)
Dayan, P., Balleine, B.W.: Reward, motivation and reinforcement learning. Neuron 36, 285–298 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Thompson, A.M., Porr, B., Wörgötter, F. (2006). Stabilising Hebbian Learning with a Third Factor in a Food Retrieval Task. In: Nolfi, S., et al. From Animals to Animats 9. SAB 2006. Lecture Notes in Computer Science(), vol 4095. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11840541_26
Download citation
DOI: https://doi.org/10.1007/11840541_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-38608-7
Online ISBN: 978-3-540-38615-5
eBook Packages: Computer ScienceComputer Science (R0)