Abstract
Three factor Isotropic sequence order (ISO3) learning is a form of differential Hebbian learning where a third factor switches on learning at relevant moments for example, after reward retreival. This switch enables learning only at specific moments and, thus, stablises the corresponding weights. The concept of using a third factor as a gating signal for learning at relevant moments has been extended in this paper to perform second order conditioning (SOC). We present a biological model of the sub-cortical nuclei of the limbic system that is capable of performing SOC in a food seeking task. The 3rd-factor is modelled by dopaminergic neurons of the VTA which are activated via a direct excitatory glutamatergic pathway, and an indirect dis-inhibitory GABAergic pathway. The latter generates an amplification in the number of tonically active DA neurons. This produces an increase in DA outside the event of a primary reward and enables SOC to be accomplished.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Robbins, T.W., Everitt, B.J.: Neurobehavioural mechanisms of reward and motivation. Curr. Opin. Neurobiol. 6(2), 228–236 (1996)
Schultz, W.: Dopamine neurons and their role in reward mechanisms. Curr. Opin. Neurobiol. 7(2), 191–197 (1997)
Sutton, R.S., Barto, A.: A temporal-difference model of classical conditioning. In: Proceedings of the Ninth Annual Conference of the Cognitive Science Society, Seattle, Washington, July 1987, pp. 355–378 (1987)
Porr, B., Wörgötter, F.: Learning with ”relevance”: Using a third factor to stabilise hebbian learning. Neural Computation (in press, 2007)
Thompson, A.M., Porr, B., Wörgötter, F.: Stabilising hebbian learning with a third factor in a food retrieval task. In: Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J.C.T., Marocco, D., Meyer, J.A., Miglino, O., Parisi, D. (eds.) SAB 2006. LNCS (LNAI), vol. 4095, pp. 313–322. Springer, Heidelberg (2006)
Floresco, S.B., West, A.R., Ash, B., Moore, H., Grace, A.A.: Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nat. Neurosci. 6(9), 968–973 (2003)
Porr, B., Wörgötter, F.: Isotropic Sequence Order learning. Neural Comp. 15, 831–864 (2003)
Zahm, D.: An integrative neuroanatomical perspective on some subcortical substrates of adaptive responding with emphasis on the nucleus accumbens. Neurosci. Biobehav Rev. 24(1), 85–105 (2000)
Everitt, B.J., Robbins, T.W.: Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat. Neurosci. 8(11), 1481–1489 (2005)
Nakamura, K., Ono, T.: Lateral hypothalamus neuron involvement in integration of natural and artificial rewards and cue signals. J. Neurophysiol. 55(1), 163–181 (1986)
Maldonado-Irizarry, C., Swanson, C., Kelley, A.: Glutamate receptors in the nucleus accumbens shell control feeding behavior via the lateral hypothalamus. J. Neurosci. 15(10), 6779–6788 (1995)
Pribram, K.H., Mishkin, M., Rosvold, H.E., Kaplan, S.J.: Effects on delayed-response performance of lesions of dorsolateral and ventromedial frontal cortex of baboons. J Comp. Physiol. Psychol. 45(6), 565–575 (1952)
Lisman, J., Grace, A.: The Hippocampal-VTA loop: Controlling the entry of information into the long term memory. Neuron 46, 703–713 (2005)
Prescott, T.J., Gonzalez, F.M.M., Gurney, K., D., H.M., Redgrave, P.: A robot model of the basal ganglia: Behavior and intrinsic processing. Neural Networks 19(1), 31–61 (2006)
Kötter, R., Wickens, J.: Interactions of glutamate and dopamine in a computational model of the striatum. J. Comput. Neurosci. 2(3), 195–214 (1995)
O’Reilly, R., Frank, M., Hazy, T., Watz, B.: The primary value and learned value pavlovian learning algorithm. Behavioral Neuroscience 121(1), 31–49 (2007)
Brown, J., Bullock, D., Grossberg, S.: How the basal ganglia use parallel exitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J Neurosci. 19(23), 10502–10511 (1999)
Kelley, A.: Ventral striatal control of appetitive motivation: Role in ingestive behavior and reward-related learning. Neurosci Biobehav Rev. 27(8), 765–776 (2004)
Zahm, D.S., Heimer, L.: Two transpallidal pathways originating in the rat nucleus accumbens. J Comp. Neurol. 302(3), 437–446 (1990)
Cragg, S.: Meaningful silences: How dopamine listens to the ACh pause. Trends in Neuroscience 29(3), 125–131 (2006)
Daw, N.D., Doya, K.: The computational neurobiology of learning and reward. Curr. Opin. Neurobiol. 16(2), 199–204 (2006)
Daw, N.D., Kakade, S., Dayan, P.: Opponent interactions between serotonin and dopamine. Neural Netw. 15(4-6), 603–616 (2002)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Thompson, A.M., Porr, B., Kolodziejski, C., Wörgötter, F. (2008). Second Order Conditioning in the Sub-cortical Nuclei of the Limbic System. In: Asada, M., Hallam, J.C.T., Meyer, JA., Tani, J. (eds) From Animals to Animats 10. SAB 2008. Lecture Notes in Computer Science(), vol 5040. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69134-1_19
Download citation
DOI: https://doi.org/10.1007/978-3-540-69134-1_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69133-4
Online ISBN: 978-3-540-69134-1
eBook Packages: Computer ScienceComputer Science (R0)