Dopamine and Norepinephrine Modulation of Cortical and Subcortical Dynamics During Visuomotor Learning

  • Hernán G. Rey
  • Sergio E. Lew
  • B. Silvano Zanutto

Understanding the mechanisms of neuromodulation exerted by catecholamines over cortical and subcortical neurons has been one of the major challenges in neuroscience that remains unsolved. Although there is compelling neurophysiological evidence indicating that both dopamine (DA) and norepinephrine (NE) systems are critically involved in the control of neuronal functions, there are still many discrepancies on how these modulations occur, perhaps as result of the different experimental approaches used to investigate the effects. Of particular interest, experiments performed in behaving animals have provided significant amount of information about how these modulatory systems may regulate complex neural networks such as during behavioral conditions in which both excitatory and inhibitory neurons are engaged. In this regard, computational models could provide a better understanding on the neuronal dynamics underlying these interactions by formalizing biologically plausible hypotheses and predictions to be tested in realistic experimental conditions. Two main approaches have been successfully implemented to tackle this issue. The first one is bottom-up, that is, brain functions are modeled from neurophysiological bases, simulating the properties of neurons and synapses as close as possible. The other approach is top-down, that is, the behavioral and brain functions are simulated as close as possible to animal behavior. Unfortunately, both approaches suffer from the lack of experimental data to link neurons and higher brain functions. Because of this, it seems obvious to choose a position in between both approaches to start the modeling work. Behavioral results constrain the modeling process, giving top-down cues in the development of a model that incorporates molecular, physiological, and behavioral evidence parsimoniously. In this chapter, we will present and discuss how neural network models that take into account realistic biological data could be used to explain the roles of DA and NE underlying learning processes involved in simple and complex tasks performances.


Conditioned Stimulus Unconditioned Stimulus Ventral Tegmental Area Locus Coeruleus Comparison Stimulus 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aston-Jones, G. and Cohen, J.D. (2005) An integrative theory of locus coeruleusnorpine-phrine function: Adaptive gain and optimal performance. Annual Review of Neu-roscience, 28, 403-450.CrossRefGoogle Scholar
  2. Atallah, H.E., Frank, M.J. and O’Reilly, R.C. (2004) Hippocampus, cortex, and basal ganglia: insights from computational models of complementary learning systems. Neurobiology of Learning and Memory, 82(3), 253-267.CrossRefPubMedGoogle Scholar
  3. Berridge, C.W. and Waterhouse, B.D. (2003) The locus coeruleus-noradrenergic system: modulation of behavioral state and state-dependent cognitive processes. Brain Research Reviews, 42(1), 33-84.CrossRefPubMedGoogle Scholar
  4. Brasted, P.J. and Wise, S.P. (2004) Comparison of learning-related neuronal activity in the dorsal premotor cortex and striatum. European Journal of Neuroscience, 19(3), 721-740.CrossRefPubMedGoogle Scholar
  5. Cavada, C., Company, T., Tejedor, J., Cruz-Rizzolo, R.J. and Reinoso-Suarez, F. (2000) The anatomical connections of the macaque monkey orbitofrontal cortex. A review. Cerebral Cortex, 10(3), 220-242.CrossRefPubMedGoogle Scholar
  6. Clayton, E.C., Rajkowski, J., Cohen, J.D. and Aston-Jones, G. (2004) Phasic activation of monkey locus coeruleus by simple decisions in a forced-choice task. Journal of Neuro-science, 24(44), 9914-9920.Google Scholar
  7. Durstewitz, D., Seamans, J.K. and Sejnowski, T.J. (2000) Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. Journal of Neurophysiology, 83 (3), 1733-1750.PubMedGoogle Scholar
  8. Freedman, D.J., Riesenhuber, M., Poggio, T. and Miller, E.K. (2002) Visual categorization and the primate prefrontal cortex: neurophysiology and behavior. Journal of Neurophysi-ology, 88(2), 929-941.Google Scholar
  9. Fuster, J.M. (1989) The prefrontal cortex: anatomy, physiology, and neuropsychology of the frontal lobe. Raven Press New York, NY.Google Scholar
  10. Garris, P.A. and Wightman, R.M. (1994) Different kinetics govern dopaminergic transmission in the amygdala, prefrontal cortex, and striatum: an in vivo voltammetric study. Journal of Neuroscience, 14(1), 442-450.PubMedGoogle Scholar
  11. Gisiger, T. and Kerszberg, M. (2006). A model for integrating elementary neural functions into delayed-response behavior. PLoS Computational Biology, 2(4), e25, 211-227.CrossRefGoogle Scholar
  12. Gonon, F. (1997) Prolonged and extrasynaptic excitatory action of dopamine mediated by D1 receptors in the rat striatum in vivo. Journal of Neuroscience, 17(15), 5972-5978.PubMedGoogle Scholar
  13. Grenhoff, J., Nisell, M., Ferre, S., Aston-Jones, G. and Svensson, T.H. (1993) Noradrenergic modulation of midbrain dopamine cell firing elicited by stimulation of the locus coeruleus in the rat. Journal of Neural Transmission. General Section, 93(1), 11-25.CrossRefPubMedGoogle Scholar
  14. Gutnisky, D.A. and Zanutto, B.S. (2004a) Cooperation in the iterated prisoner’s dilemma is learned by operant conditioning mechanisms. Artificial Life, 10(4), 433-461.CrossRefPubMedGoogle Scholar
  15. Gutnisky, D.A. and Zanutto, B.S. (2004b) Learning obstacle avoidance with an operant behav-ior model. Artificial Life, 10(1), 65-81.CrossRefPubMedGoogle Scholar
  16. Lewis, B.L. and O’Donnell, P. (2000) Ventral tegmental area afferents to the prefrontal cortex maintain membrane potential ‘up’ states in pyramidal neurons via D1 dopamine receptors. Cerebral Cortex, 10, 1168-1175.CrossRefPubMedGoogle Scholar
  17. McClure, S.M., Gilzenrat, M.S. and Cohen, J.D. (2005) An exploration-exploitation model based on norepinephrine and dopamine activity. Advances in Neural Information Process-ing Systems, 18, 867-874.Google Scholar
  18. Mink, J.W. (1996) The basal ganglia: focused selection and inhibition of competing motor programs. Progress in Neurobiology, 50(4), 381-425.CrossRefPubMedGoogle Scholar
  19. O’Reilly, R.C., Noelle, D.C., Braver, T.S. and Cohen, J.D. (2002) Prefrontal cortex and dy-namic categorization tasks: representational organization and neuromodulatory control. Cerebral Cortex, 12(3), 246-257.CrossRefPubMedGoogle Scholar
  20. Otani, S., Daniel, H., Roisin, M.P. and Crepel, F. (2003) Dopaminergic modulation of long-term synaptic plasticity in rat prefrontal neurons. Cerebral Cortex, 13(11), 1251-1256.CrossRefPubMedGoogle Scholar
  21. Paladini, C.A. and Williams, J.T. (2004) Noradrenergic inhibition of midbrain dopamine neurons. Journal of Neuroscience, 24(19), 4568-4575.CrossRefPubMedGoogle Scholar
  22. Pan, W.X., Schmidt, R. and Wickens, J.R. and Hyland, B.I. (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. Journal of Neuroscience, 25(26), 6235-6242.CrossRefPubMedGoogle Scholar
  23. Pasupathy, A. and Miller, E.K. (2005) Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature, 433(7028), 873-876.CrossRefPubMedGoogle Scholar
  24. Petrides, M. (2000) Dissociable roles of mid-dorsolateral prefrontal and anterior inferotempo-ral cortex in visual working memory. Journal of Neuroscience, 20(19), 7496-7503.PubMedGoogle Scholar
  25. Rescorla, R.A. and Wagner, A.R. (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: A. H. Black and W. F. Prokasy (Eds), Classical Conditioning II: Current Research and Theory. Appleton Century Crofts, New York, pp. 64-99.Google Scholar
  26. Reynolds, J.N.J. and Wickens, J.R. (2002) Dopamine-dependent plasticity of corticostriatal synapses. Neural Networks, 15(4-6), 507-521.CrossRefPubMedGoogle Scholar
  27. Roychowdhury, V.P., Kai-Yeung Siu, and Kailath, T. (1995) Classification of linearly non-separable patterns by linear threshold elements. IEEE Transactions on Neural Networks, 6 (2), 318-331.CrossRefPubMedGoogle Scholar
  28. Schmajuk, N.A. and Zanutto, B.S. (1997) Escape, avoidance, and imitation: a neural network approach. Adaptive Behavior, 6(1), 63-129.CrossRefGoogle Scholar
  29. Schultz, W., Dayan, P. and Montague, P.R (1997). A neural substrate of prediction and re-ward. Science, 275 (5306), 1593-1599.CrossRefPubMedGoogle Scholar
  30. Schultz, W. (2002) Getting formal with dopamine and reward. Neuron, 36(2), 241-263.CrossRefPubMedGoogle Scholar
  31. Tseng, K.Y., Mallet, N., Toreson, K., Le Moine, C., Gonon, F. and O’Donnell, P. (2006) Excitatory response of prefrontal cortical fast-spiking interneurons to ventral tegmental area stimulation in vivo. Synapse, 59, 412-417.CrossRefPubMedGoogle Scholar
  32. Tseng, K.Y. and O’Donnell, P. (2005) Post-pubertal emergence of prefrontal cortical up states induced by D1-NMDA co-activation. Cerebral Cortex, 15, 49-57.CrossRefPubMedGoogle Scholar
  33. Usher, M., Cohen, J.D., Servan-Schreiber, D., Rajkowski, J. and Aston-Jones, G. (1999) The role of locus coeruleus in the regulation of cognitive performance. Science, 283(5401), 549-554.CrossRefPubMedGoogle Scholar
  34. Wang, J. and O’Donnell, P. (2001) D(1) dopamine receptors potentiate NMDA-mediated excitability increase in layer V prefrontal cortical pyramidal neurons. Cerebral Cortex, 11 (5), 452-462.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Hernán G. Rey
    • 1
  • Sergio E. Lew
    • 1
  • B. Silvano Zanutto
    • 1
  1. 1.Instituto de Ingeniería Biomedica, Facultad de IngenieríaUniversidad de Buenos Aires and Instituto de Biologia y Medicina Experimental—CONICETBuenos Aires

Personalised recommendations