Abstract
Prefrontal cortex (PFC) has been implicated in the ability to switch behavioral strategies in response to changes in reward contingencies. A recent experimental study has shown that separate subpopulations of neurons in the prefrontal cortex were activated when rats switched between allocentric place strategies and egocentric response strategies in the plus maze. In this paper we propose a simple neural-network model of strategy switching, in which the learning of the two strategies as well as learning to select between those strategies is governed by the same temporal-difference (TD) learning algorithm. We show that the model reproduces the experimental data on both behavioral and neural levels. On the basis of our results we derive testable prediction concerning a spatial dynamics of the phasic dopamine signal in the PFC, which is thought to encode reward-prediction error in the TD-learning theory.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Arleo, A., Rondi-Reig, L.: Multimodal sensory integration and concurrent navigation strategies for spatial cognition in real and artificial organisms. J. Integr. Neurosci. 6, 327–366 (2007)
White, N.M., McDonald, R.J.: Multiple parallel memory systems in the brain of the rat. Neurobiol. Learn. Mem. 77, 125–184 (2002)
Packard, M.G., Hirsh, R., White, N.M.: Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: evidence for multiple memory systems. J. Neurosci. 9, 1465–1472 (1989)
Packard, M.G., McGaugh, J.L.: Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning. Neurobiol. Learn. Mem. 65, 65–72 (1996)
Chang, Q., Gold, P.E.: Intra-hippocampal lidocaine injections impair acquisition of a place task and facilitate acquisition of a response task in rats. Behav. Brain. Res. 144, 19–24 (2003)
Ragozzino, M.E., Detrick, S., Kesner, R.P.: Involvement of the Prelimbic-Infralimbic areas of the rodent prefrontal cortex in behavioral flexibility for place and response learning. J. Neurosci. 19, 4585–4594 (1999)
Rich, E.L., Shapiro, M.L.: Prelimbic/infralimbic inactivation impairs memory for multiple task switches, but not flexible selection of familiar tasks. J. Neurosci. 27, 4747–4755 (2007)
Rich, E.L., Shapiro, M.: Rat prefrontal cortical neurons selectively code strategy switches. J. Neurosci. 29, 7208–7219 (2009)
Miller, E.K., Cohen, J.D.: An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001)
Sutton, R., Barto, A.G.: Reinforcement Learning - An Introduction. MIT Press, Cambridge (1998)
Schultz, W., Dayan, P., Montague, P.R.: A neural substrate of prediction and reward. Science 275, 1593–1599 (1997)
Arleo, A., Gerstner, W.: Spatial cognition and neuro-mimetic navigation: A model of hippocampal place cell activity. Biol. Cybern. 83, 287–299 (2000)
Sheynikhovich, D., Chavarriaga, R., Strösslin, T., Arleo, A., Gerstner, W.: Is there a geometric module for spatial orientation? Insights from a rodent navigation model. Psychol. Rev. 116, 540–566 (2009)
Schultz, W.: Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27 (1998)
Young, J.J., Shapiro, M.L.: Double dissociation and hierarchical organization of strategy switches and reversals in the rat PFC. Behav. Neurosci. 123, 1028–1035 (2009)
Chavarriaga, R., Strösslin, T., Sheynikhovich, D., Gerstner, W.: A computational model of parallel navigation systems in rodents. Neuroinformatics 3, 223–242 (2005)
Dollé, L., Khamassi, M., Girard, B., Guillot, A., Chavarriaga, R.: Analyzing interactions between navigation strategies using a computational model of action selection. In: Freksa, C., Newcombe, N.S., Gärdenfors, P., Wölfl, S. (eds.) Spatial Cognition VI. LNCS (LNAI), vol. 5248, pp. 71–86. Springer, Heidelberg (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sheynikhovich, D., Dollé, L., Chavarriaga, R., Arleo, A. (2010). Minimal Model of Strategy Switching in the Plus-Maze Navigation Task. In: Doncieux, S., Girard, B., Guillot, A., Hallam, J., Meyer, JA., Mouret, JB. (eds) From Animals to Animats 11. SAB 2010. Lecture Notes in Computer Science(), vol 6226. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15193-4_37
Download citation
DOI: https://doi.org/10.1007/978-3-642-15193-4_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15192-7
Online ISBN: 978-3-642-15193-4
eBook Packages: Computer ScienceComputer Science (R0)