Skip to main content
SpringerLink
Log in

Linking Across Levels of Computation in Model-Based Cognitive Neuroscience

  • Chapter
  • First Online:
Book cover An Introduction to Model-Based Cognitive Neuroscience

Abstract

Computational approaches to cognitive neuroscience encompass multiple levels of analysis, from detailed biophysical models of neural activity to abstract algorithmic or normative models of cognition, with several levels in between. Despite often strong opinions on the ‘right’ level of modeling, there is no single panacea: attempts to link biological with higher level cognitive processes require a multitude of approaches. Here I argue that these disparate approaches should not be viewed as competitive, nor should they be accessible to only other researchers already endorsing the particular level of modeling. Rather, insights gained from one level of modeling should inform modeling endeavors at the level above and below it. One way to achieve this synergism is to link levels of modeling by quantitatively fitting the behavioral outputs of detailed mechanistic models with higher level descriptions. If the fits are reasonable (e.g., similar to those achieved when applying high level models to human behavior), one can then derive plausible links between mechanism and computation. Model-based cognitive neuroscience approaches can then be employed to manipulate or measure neural function motivated by the candidate mechanisms, and to test whether these are related to high level model parameters. I describe several examples of this approach in the domain of reward-based learning, cognitive control, and decision making and show how neural and algorithmic models have each informed or refined the other.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alexander GE, Crutcher MD (1990) Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci 13(7):266–271

    Article  CAS  PubMed  Google Scholar 

  2. Anderson JR (1991) The adaptive nature of human categorization. Psychol Rev 98(3):409–429

    Article  Google Scholar 

  3. Aron AR, Behrens TE, Smith S, Frank MJ, Poldrack R (2007) Triangulating a cognitive control network using diffusion-weighted magnetic resonance imaging (MRI) and functional MRI. J Neurosci 27(14):3743–3752

    Article  CAS  PubMed  Google Scholar 

  4. Badre D, Frank MJ (2012) Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 2: Evidence from fMRI. Cerebral Cortex 22:527–536

    Google Scholar 

  5. Badre D, Doll BB, Long NM, Frank MJ (2012) Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration. Neuron 73:595–607

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  6. Bódi N, Kéri S, Nagy H, Moustafa A, Myers CE, Daw N, Dibó G, Takáts A, Bereczki D, Gluck MA (2009). Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients. Brain 132:2385–2395

    Google Scholar 

  7. Bogacz R, Gurney K (2007) The basal ganglia and cortex implement optimal decision making between alternative actions. Neural Comput 19(2):442–477

    Article  PubMed  Google Scholar 

  8. Bogacz R, Larsen T (2011) Integration of reinforcement learning and optimal decision-making theories of the basal ganglia. Neural Comput 23(4):817–851

    Article  PubMed  Google Scholar 

  9. Bogacz R, Wagenmaker EJ, Forstmann BU, Nieuwenhuis S (2010) The neural basis of the speed-accuracy tradeoff. Trends Neurosci 33(1):10–16

    Article  CAS  PubMed  Google Scholar 

  10. Botvinick MM, Niv Y, Barto AC (2009) Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition 113(3):262–280

    Article  PubMed Central  PubMed  Google Scholar 

  11. Braver TS, Cohen JD (2000) On the control of control: the role of dopamine in regulating pre- frontal function and working memory. In: Monsell S, Driver J (eds) Control of cognitive processes: attention and performance XVIII. MIT Press, Cambridge, pp 713–737

    Google Scholar 

  12. Brittain JS, Watkins KE, Joundi RA, Ray NJ, Holland P, Green AL, Aziz TJ, Jenkinson N (2012) A role for the subthalamic nucleus in response inhibition during conflict. J Neurosci 32(39):13396–13401

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  13. Cavanagh JF, Frank MJ, Klein TJ, Allen JJB (2010) Frontal theta links prediction error to behavioral adaptation in reinforcement learning. Neuroimage 49(4):3198–3209

    Article  PubMed Central  PubMed  Google Scholar 

  14. Cavanagh JF, Figueroa CM, Cohen MX, Frank MJ (2011a) Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation. Cereb Cortex 22(11):2575–2586

    Article  PubMed Central  PubMed  Google Scholar 

  15. Cavanagh JF, Wiecki TV, Cohen MX, Figueroa CM, Samanta J, Sherman SJ, Frank MJ (2011b) Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold. Nat Neurosci 14(11):1462–1467

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  16. Collins AGE, Frank MJ (2012) How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. Eur J Neurosci 35(7):1024–1035

    Article  PubMed Central  PubMed  Google Scholar 

  17. Collins AGE, Frank MJ (2013) Cognitive control over learning: creating, clustering and generalizing task-set structure. Psychol Rev 120(1):190–229

    Article  PubMed Central  PubMed  Google Scholar 

  18. Collins AGE, Frank MJ (2014) Opponent Actor Learning (OpAL): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychol Rev 121:337–366

    Google Scholar 

  19. Coulthard EJ, Bogacz R, Javed S, Mooney LK, Murphy G, Keeley S, Whone AL (2012). Distinct roles of dopamine and subthalamic nucleus in learning and probabilistic decision making. Brain 135:3721–3734

    Article  PubMed Central  PubMed  Google Scholar 

  20. Dayan P, Sejnowksi T (1996) Exploration bonuses and dual control. Mach Learn 25:5–22

    Google Scholar 

  21. Doll BB, Jacobs WJ, Sanfey AG, Frank MJ (2009) Instructional control of reinforcement learning: a behavioral and neurocomputational investigation. Brain Res 1299:74–94

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  22. Doll BB, Hutchison KE, Frank MJ (2011) Dopaminergic genes predict individual differences in susceptibility to confirmation bias. J Neurosci 31(16):6188–6198

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  23. Ford KA, Everling S (2009) Neural activity in primate caudate nucleus associated with pro- and antisaccades. J Neurophysiol 102(4):2334–2341

    Article  PubMed  Google Scholar 

  24. Forstmann BU, Dutilh G, Brown S, Neumann J, von Cramon DY, Ridderinkhof KR, Wagenmakers EJ (2008a) Striatum and pre-SMA facilitate decision-making under time pressure. Proc Natl Acad Sci USA 105(45):17538–17542

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  25. Forstmann BU, Jahfari S, Scholte HS, Wolfensteller U, van den Wildenberg WP, Ridderinkhof KR (2008b) Function and structure of the right inferior frontal cortex predict individual differences in response inhibition: a model-based approach. J Neurosci 28(39):9790–9796

    Article  CAS  PubMed  Google Scholar 

  26. Frank MJ (2005) Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J Cogn Neurosci 17(1):51–72

    Article  PubMed  Google Scholar 

  27. Frank MJ (2006) Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making. Neural Netw 19(8):1120–1136

    Article  PubMed  Google Scholar 

  28. Frank MJ (2011) Computational models of motivated action selection in corticostriatal circuits. Curr Opin Neurobiol 2:381–386

    Article  Google Scholar 

  29. Frank MJ, Badre D (2012) Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. Cereb Cortex 22(3):509–526

    Article  PubMed Central  PubMed  Google Scholar 

  30. Frank MJ, Claus ED (2006) Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychol Rev 113(2):300–326

    Article  PubMed  Google Scholar 

  31. Frank MJ, Fossella JA (2011) Neurogenetics and pharmacology of learning, motivation, and cognition. Neuropsychopharmacology 36:133–152

    Article  PubMed Central  PubMed  Google Scholar 

  32. Frank MJ, Hutchison K (2009) Genetic contributions to avoidance-based decisions: striatal D2 receptor polymorphisms. Neuroscience 164(1):131–140

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  33. Frank MJ, O'Reilly RC (2006) A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav Neurosci 120(3):497–517

    Article  CAS  PubMed  Google Scholar 

  34. Frank MJ, Loughry B, O'Reilly RC (2001) Interactions between frontal cortex and basal ganglia in working memory: a computational model. Cogn Affect Behav Neurosci 1(2):137–160

    Article  CAS  PubMed  Google Scholar 

  35. Frank MJ, Seeberger LC, O'Reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306(5703):1940–1943

    Article  CAS  PubMed  Google Scholar 

  36. Frank MJ, Santamaria A, O'Reilly R, Willcutt E (2007a) Testing computational models of dopamine and noradrenaline dysfunction in attention deficit/hyperactivity disorder. Neuropsychopharmacology 32(7):1583–1599

    Article  CAS  PubMed  Google Scholar 

  37. Frank MJ, D’Lauro C, Curran T (2007b) Cross-task individual differences in error processing: neural, electrophysiological, and genetic components. Cogn Affect Behav Neurosci 7(4):297–308

    Article  PubMed  Google Scholar 

  38. Frank MJ, Moustafa AA, Haughey H, Curran T, Hutchison K (2007c) Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci 104(41):16311–16316

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  39. Frank MJ, Samanta J, Moustafa AA, Sherman SJ (2007d) Hold your horses: impulsivity, deep brain stimulation and medication in Parkinsonism. Science 318:1309–1312

    Article  CAS  Google Scholar 

  40. Frank MJ, Doll BB, Oas-Terpstra J, Moreno F (2009). Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat Neurosci 12(8):1062–1068

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  41. Frank MJ, Gagne C, Nyhus E, Masters S, Wiecki TV, Cavanagh JF, Badre D (2015) fMRI and EEG Predictors of dynamic decision parameters during human reinforcement learning. J Neurosci 35:484-494

    Google Scholar 

  42. Frazier P, Yu AJ (2008) Sequential hypothesis testing under stochastic deadlines. Adv Neural Inf Process Syst 20:465–472. (MIT Press, Cambridge)

    Google Scholar 

  43. Gurney K, Prescott TJ, Redgrave P (2001) A computational model of action selection in the basal ganglia. II. Analysis and simulation of behaviour. Biol Cybern 84(6):411–423

    Article  CAS  PubMed  Google Scholar 

  44. Hikida T, Kimura K, Wada N, Funabiki K, Nakanishi S (2010) Distinct roles of synaptic transmission in direct and indirect striatal pathways to reward and aversive behavior. Neuron 66(6):896–907

    Article  CAS  PubMed  Google Scholar 

  45. Isoda M, Hikosaka O (2007) Switching from automatic to controlled action by monkey medial frontal cortex. Nat Neurosci 10(2):240–248

    Article  CAS  PubMed  Google Scholar 

  46. Isoda M, Hikosaka O (2008) Role for subthalamic nucleus neurons in switching from automatic to controlled eye movement. J Neurosci 28(28):7209–7218

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  47. Jahfari S, Verbruggen F, Frank MJ, Waldorp LJ, Colzato L, Ridderinkhof KR, Forstmann BU (2012) How preparation changes the need for top-down control of the basal ganglia when inhibiting premature actions. J Neurosci 32(32):10870–10878

    Article  CAS  PubMed  Google Scholar 

  48. Koechlin E, Summerfield C (2007) An information theoretical approach to the prefrontal executive function. Trends Cogn Sci 11(6):229–235

    Article  PubMed  Google Scholar 

  49. Kravitz AV, Freeze BS, Parker PRL, Kay K, Thwin MT, Deisseroth K, Kreitzer AC (2010) Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature 466(7306):622–626

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  50. Kravitz AV, Tye LD, Kreitzer AC (2012) Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat Neurosci 15:816–818

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  51. Lau B, Glimcher PW (2008) Value representations in the primate striatum during matching behavior. Neuron 58(3):451–463

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  52. Lo CC, Wang XJ (2006) Cortico-basal ganglia circuit mechanism for a decision threshold in reaction time tasks. Nat Neurosci 9(7):956–963

    Article  CAS  PubMed  Google Scholar 

  53. Maia TV, Frank MJ (2011) From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci 2:154–162

    Article  Google Scholar 

  54. Mink JW (1996) The basal ganglia: focused selection and inhibition of competing motor programs. Prog Neurobiol 50(4):381–425

    Article  CAS  PubMed  Google Scholar 

  55. Munafò MR, Stothart G, Flint J (2009) Bias in genetic association studies and impact factor. Mol Psychiatry 14:119–120

    Article  PubMed  Google Scholar 

  56. O'Reilly RC, McClelland JL (1994) Hippocampal conjunctive encoding, storage, and recall: avoiding a trade-off. Hippocampus 4(6):661–682

    Article  PubMed  Google Scholar 

  57. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442(7106):1042–1045

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  58. Ratcliff R, Frank MJ (2012) Reinforcement-based decision making in corticostriatal circuits: mutual constraints by neurocomputational and diffusion models. Neural Comput 24:1186–1229

    Article  PubMed  Google Scholar 

  59. Ratcliff R, McKoon G (2008) The diffusion decision model: theory and data for two-choice decision tasks. Neural Comput 20(4):873–922

    Article  PubMed Central  PubMed  Google Scholar 

  60. Reynolds JR, O’Reilly RC (2009) Developing PFC representations using reinforcement learning. Cognition 113(3):281–292

    Article  PubMed Central  PubMed  Google Scholar 

  61. Samejima K, Ueda Y, Doya K, Kimura M (2005) Representation of action-specific reward values in the striatum. Science 310(5752):1337–1340

    Article  CAS  PubMed  Google Scholar 

  62. Sanborn AN, Griffiths TL, Navarro DJ (2010) Rational approximations to rational models: alternative algorithms for category learning. Psychol Rev 117(4):1144–1167

    Article  PubMed  Google Scholar 

  63. Schultz W (2002) Getting formal with dopamine and reward. Neuron 36(2):241–263

    Article  CAS  PubMed  Google Scholar 

  64. Shen W, Flajolet M, Greengard P, Surmeier DJ (2008) Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321(5890):848–851

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  65. Strauss GP, Frank MJ, Waltz JA, Kasanova Z, Herbener ES, Gold JM (2011) Deficits in positive reinforcement learning and uncertainty-driven exploration are associated with distinct aspects of negative symptoms in schizophrenia. Biol Psychiatry 69:424–431

    Article  PubMed Central  PubMed  Google Scholar 

  66. Tai LH, Lee AM, Benavidez N, Bonci A, Wilbrecht L (2012) Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nat Neurosci 15:1281–1289

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  67. Tan KR, Yvon C, Turiault M, Mirabekov JJ, Doehner J, Labouèbe G, Deisseroth K, Tye KM, Lüscher C (2012) GABA neurons of the VTA drive conditioned place aversion. Neuron 73:1173–1183

    Article  CAS  PubMed  Google Scholar 

  68. Tsai HC, Zhang F, Adamantidis A, Stuber GD, Bonci A, de Lecea L, Deisseroth K (2009) Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324:1080–1084

    Article  CAS  PubMed  Google Scholar 

  69. Voon V, Pessiglione M, Brezing C, Gallea C, Fernandez HH, Dolan RJ, Hallett M (2010) Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. Neuron 65(1):135–142

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  70. Wang XJ (2012) Neural dynamics and circuit mechanisms of decision-making. Curr Opin Neurobiol 22:1039–1046

    Article  PubMed Central  PubMed  Google Scholar 

  71. Watanabe M, Munoz DP (2009) Neural correlates of conflict resolution between automatic and volitional actions by basal ganglia. Eur J Neurosci 30(11):2165–2176

    Article  PubMed  Google Scholar 

  72. Watkins CJCH, Dayan P (1992) Q-Learning. Mach Learn 8:279–292

    Google Scholar 

  73. Wiecki TV, Frank MJ (2010) Neurocomputational models of motor and cognitive deficits in Parkinson’s disease. Prog Brain Res 183:275–297

    Article  CAS  PubMed  Google Scholar 

  74. Wiecki TV, Frank MJ (in press). A computational model of inhibitory control in frontal cortex and basal ganglia. Psychological Review.

    Google Scholar 

  75. Wiecki TV, Riedinger K, Meyerhofer A, Schmidt W, Frank MJ (2009) A neurocomputational account of catalepsy sensitization induced by D2 receptor blockade in rats: context dependency, extinction, and renewal. Psychopharmacology (Berl) 204:265–277

    Article  CAS  Google Scholar 

  76. Wiecki TV, Sofer I, Frank MJ (2012). Hierarchical Bayesian parameter estimation of Drift Diffusion Models (Version 0.4RC1) [software]. http://ski.clps.brown.edu/hddm_docs/.

  77. Wiecki TV, Sofer I, Frank MJ (2013) HDDM: Hierarchical Bayesian estimation of the Drift-Diffusion Model in Python. Fron Neuroinformatics 7:1–10

    Google Scholar 

  78. Wong KF, Wang XJ (2006) A recurrent network mechanism of time integration in perceptual decisions. J Neurosci 26(4):1314–1328

    Article  CAS  PubMed  Google Scholar 

  79. Zaghloul K, Weidemann CT, Lega BC, Jaggi JL, Baltuch GH, Kahana MJ (2012) Neuronal activity in the human subthalamic nucleus encodes decision conflict during action selection. J Neurosci 32(7):2453–2460

    Article  PubMed Central  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cognitive, Linguistic & Psychological Sciences, Brown Institute for Brain Science, Providence, USA

    Michael J. Frank

Authors
  1. Michael J. Frank
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael J. Frank .

Editor information

Editors and Affiliations

  1. Cognitive Science Center, University of Amsterdam, Amsterdam, The Netherlands

    Birte U. Forstmann

  2. Department of Psychological Methods, University of Amsterdam, Amsterdam, The Netherlands

    Eric-Jan Wagenmakers

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Frank, M. (2015). Linking Across Levels of Computation in Model-Based Cognitive Neuroscience. In: Forstmann, B., Wagenmakers, EJ. (eds) An Introduction to Model-Based Cognitive Neuroscience. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-2236-9_8

Download citation

Publish with us

Policies and ethics

Search

Navigation