Rodent Models of Adaptive Value Learning and Decision-Making

  • Alicia IzquierdoEmail author
  • Claudia Aguirre
  • Evan E. Hart
  • Alexandra Stolyarova
Part of the Methods in Molecular Biology book series (MIMB, volume 2011)


Real-world decisions are rarely as straightforward as choosing between clearly “good” vs. “bad” options. More often, options must be evaluated carefully because they differ in relative value. For example, we typically learn about (and make decisions between) options in comparison, where one outcome may be more costly or risky than the other. Several neuropsychiatric conditions are characterized by atypical evaluation of effort and risk costs, including major depression, schizophrenia, autism, obsessive-compulsive disorder, and substance use disorders. Aberrant value learning and decision-making have long been considered a cognitive-behavioral endophenotype of these disorders and can be modeled in rodents. This chapter presents two general methodological domains that the experimenter can manipulate in animal decision-making tasks: risk and effort. Here, we present detailed methods of rodent tasks frequently employed within these domains: probabilistic reversal learning (PRL) and effort choice. These tasks recruit regions within rodent frontal cortex, the amygdala, and the striatum, and performance is heavily modulated by dopamine, making these assays highly valid measures in the study of behavioral and substance addictions, in particular.

Key words

Reversal learning Effort discounting Orbitofrontal cortex Anterior cingulate cortex Basolateral amygdala 



This research was supported by the UCLA Academic Senate Grant and the UCLA Division of Life Sciences Recruitment and Retention Fund.


  1. 1.
    Deisseroth K (2015) Optogenetics: 10 years of microbial opsins in neuroscience. Nat Neurosci 18:1213–1225CrossRefGoogle Scholar
  2. 2.
    English JG, Roth BL (2015) Chemogenetics-a transformational and translational platform. JAMA Neurol 72:1361–1366CrossRefGoogle Scholar
  3. 3.
    Cai DJ, Aharoni D, Shuman T, Shobe J, Biane J, Song W, Wei B, Veshkini M, La-Vu M, Lou J, Flores SE, Kim I, Sano Y, Zhou M, Baumgaertel K, Lavi A, Kamata M, Tuszynski M, Mayford M, Golshani P, Silva AJ (2016) A shared neural ensemble links distinct contextual memories encoded close in time. Nature 534:115–118CrossRefGoogle Scholar
  4. 4.
    Izquierdo A, Brigman JL, Radke AK, Rudebeck PH, Holmes A (2017) The neural basis of reversal learning: an updated perspective. Neuroscience 345:12–26CrossRefGoogle Scholar
  5. 5.
    Wassum KM, Izquierdo A (2015) The basolateral amygdala in reward learning and addiction. Neurosci Biobehav Rev 57:271–283CrossRefGoogle Scholar
  6. 6.
    Dayan P, Daw ND (2008) Decision theory, reinforcement learning, and the brain. Cogn Affect Behav Neurosci 8:429–453CrossRefGoogle Scholar
  7. 7.
    Addicott MA, Pearson JM, Sweitzer MM, Barack DL, Platt ML (2017) A primer on foraging and the explore/exploit trade-off for psychiatry research. Neuropsychopharmacology 42:1931–1939CrossRefGoogle Scholar
  8. 8.
    McNamara JM, Fawcett TW, Houston AI (2013) An adaptive response to uncertainty generates positive and negative contrast effects. Science 340:1084–1086CrossRefGoogle Scholar
  9. 9.
    Rangel A, Camerer C, Montague PR (2008) A framework for studying the neurobiology of value-based decision making. Nat Rev Neurosci 9:545–556CrossRefGoogle Scholar
  10. 10.
    O’Leary JD, O’Leary OF, Cryan JF, Nolan YM (2018) A low-cost touchscreen operant chamber using a Raspberry Pi. Behav Res Methods 50(6):2523–2530CrossRefGoogle Scholar
  11. 11.
    Izquierdo A, Belcher AM, Scott L, Cazares VA, Chen J, O’Dell SJ, Malvaez M, Wu T, Marshall JF (2010) Reversal-specific learning impairments after a binge regimen of methamphetamine in rats: possible involvement of striatal dopamine. Neuropsychopharmacology 35:505–514CrossRefGoogle Scholar
  12. 12.
    Stolyarova A, Izquierdo A (2017) Complementary contributions of basolateral amygdala and orbitofrontal cortex to value learning under uncertainty. elife 6:e27483CrossRefGoogle Scholar
  13. 13.
    Izquierdo A (2017) Functional heterogeneity within rat orbitofrontal cortex in reward learning and decision making. J Neurosci 37:10529–10540CrossRefGoogle Scholar
  14. 14.
    Izquierdo A, Wiedholz LM, Millstein RA, Yang RJ, Bussey TJ, Saksida LM, Holmes A (2006) Genetic and dopaminergic modulation of reversal learning in a touchscreen-based operant procedure for mice. Behav Brain Res 171:181–188CrossRefGoogle Scholar
  15. 15.
    Morton AJ, Skillings E, Bussey TJ, Saksida LM (2006) Measuring cognitive deficits in disabled mice using an automated interactive touchscreen system. Nat Methods 3:767CrossRefGoogle Scholar
  16. 16.
    Salamone JD, Steinpreis RE, McCullough LD, Smith P, Grebel D, Mahan K (1991) Haloperidol and nucleus accumbens dopamine depletion suppress lever pressing for food but increase free food consumption in a novel food choice procedure. Psychopharmacology 104:515–521CrossRefGoogle Scholar
  17. 17.
    Nowend KL, Arizzi M, Carlson BB, Salamone JD (2001) D1 or D2 antagonism in nucleus accumbens core or dorsomedial shell suppresses lever pressing for food but leads to compensatory increases in chow consumption. Pharmacol Biochem Behav 69:373–382CrossRefGoogle Scholar
  18. 18.
    Salamone JD, Wisniecki A, Carlson BB, Correa M (2001) Nucleus accumbens dopamine depletions make animals highly sensitive to high fixed ratio requirements but do not impair primary food reinforcement. Neuroscience 105:863–870CrossRefGoogle Scholar
  19. 19.
    Salamone JD, Correa M, Farrar A, Mingote SM (2007) Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology 191:461–482CrossRefGoogle Scholar
  20. 20.
    Nunes EJ, Randall PA, Hart EE, Freeland C, Yohn SE, Baqi Y, Muller CE, Lopez-Cruz L, Correa M, Salamone JD (2013) Effort-related motivational effects of the VMAT-2 inhibitor tetrabenazine: implications for animal models of the motivational symptoms of depression. J Neurosci 33:19120–19130CrossRefGoogle Scholar
  21. 21.
    Floresco SB, Ghods-Sharifi S (2007) Amygdala-prefrontal cortical circuitry regulates effort-based decision making. Cereb Cortex 17:251–260CrossRefGoogle Scholar
  22. 22.
    Rudebeck PH, Walton ME, Smyth AN, Bannerman DM, Rushworth MF (2006) Separate neural pathways process different decision costs. Nat Neurosci 9:1161–1168CrossRefGoogle Scholar
  23. 23.
    Walton ME, Bannerman DM, Alterescu K, Rushworth MF (2003) Functional specialization within medial frontal cortex of the anterior cingulate for evaluating effort-related decisions. J Neurosci 23:6475–6479CrossRefGoogle Scholar
  24. 24.
    Walton ME, Bannerman DM, Rushworth MF (2002) The role of rat medial frontal cortex in effort-based decision making. J Neurosci 22:10996–11003CrossRefGoogle Scholar
  25. 25.
    Walton ME, Rudebeck PH, Bannerman DM, Rushworth MF (2007) Calculating the cost of acting in frontal cortex. Ann N Y Acad Sci 1104:340–356CrossRefGoogle Scholar
  26. 26.
    Hart EE, Gerson JO, Zoken Y, Garcia M, Izquierdo A (2017) Anterior cingulate cortex supports effort allocation toward a qualitatively preferred option. Eur J Neurosci 46(1):1682–1688CrossRefGoogle Scholar
  27. 27.
    Hart EE, Izquierdo A (2017) Basolateral amygdala supports the maintenance of value and effortful choice of a preferred option. Eur J Neurosci 45:388–397CrossRefGoogle Scholar
  28. 28.
    Ostrander S, Cazares VA, Kim C, Cheung S, Gonzalez I, Izquierdo A (2011) Orbitofrontal cortex and basolateral amygdala lesions result in suboptimal and dissociable reward choices on cue-guided effort in rats. Behav Neurosci 125:350–359CrossRefGoogle Scholar
  29. 29.
    Stolyarova A, Thompson AB, Barrientos RM, Izquierdo A (2015) Reductions in frontocortical cytokine levels are associated with long-lasting alterations in reward valuation after methamphetamine. Neuropsychopharmacology 40:1234–1242CrossRefGoogle Scholar
  30. 30.
    Winstanley CA, Floresco SB (2016) Deciphering decision making: variation in animal models of effort- and uncertainty-based choice reveals distinct neural circuitries underlying core cognitive processes. J Neurosci 36:12069–12079CrossRefGoogle Scholar
  31. 31.
    Izquierdo A, Pozos H, Torre Ade L, DeShields S, Cevallos J, Rodriguez J, Stolyarova A (2016) Sex differences, learning flexibility, and striatal dopamine D1 and D2 following adolescent drug exposure in rats. Behav Brain Res 308:104–114CrossRefGoogle Scholar
  32. 32.
    Stolyarova A, Izquierdo A (2015) Distinct patterns of outcome valuation and amygdala-prefrontal cortex synaptic remodeling in adolescence and adulthood. Front Behav Neurosci 9:115CrossRefGoogle Scholar
  33. 33.
    Kosheleff AR, Rodriguez D, O’Dell SJ, Marshall JF, Izquierdo A (2012) Comparison of single-dose and extended methamphetamine administration on reversal learning in rats. Psychopharmacology 224:459–467CrossRefGoogle Scholar
  34. 34.
    Randall PA, Pardo M, Nunes EJ, Lopez Cruz L, Vemuri VK, Makriyannis A, Baqi Y, Muller CE, Correa M, Salamone JD (2012) Dopaminergic modulation of effort-related choice behavior as assessed by a progressive ratio chow feeding choice task: pharmacological studies and the role of individual differences. PLoS One 7:e47934CrossRefGoogle Scholar
  35. 35.
    Soltani A, Noudoost B, Moore T (2013) Dissociable dopaminergic control of saccadic target selection and its implications for reward modulation. Proc Natl Acad Sci U S A 110:3579–3584CrossRefGoogle Scholar
  36. 36.
    Hosking JG, Cocker PJ, Winstanley CA (2014) Dissociable contributions of anterior cingulate cortex and basolateral amygdala on a rodent cost/benefit decision-making task of cognitive effort. Neuropsychopharmacology 39:1558–1567CrossRefGoogle Scholar
  37. 37.
    Hosking JG, Floresco SB, Winstanley CA (2015) Dopamine antagonism decreases willingness to expend physical, but not cognitive, effort: a comparison of two rodent cost/benefit decision-making tasks. Neuropsychopharmacology 40:1005–1015CrossRefGoogle Scholar
  38. 38.
    Hosking JG, Lam FC, Winstanley CA (2014) Nicotine increases impulsivity and decreases willingness to exert cognitive effort despite improving attention in “slacker” rats: insights into cholinergic regulation of cost/benefit decision making. PLoS One 9:e111580CrossRefGoogle Scholar
  39. 39.
    Randall PA, Pardo M, Nunes EJ, López Cruz L, Vemuri VK, Makriyannis A, Baqi Y, Müller CE, Correa M, Salamone JD (2012) Dopaminergic modulation of effort-related choice behavior as assessed by a progressive ratio chow feeding choice task: pharmacological studies and the role of individual differences. PLoS One 7:e47934CrossRefGoogle Scholar
  40. 40.
    Thompson AB, Gerson J, Stolyarova A, Bugarin A, Hart EE, Jentsch JD, Izquierdo A (2017) Steep effort discounting of a preferred reward over a freely-available option in prolonged methamphetamine withdrawal in male rats. Psychopharmacology 234:2697–2705CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Alicia Izquierdo
    • 1
    • 2
    • 3
    • 4
    Email author
  • Claudia Aguirre
    • 1
  • Evan E. Hart
    • 1
  • Alexandra Stolyarova
    • 1
  1. 1.Department of PsychologyUniversity of California at Los AngelesLos AngelesUSA
  2. 2.The Brain Research InstituteUniversity of California at Los AngelesLos AngelesUSA
  3. 3.Integrative Center for Learning and MemoryUniversity of California at Los AngelesLos AngelesUSA
  4. 4.Integrative Center for AddictionsUniversity of California at Los AngelesLos AngelesUSA

Personalised recommendations