, Volume 236, Issue 8, pp 2543–2556 | Cite as

Relative insensitivity to time-out punishments induced by win-paired cues in a rat gambling task

  • Angela J. LangdonEmail author
  • Brett A. Hathaway
  • Samuel Zorowitz
  • Cailean B. W. Harris
  • Catharine A. WinstanleyEmail author
Original Investigation



Pairing rewarding outcomes with audiovisual cues in simulated gambling games increases risky choice in both humans and rats. However, the cognitive mechanism through which this sensory enhancement biases decision-making is unknown.


To assess the computational mechanisms that promote risky choice during gambling, we applied a series of reinforcement learning models to a large dataset of choices acquired from rats as they each performed one of two variants of a rat gambling task (rGT), in which rewards on “win” trials were delivered either with or without salient audiovisual cues.


We used a sampling technique based on Markov chain Monte Carlo to obtain posterior estimates of model parameters for a series of RL models of increasing complexity, in order to assess the relative contribution of learning about positive and negative outcomes to the latent valuation of each choice option on the cued and uncued rGT.


Rats which develop a preference for the risky options on the rGT substantially down-weight the equivalent cost of the time-out punishments during these tasks. For each model tested, the reduction in learning from the negative time-outs correlated with the degree of risk preference in individual rats. We found no apparent relationship between risk preference and the parameters that govern learning from the positive rewards.


The emergence of risk-preferring choice on the rGT derives from a relative insensitivity to the cost of the time-out punishments, as opposed to a relative hypersensitivity to rewards. This hyposensitivity to punishment is more likely to be induced in individual rats by the addition of salient audiovisual cues to rewards delivered on win trials.


Decision-making Reward Gambling Risk Individual differences Computational modeling 



This work was supported by the National Institutes of Health grant R01DA042065 from NIDA and the Swartz Center for Theoretical Neuroscience at Princeton University (AJL) and an operating grant awarded to CAW from the Canadian Institutes for Health Research (CIHR; PJT-162312).

Compliance with ethical standards

All housing conditions and testing procedures were in accordance with the guidelines of the Canadian Council on Animal Care, and all protocols were approved by the Animal Care Committee of the University of British Columbia.

Conflict of interest

In the past 3 years, CAW has consulted for Hogan Lovells LLP and received due compensation. The authors confirm they have no other conflicts of interest or financial disclosures to make.


  1. Adams WK, Barkus C, Ferland J-MN, Sharp T, Winstanley CA (2017) Pharmacological evidence that 5-HT2C receptor blockade selectively improves decision making when rewards are paired with audiovisual cues in a rat gambling task. Psychopharmacology 234:3091–3104CrossRefGoogle Scholar
  2. Ahn W-Y, Haines N, Zhang L (2017) Revealing neurocomputational mechanisms of reinforcement learning and decision-making with the hBayesDM package. Comput Psychiatry 1:24–57CrossRefGoogle Scholar
  3. Alter A (2017) Irresistible: the rise of addictive technology and the business of keeping us hooked. Penguin Press, New YorkGoogle Scholar
  4. Anderson BA, Laurent PA, Yantis S (2011) Value-driven attentional capture. PNAS 108:10367–10371CrossRefGoogle Scholar
  5. Barrus MM, Winstanley CA (2016) Dopamine D3 receptors modulate the ability of win-paired cues to increase risky choice in a rat gambling task. J Neurosci 36:785–794CrossRefGoogle Scholar
  6. Barrus MM, Hosking JG, Zeeb FD, Tremblay M, Winstanley CA (2015) Disadvantageous decision-making on a rodent gambling task is associated with increased motor impulsivity in a population of male rats. J Psychiatry Neurosci 40:108–117Google Scholar
  7. Bechara A, Dolan S, Denburg N, Hindes A, Anderson SW, Nathan PE (2001) Decision-making deficits, linked to a dysfunctional ventromedial prefrontal cortex, revealed in alcohol and stimulant abusers. Neuropsychologia 39:376–389CrossRefGoogle Scholar
  8. Berridge KC, Robinson TE (1998) What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Rev 28:309–369CrossRefGoogle Scholar
  9. Breen RB, Zimmerman M (2002) Rapid onset of pathological gambling in machine gamblers. J Gambl Stud 18:31–43CrossRefGoogle Scholar
  10. Carpenter B, Gelman A, Hoffman MD, Lee D, Goodrich B, Betancourt M, Brubaker M, Guo J, Li P, Riddell A (2017) Stan: a probabilistic programming language. J Stat Softw 76:1–32CrossRefGoogle Scholar
  11. Cherkasova MV, Clark L, Barton JJS, Schulzer M, Shafiee M, Kingstone A, Stoessl AJ, Winstanley CA (2018) Win-concurrent sensory cues can promote riskier choice. J Neurosci 38:10362–10370CrossRefGoogle Scholar
  12. Chudasama Y, Robbins TW (2003) Dissociable contributions of the orbitofrontal and infralimbic cortex to Pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J Neurosci 23:8771–8780CrossRefGoogle Scholar
  13. Clark L, Lawrence AJ, Astley-Jones F, Gray N (2009) Gambling near-misses enhance motivation to gamble and recruit win-related brain circuitry. Neuron 61:481–490CrossRefGoogle Scholar
  14. Cocker PJ, Hosking JG, Benoit J, Winstanley CA (2012) Sensitivity to cognitive effort mediates psychostimulant effects on a novel rodent cost/benefit decision-making task. Neuropsychopharmacology 37:1825–1837CrossRefGoogle Scholar
  15. Dixon MJ, Harrigan KA, Sandhu R, Collins K, Fugelsang JA (2010) Losses disguised as wins in modern multi-line video slot machines. Addiction 105:1819–1824CrossRefGoogle Scholar
  16. Dixon MJ, Harrigan KA, Santesso DL, Graydon C, Fugelsang JA, Collins K (2014) The impact of sound in modern multiline video slot machine play. J Gambl Stud 30:913–929CrossRefGoogle Scholar
  17. Dixon MJ, Collins K, Harrigan KA, Graydon C, Fugelsang JA (2015) Using sound to unmask losses disguised as wins in multiline slot machines. J Gambl Stud 31:183–196CrossRefGoogle Scholar
  18. Dow Schull N (2014) Addiction by design: machine gambling in Las Vegas. Princeton University Press, PrincetonGoogle Scholar
  19. Dowling N, Smith D, Thomas T (2005) Electronic gaming machines: are they the ‘crack-cocaine’ of gambling? Addiction 100:33–45CrossRefGoogle Scholar
  20. Ferland J-MN, Hynes TJ, Hounjet CD, Lindenbach D, Haar CV, Adams WK, Phillips AG, Winstanley CA (2019) Prior exposure to salient win-paired cues in a rat gambling task increases sensitivity to cocaine self-administration and suppresses dopamine efflux in nucleus accumbens: support for the reward deficiency hypothesis of addiction. J Neurosci 39:1842–1854CrossRefGoogle Scholar
  21. Flagel SB, Akil H, Robinson TE (2009) Individual differences in the attribution of incentive salience to reward-related cues: implications for addiction. Neuropharmacology 56:139–148CrossRefGoogle Scholar
  22. Flagel SB, Robinson TE, Clark JJ, Clinton SM, Watson SJ, Seeman P, Phillips PEM, Akil H (2010) An animal model of genetic vulnerability to behavioral disinhibition and responsiveness to reward-related cues: implications for addiction. Neuropsychopharmacology 35:388–400CrossRefGoogle Scholar
  23. Flagel SB, Cameron CM, Pickup KN, Watson SJ, Akil H, Robinson TE (2011) A food predictive cue must be attributed with incentive salience for it to induce c-fos mRNA expression in cortico-striatal-thalamic brain regions. Neuroscience 196:80–96CrossRefGoogle Scholar
  24. Gallagher M, McMahan RW, Schoenbaum G (1999) Orbitofrontal cortex and representation of incentive value in associative learning. J Neurosci 19:6610–6614CrossRefGoogle Scholar
  25. Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB, Carlin JB, Stern HS, Dunson DB, Vehtari A, et al (2013) Bayesian data analysis (Chapman and Hall/CRC)Google Scholar
  26. Georgiou P, Zanos P, Bhat S, Tracy JK, Merchenthaler IJ, McCarthy MM, Gould TD (2018) Dopamine and stress system modulation of sex differences in decision making. Neuropsychopharmacology 43:313–324CrossRefGoogle Scholar
  27. Gonzalez R, Schuster RM, Mermelstein RM, Diviak KR (2015) The role of decision-making in cannabis-related problems among young adults. Drug Alcohol Depend 154:214–221CrossRefGoogle Scholar
  28. Griffiths M (1991) Psychobiology of the near-miss in fruit machine gambling. J Psychol 125:347–357CrossRefGoogle Scholar
  29. Griffiths M, Scarfe A, Bellringer P (1999) The UK National Telephone Gambling Helpline—results on the first year of operation. J Gambl Stud 15:83–90CrossRefGoogle Scholar
  30. Kruschke J (2014) Doing Bayesian data analysis: a tutorial with R, JAGS, and Stan (Academic)Google Scholar
  31. Limbrick-Oldfield EH, Mick I, Cocks RE, McGonigle J, Sharman SP, Goldstone AP, Stokes PRA, Waldman A, Erritzoe D, Bowden-Jones H, Nutt D, Lingford-Hughes A, Clark L (2017) Neural substrates of cue reactivity and craving in gambling disorder. Transl Psychiatry 7:e992CrossRefGoogle Scholar
  32. Meyer PJ, Lovic V, Saunders BT, Yager LM, Flagel SB, Morrow JD, Robinson TE (2012) Quantifying individual variation in the propensity to attribute incentive salience to reward cues. PLoS One 7:e38987CrossRefGoogle Scholar
  33. Petry NM (2000) Psychiatric symptoms in problem gambling and non-problem gambling substance abusers. Am J Addict 9:163–171CrossRefGoogle Scholar
  34. Petry NM, Stinson FS, Grant BF (2005) Comorbidity of DSM-IV pathological gambling and other psychiatric disorders: results from the National Epidemiologic Survey on alcohol and related conditions. J Clin Psychiatry 66:564–574CrossRefGoogle Scholar
  35. Robinson MJF, Fischer AM, Ahuja A, Lesser EN, Maniates H (2016) Roles of “wanting” and “liking” in motivating behavior: gambling, food, and drug addictions. In: Simpson EH, Balsam PD (eds) Behavioral neuroscience of motivation. Cham, Springer, pp 105–136Google Scholar
  36. Rømer Thomsen K, Fjorback LO, Møller A, Lou HC (2014) Applying incentive sensitization models to behavioral addiction. Neurosci Biobehav Rev 45:343–349CrossRefGoogle Scholar
  37. Rudebeck PH, Murray EA (2008) Amygdala and orbitofrontal cortex lesions differentially influence choices during object reversal learning. J Neurosci 28:8338–8343CrossRefGoogle Scholar
  38. Saunders BT, Robinson TE (2013) Individual variation in resisting temptation: implications for addiction. Neurosci Biobehav Rev 37:1955–1975CrossRefGoogle Scholar
  39. Silveira MM, Murch WS, Clark L, Winstanley CA (2016) Chronic atomoxetine treatment during adolescence does not influence decision-making on a rodent gambling task, but does modulate amphetamine’s effect on impulsive action in adulthoodGoogle Scholar
  40. Stalnaker TA, Berg B, Aujla N, Schoenbaum G (2016) Cholinergic interneurons use orbitofrontal input to track beliefs about current state. J Neurosci 36:6242–6257CrossRefGoogle Scholar
  41. Stevens L, Betanzos-Espinosa P, Crunelle CL, Vergara-Moragues E, Roeyers H, Lozano O, Dom G, Gonzalez-Saiz F, Vanderplasschen W, Verdejo-García A, Pérez-García M (2013) Disadvantageous decision-making as a predictor of drop-out among cocaine-dependent individuals in long-term residential treatment. Front Psychiatry 4Google Scholar
  42. Stevens L, Goudriaan AE, Verdejo-Garcia A, Dom G, Roeyers H, Vanderplasschen W (2015) Impulsive choice predicts short-term relapse in substance-dependent individuals attending an in-patient detoxification programme. Psychol Med 45:2083–2093CrossRefGoogle Scholar
  43. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, CambridgeGoogle Scholar
  44. Takahashi YK, Roesch MR, Wilson RC, Toreson K, O’Donnell P, Niv Y, Schoenbaum G (2011) Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat Neurosci 14:1590–1597CrossRefGoogle Scholar
  45. Tomb I, Hauser M, Deldin P, Caramazza A (2002) Do somatic markers mediate decisions on the gambling task? Nat Neurosci 5:1103–1104CrossRefGoogle Scholar
  46. van den Bos R, Jolles J, van der Knaap L, Baars A, de Visser L (2012) Male and female Wistar rats differ in decision-making performance in a rodent version of the Iowa gambling task. Behav Brain Res 234:375–379CrossRefGoogle Scholar
  47. van den Bos R, Homberg J, de Visser L (2013) A critical review of sex differences in decision-making tasks: focus on the Iowa gambling task. Behav Brain Res 238:95–108CrossRefGoogle Scholar
  48. Vu MAT, Adalı T, Ba D, Buzsáki G, Carlson D, Heller K, Liston C, Rudin C, Sohal VS, Widge AS, Mayberg HS, Sapiro G, Dzirasa K (2018) A Shared Vision for Machine Learning in Neuroscience. J Neurosci 38 (7):1601-1607Google Scholar
  49. Watanabe S (2010) Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. J Mach Learn Res 11:3571–3594Google Scholar
  50. Wilson RC, Takahashi YK, Schoenbaum G, Niv Y (2014) Orbitofrontal cortex as a cognitive map of task space. Neuron 81:267–279CrossRefGoogle Scholar
  51. Zeeb FD, Winstanley CA (2011) Lesions of the basolateral amygdala and orbitofrontal cortex differentially affect acquisition and performance of a rodent gambling task. J Neurosci 31:2197–2204CrossRefGoogle Scholar
  52. Zeeb FD, Floresco SB, Winstanley CA (2010) Contributions of the orbitofrontal cortex to impulsive choice: interactions with basal levels of impulsivity, dopamine signalling, and reward-related cues. Psychopharmacology 211:87–98CrossRefGoogle Scholar
  53. Zhang F, Xiao L, Gu R (2017) Does gender matter in the relationship between anxiety and decision-making? Front Psychol 8Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Princeton Neuroscience Institute and Department of PsychologyPrinceton UniversityPrincetonUSA
  2. 2.Department of PsychologyUniversity of British ColumbiaVancouverCanada

Personalised recommendations