Advertisement

Animal Cognition

, Volume 21, Issue 5, pp 671–684 | Cite as

Opposing pupil responses to offered and anticipated reward values

  • Tyler Cash-Padgett
  • Habiba Azab
  • Seng Bum Michael Yoo
  • Benjamin Y. Hayden
Original Paper

Abstract

Previous studies have shown that the pupils dilate more in anticipation of larger rewards. This finding raises the possibility of a more general association between reward amount and pupil size. We tested this idea by characterizing macaque pupil responses to offered rewards during evaluation and comparison in a binary choice task. To control attention, we made use of a design in which offers occurred in sequence. By looking at pupil responses after choice but before reward, we confirmed the previously observed positive association between pupil size and anticipated reward values. Surprisingly, however, we find that pupil size is negatively correlated with the value of offered gambles before choice, during both evaluation and comparison stages of the task. These results demonstrate a functional distinction between offered and anticipated rewards and present evidence against a narrow version of the simulation hypothesis; the idea that we represent offers by reactivating states associated with anticipating them. They also suggest that pupil size is correlated with relative, not absolute, values of offers, suggestive of an accept–reject model of comparison.

Keywords

Monkey Decision-making Pupil size Value 

Notes

Acknowledgements

This work is supported by a CAREER award from NSF (BCS1253576) and a R01 from NIH (DA038615) to BYH. We thank Meghan Castagno, Marc Mancarella and Caleb Strait for assistance with data collection, and the rest of the Hayden lab for valuable discussions.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Statement on the Welfare of Animals

All procedures performed in this study involving animals were in accordance with the ethical standards of the University of Rochester.

Supplementary material

10071_2018_1202_MOESM1_ESM.tif (7.9 mb)
Supplementary material 1 (TIF 8092 KB)

References

  1. Azab H, Hayden BY (2017) Correlates of decisional dynamics in the dorsal anterior cingulate cortex. PLoS Biol 15:e2003091.  https://doi.org/10.1371/journal.pbio.2003091 CrossRefPubMedPubMedCentralGoogle Scholar
  2. Azab H, Hayden BY (2018) Correlates of economic decisions in the dorsal and subgenual anterior cingulate cortices. Eur J Neursoci 47:979–993.  https://doi.org/10.1111/ejn.13865 CrossRefGoogle Scholar
  3. Blanchard TC, Hayden BY (2015) Monkeys are more patient in a foraging task than in a standard intertemporal choice task. PloS one 10:e0117057.  https://doi.org/10.1371/journal.pone.0117057 CrossRefPubMedPubMedCentralGoogle Scholar
  4. Blanchard TC, Wilke A, Hayden BY (2014a) Hot-hand bias in rhesus monkeys. J Exp Psychol: Anim Learn Cognit 40:280–286.  https://doi.org/10.1037/xan0000033 Google Scholar
  5. Blanchard TC, Wolfe LS, Vlaev I, Winston JS, Hayden BY (2014b) Biases in preferences for sequences of outcomes in monkeys. Cognition 130:289–299.  https://doi.org/10.1016/j.cognition.2013.11.012 CrossRefPubMedPubMedCentralGoogle Scholar
  6. Blanchard TC, Strait CE, Hayden BY (2015) Ramping ensemble activity in dorsal anterior cingulate neurons during persistent commitment to a decision. J Neurophysiol 114:2439–2449.  https://doi.org/10.1152/jn.00711.2015 CrossRefPubMedPubMedCentralGoogle Scholar
  7. Blanchard TC, Piantadosi ST, Hayden BY (2018) Robust mixture modeling reveals category-free selectivity in reward region neuronal ensembles. J Neurophysiol 119:1305–1318.  https://doi.org/10.1152/jn.00808.2017 CrossRefPubMedGoogle Scholar
  8. Brainard DH (1997) The psychophysics toolbox. Spat Vis 10:433–436CrossRefPubMedGoogle Scholar
  9. Britten KH, Shadlen MN, Newsome WT, Movshon JA (1992) The analysis of visual motion: a comparison of neuronal and psychophysical performance. J Neurosci 12:4745–4765CrossRefPubMedGoogle Scholar
  10. Britten KH, Newsome WT, Shadlen MN, Celebrini S, Movshon JA (1996) A relationship between behavioral choice and the visual responses of neurons in macaque MT. Vis Neurosci 13:87–100CrossRefPubMedGoogle Scholar
  11. Cisek P (2012) Making decisions through a distributed consensus. Curr Opin Neurobiol 22:927–936.  https://doi.org/10.1016/j.conb.2012.05.007 CrossRefPubMedGoogle Scholar
  12. Cornelissen FW, Peters EM, Palmer J (2002) The eyelink toolbox: eye tracking with MATLAB and the psychophysics toolbox. Behav Res Methods Instrum Comput 34:613–617CrossRefPubMedGoogle Scholar
  13. de Gee JW, Knapen T, Donner TH (2014) Decision-related pupil dilation reflects upcoming choice and individual bias. Proc Natl Acad Sci USA 111:E618–E625.  https://doi.org/10.1073/pnas.1317557111 CrossRefPubMedGoogle Scholar
  14. Ebitz RB, Hayden BY (2016) Dorsal anterior cingulate: a Rorschach test for cognitive neuroscience. Nat Neurosci 19:1278–1279.  https://doi.org/10.1038/nn.4387 CrossRefPubMedGoogle Scholar
  15. Efron B, Tibshirani RJ (1993) Permutation tests. In: An introduction to the bootstrap. Springer, pp 202–219Google Scholar
  16. Einhauser W, Stout J, Koch C, Carter O (2008) Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry. Proc Natl Acad Sci USA 105:1704–1709.  https://doi.org/10.1073/pnas.0707727105 CrossRefPubMedGoogle Scholar
  17. Einhauser W, Koch C, Carter OL (2010) Pupil dilation betrays the timing of decisions. Front Hum Neurosci 4:18.  https://doi.org/10.3389/fnhum.2010.00018 PubMedPubMedCentralGoogle Scholar
  18. Farashahi S, Azab H, Hayden B, Soltani A (2018) On the flexibility of basic risk attitudes in monkeys. J Neurosci 38:4383–4398.  https://doi.org/10.1523/JNEUROSCI.2260-17.2018 CrossRefPubMedGoogle Scholar
  19. Farovik A, Place RJ, McKenzie S, Porter B, Munro CE, Eichenbaum H (2015) Orbitofrontal cortex encodes memories within value-based schemas and represents contexts that guide memory retrieval. J Neurosci 35:8333–8344.  https://doi.org/10.1523/JNEUROSCI.0134-15.2015 CrossRefPubMedPubMedCentralGoogle Scholar
  20. Freidin E, Aw J, Kacelnik A (2009) Sequential and simultaneous choices: testing the diet selection and sequential choice models. Behav Processes 80:218–223.  https://doi.org/10.1016/j.beproc.2008.12.001 CrossRefGoogle Scholar
  21. Gamlin PD, Zhang H, Harlow A, Barbur JL (1998) Pupil responses to stimulus color, structure and light flux increments in the rhesus monkey. Vis Res 38:3353–3358CrossRefPubMedGoogle Scholar
  22. Geng JJ, Blumenfeld Z, Tyson TL, Minzenberg MJ (2015) Pupil diameter reflects uncertainty in attentional selection during visual search. Front Hum Neurosci 9:435.  https://doi.org/10.3389/fnhum.2015.00435 CrossRefPubMedPubMedCentralGoogle Scholar
  23. Hayden BY (2018) Economic choice: the foraging perspective. Curr Opin Behav Sci 24:1–6.  https://doi.org/10.1016/j.cobeha.2017.12.002 CrossRefGoogle Scholar
  24. Hayden BY, Gallant JL (2013) Working memory and decision processes in visual area v4. Front Neurosci 7:18.  https://doi.org/10.3389/fnins.2013.00018 CrossRefPubMedPubMedCentralGoogle Scholar
  25. Hayden BY, Moreno-Bote R (2018) A neuronal theory of sequential economic choice. Brain Neurosci Adv 2:1–15.  https://doi.org/10.1177/2398212818766675 CrossRefGoogle Scholar
  26. Hoeks B, Levelt WJ (1993) Pupillary dilation as a measure of attention: a quantitative system analysis. Behav Res Methods 25:16–26CrossRefGoogle Scholar
  27. Howard JD, Gottfried JA, Tobler PN, Kahnt T (2015) Identity-specific coding of future rewards in the human orbitofrontal cortex. Proc Natl Acad Sci USA 112:5195–5200.  https://doi.org/10.1073/pnas.1503550112 CrossRefPubMedGoogle Scholar
  28. Hunt LT, Hayden BY (2017) A distributed, hierarchical and recurrent framework for reward-based choice. Nat Rev Neurosci 18:172–182.  https://doi.org/10.1038/nrn.2017.7 CrossRefPubMedPubMedCentralGoogle Scholar
  29. Just MA, Carpenter PA, Miyake A (2003) Neuroindices of cognitive workload: neuroimaging, pupillometric and event-related potential studies of brain work. Theor Issues Ergon Sci 4:56–88CrossRefGoogle Scholar
  30. Kacelnik A, Vasconcelos M, Monteiro T, Aw J (2011) Darwin’s “tug-of-war” vs. starlings’“horse-racing”: how adaptations for sequential encounters drive simultaneous choice. Behav Ecol Sociobiol 65:547–558CrossRefGoogle Scholar
  31. Kahneman D, Beatty J (1966) Pupil diameter and load on memory. Science 154:1583–1585CrossRefPubMedGoogle Scholar
  32. Kahnt T, Heinzle J, Park SQ, Haynes JD (2010) The neural code of reward anticipation in human orbitofrontal cortex. Proc Natl Acad Sci USA 107:6010–6015.  https://doi.org/10.1073/pnas.0912838107 CrossRefPubMedGoogle Scholar
  33. Krajbich I, Armel C, Rangel A (2010) Visual fixations and the computation and comparison of value in simple choice. Nat Neurosci 13:1292–1298.  https://doi.org/10.1038/nn.2635 CrossRefPubMedGoogle Scholar
  34. Lavín C, San Martín R, Jubal ER (2013) Pupil dilation signals uncertainty and surprise in a learning gambling task. Front Behav Neurosci 7Google Scholar
  35. Levy DJ, Glimcher PW (2012) The root of all value: a neural common currency for choice. Curr Opin Neurobiol 22:1027–1038.  https://doi.org/10.1016/j.conb.2012.06.001 CrossRefPubMedPubMedCentralGoogle Scholar
  36. Lui LL, Pasternak T (2011) Representation of comparison signals in cortical area MT during a delayed direction discrimination task. J Neurophysiol 106:1260–1273.  https://doi.org/10.1152/jn.00016.2011 CrossRefPubMedPubMedCentralGoogle Scholar
  37. Machens CK, Romo R, Brody CD (2005) Flexible control of mutual inhibition: a neural model of two-interval discrimination. Science 307:1121–1124.  https://doi.org/10.1126/science.1104171 CrossRefPubMedGoogle Scholar
  38. McNamee D, Liljeholm M, Zika O, O’Doherty JP (2015) Characterizing the associative content of brain structures involved in habitual and goal-directed actions in humans: a multivariate FMRI study. J Neurosci 35:3764–3771.  https://doi.org/10.1523/JNEUROSCI.4677-14.2015 CrossRefPubMedPubMedCentralGoogle Scholar
  39. Miller EK, Desimone R (1994) Parallel neuronal mechanisms for short-term memory. Science 263:520–522CrossRefPubMedGoogle Scholar
  40. Miller EK, Li L, Desimone R (1991) A neural mechanism for working and recognition memory in inferior temporal cortex. Science 254:1377–1379CrossRefPubMedGoogle Scholar
  41. Mirabella G, Bertini G, Samengo I, Kilavik BE, Frilli D, Della Libera C, Chelazzi L (2007) Neurons in area V4 of the macaque translate attended visual features into behaviorally relevant categories. Neuron 54:303–318.  https://doi.org/10.1016/j.neuron.2007.04.007 CrossRefPubMedGoogle Scholar
  42. Nassar MR, Rumsey KM, Wilson RC, Parikh K, Heasly B, Gold JI (2012) Rational regulation of learning dynamics by pupil-linked arousal systems. Nat Neurosci 15:1040–1046.  https://doi.org/10.1038/nn.3130 CrossRefPubMedPubMedCentralGoogle Scholar
  43. O’Doherty JP (2011) Contributions of the ventromedial prefrontal cortex to goal-directed action selection. Ann N Y Acad Sci 1239:118–129.  https://doi.org/10.1111/j.1749-6632.2011.06290.x CrossRefPubMedGoogle Scholar
  44. Pearson JM, Watson KK, Platt ML (2014) Decision making: the neuroethological turn. Neuron 82:950–965.  https://doi.org/10.1016/j.neuron.2014.04.037 CrossRefPubMedPubMedCentralGoogle Scholar
  45. Preuschoff K, Marius’t Hart B, Einhäuser W (2011) Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making. Front Neurosci 5Google Scholar
  46. Rich EL, Wallis JD (2016) Decoding subjective decisions from orbitofrontal cortex. Nat Neurosci 19:973–980.  https://doi.org/10.1038/nn.4320 CrossRefPubMedPubMedCentralGoogle Scholar
  47. Rigato S, Rieger G, Romei V (2016) Multisensory signalling enhances pupil dilation. Sci Rep 6:26188.  https://doi.org/10.1038/srep26188 CrossRefPubMedPubMedCentralGoogle Scholar
  48. Rudebeck PH, Putnam PT, Daniels TE, Yang T, Mitz AR, Rhodes SE, Murray EA (2014) A role for primate subgenual cingulate cortex in sustaining autonomic arousal. Proc Natl Acad Sci USA 111:5391–5396.  https://doi.org/10.1073/pnas.1317695111 CrossRefPubMedGoogle Scholar
  49. Schoenbaum G, Setlow B, Saddoris MP, Gallagher M (2003) Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39:855–867CrossRefPubMedGoogle Scholar
  50. Schultz W (2008) Introduction. Neuroeconomics: the promise and the profit. Philos Trans R Soc Lond B Biol Sci 363:3767–3769.  https://doi.org/10.1098/rstb.2008.0153 CrossRefPubMedPubMedCentralGoogle Scholar
  51. Sleezer BJ, Castagno MD, Hayden BY (2016) Rule Encoding in Orbitofrontal Cortex and Striatum Guides Selection. J Neurosci 36:11223–11237.  https://doi.org/10.1523/JNEUROSCI.1766-16.2016 CrossRefPubMedPubMedCentralGoogle Scholar
  52. Stalnaker TA, Roesch MR, Franz TM, Burke KA, Schoenbaum G (2006) Abnormal associative encoding in orbitofrontal neurons in cocaine-experienced rats during decision-making. Eur J Neurosci 24:2643–2653.  https://doi.org/10.1111/j.1460-9568.2006.05128.x CrossRefPubMedPubMedCentralGoogle Scholar
  53. Strait CE, Blanchard TC, Hayden BY (2014) Reward value comparison via mutual inhibition in ventromedial prefrontal cortex. Neuron 82:1357–1366.  https://doi.org/10.1016/j.neuron.2014.04.032 CrossRefPubMedPubMedCentralGoogle Scholar
  54. Strait CE, Sleezer BJ, Hayden BY (2015) Signatures of value comparison in ventral striatum neurons. PLoS Biol 13:e1002173.  https://doi.org/10.1371/journal.pbio.1002173 CrossRefPubMedPubMedCentralGoogle Scholar
  55. Strait CE, Sleezer BJ, Blanchard TC, Azab H, Castagno MD, Hayden BY (2016) Neuronal selectivity for spatial positions of offers and choices in five reward regions. J Neurophysiol 115:1098–1111.  https://doi.org/10.1152/jn.00325.2015 CrossRefPubMedGoogle Scholar
  56. Suzuki TW, Kunimatsu J, Tanaka M (2016) Correlation between pupil size and subjective passage of time in non-human primates. J Neurosci 36:11331–11337CrossRefPubMedGoogle Scholar
  57. Tsujimoto S, Genovesio A, Wise SP (2012) Neuronal activity during a cued strategy task: comparison of dorsolateral, orbital, and polar prefrontal cortex. J Neurosci 32:11017–11031.  https://doi.org/10.1523/JNEUROSCI.1230-12.2012 CrossRefPubMedPubMedCentralGoogle Scholar
  58. van den Brink RL, Murphy PR, Nieuwenhuis S (2016) Pupil diameter tracks lapses of attention. PloS One 11:e0165274.  https://doi.org/10.1371/journal.pone.0165274 CrossRefPubMedPubMedCentralGoogle Scholar
  59. Varazzani C, San-Galli A, Gilardeau S, Bouret S (2015) Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys. J Neurosci 35:7866–7877.  https://doi.org/10.1523/JNEUROSCI.0454-15.2015 CrossRefPubMedGoogle Scholar
  60. Wallis JD, Rich EL (2011) Challenges of interpreting frontal neurons during value-based decision-making. Front Neurosci 5:124.  https://doi.org/10.3389/fnins.2011.00124 CrossRefPubMedPubMedCentralGoogle Scholar
  61. Wang MZ, Hayden BY (2017) Reactivation of associative structure specific outcome responses during prospective evaluation in reward-based choices. Nat Commun 8:15821.  https://doi.org/10.1038/ncomms15821 CrossRefPubMedPubMedCentralGoogle Scholar
  62. Xie J, Frazier PI, Chick SE (2016) Bayesian optimization via simulation with pairwise sampling and correlated prior beliefs. Oper Res 64:542–559.  https://doi.org/10.1287/opre.2016.1480 CrossRefGoogle Scholar
  63. Yamada H, Tymula A, Louie K, Glimcher PW (2013) Thirst-dependent risk preferences in monkeys identify a primitive form of wealth. Proc Natl Acad Sci USA 110:15788–15793.  https://doi.org/10.1073/pnas.1308718110 CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Neuroscience and Center for Magnetic Resonance ResearchUniversity of MinnesotaMinneapolisUSA
  2. 2.Department of Brain and Cognitive Sciences and Center for Visual Sciences, Center for the Origins of CognitionUniversity of RochesterRochesterUSA

Personalised recommendations