Advertisement

Psychopharmacology

, Volume 236, Issue 8, pp 2425–2435 | Cite as

Depressive symptoms bias the prediction-error enhancement of memory towards negative events in reinforcement learning

  • Nina RouhaniEmail author
  • Yael Niv
Original Investigation

Abstract

Rationale

Depression is a disorder characterized by sustained negative affect and blunted positive affect, suggesting potential abnormalities in reward learning and its interaction with episodic memory.

Objectives

This study investigated how reward prediction errors experienced during learning modulate memory for rewarding events in individuals with depressive and non-depressive symptoms.

Methods

Across three experiments, participants learned the average values of two scene categories in two learning contexts. Each learning context had either high or low outcome variance, allowing us to test the effects of small and large prediction errors on learning and memory. Participants were later tested for their memory of trial-unique scenes that appeared alongside outcomes. We compared learning and memory performance of individuals with self-reported depressive symptoms (N = 101) to those without (N = 184).

Results

Although there were no overall differences in reward learning between the depressive and non-depressive group, depression severity within the depressive group predicted greater error in estimating the values of the scene categories. Similarly, there were no overall differences in memory performance. However, in depressive participants, negative prediction errors enhanced episodic memory more so than did positive prediction errors, and vice versa for non-depressive participants who showed a larger effect of positive prediction errors on memory. These results reflected differences in memory both within group and across groups.

Conclusions

Individuals with self-reported depressive symptoms showed relatively intact reinforcement learning, but demonstrated a bias for encoding events that accompanied surprising negative outcomes versus surprising positive ones. We discuss a potential neural mechanism supporting these effects, which may underlie or contribute to the excessive negative affect observed in depression.

Keywords

Depression Memory Reinforcement learning Reward Predictions errors Surprise 

Notes

Funding information

This work was supported by grant W911NF-14-1-0101 from the Army Research Office (Y.N.), the Ellison Foundation (Y.N.), grant R01MH098861 from the National Institute for Mental Health (Y.N.), and the National Science Foundation’s Graduate Research Fellowship Program (N.R.).

Compliance with ethical standards

Participants completed informed consent online and were required to correctly answer questions checking for their understanding of the task before proceeding; procedures were approved by Princeton University’s Institutional Review Board.

Conflict of interest

The authors declare that they have no conflict of interest.

References

  1. Admon R, Pizzagalli DA (2015) Dysfunctional reward processing in depression. Curr Opin Psychol 4:114–118.  https://doi.org/10.1016/j.copsyc.2014.12.011 CrossRefGoogle Scholar
  2. Admon R, Nickerson LD, Dillon DG, Holmes AJ, Bogdan R, Kumar P, Dougherty DD, Iosifescu DV, Mischoulon D, Fava M, Pizzagalli DA (2015) Dissociable cortico-striatal connectivity abnormalities in major depression in response to monetary gains and penalties. Psychol Med 45(1):121–131.  https://doi.org/10.1017/S0033291714001123 CrossRefGoogle Scholar
  3. Barto AG (1995) Adaptive critic and the basal ganglia. In: Houk JC, Davis JL, Beiser DG (eds) Models of information processing in the basal ganglia. MIT press, p 215Google Scholar
  4. Bates D, Maechler M, Bolker B, Walker S, Christensen RHB, Singmann H, … Grothendieck G (2015) Fitting linear mixed-effects models using lme4. J Stat Softw 67(1):1–48.  https://doi.org/10.18637/jss.v067.i01
  5. Bornstein AM, Khaw MW, Shohamy D, Daw ND (2017) Reminders of past choices bias decisions for reward in humans. Nat Commun 8:15958.  https://doi.org/10.1038/ncomms15958 CrossRefGoogle Scholar
  6. Burt DB, Zembar MJ, Niederehe G (1995) Depression and memory impairment: a meta-analysis of the association, its pattern, and specificity. Psychol Bull 117(2):285–305.  https://doi.org/10.1037/0033-2909.117.2.285 CrossRefGoogle Scholar
  7. Chase HW, Frank MJ, Michael A, Bullmore ET, Sahakian BJ, Robbins TW (2010) Approach and avoidance learning in patients with major depression and healthy controls: relation to anhedonia. Psychol Med 40(3):433–440.  https://doi.org/10.1017/S0033291709990468 CrossRefGoogle Scholar
  8. Clewett D, Huang R, Velasco R, Lee T-H, Mather M (2018) Locus coeruleus activity strengthens prioritized memories under arousal. J Neurosci 2097–17.  https://doi.org/10.1523/JNEUROSCI.2097-17.2017
  9. Dillon DG, Pizzagalli DA (2018) Mechanisms of memory disruption in depression. Trends Neurosci 41:137–149.  https://doi.org/10.1016/j.tins.2017.12.006 CrossRefGoogle Scholar
  10. Dillon DG, Dobbins IG, Pizzagalli DA (2014) Weak reward source memory in depression reflects blunted activation of VTA/SN and parahippocampus. Soc Cogn Affect Neurosci 9(10):1576–1583 Retrieved from http://scan.oxfordjournals.org/. Accessed 31 Oct 2018CrossRefGoogle Scholar
  11. Duncan KD, Shohamy D (2016) Memory states influence value-based decisions. J Exp Psychol Gen 145(9):3–9Google Scholar
  12. Duszkiewicz AJ, Mcnamara CG, Takeuchi T, Genzel L (2018) Novelty and dopaminergic modulation of memory persistence: a tale of two systems. Trends Neurosci.  https://doi.org/10.1016/j.tins.2018.10.002
  13. Gaddy MA, Ingram RE (2014) A meta-analytic review of mood-congruent implicit memory in depressed mood. Clin Psychol Rev 34:402–416.  https://doi.org/10.1016/j.cpr.2014.06.001 CrossRefGoogle Scholar
  14. Gershman SJ, Daw ND (2017) Reinforcement learning and episodic memory in humans and animals: an integrative framework. Annu Rev Psychol 68:101–128.  https://doi.org/10.1146/annurev-psych-122414-033625 CrossRefGoogle Scholar
  15. Gotlib IH, Joormann J (2010) Cognition and depression: current status and future directions. SSRN.  https://doi.org/10.1146/annurev.clinpsy.121208.131305
  16. Gradin VB, Kumar P, Waiter G, Ahearn T, Stickle C, Milders M, Reid I, Hall J, Steele JD (2011) Expected value and prediction error abnormalities in depression and schizophrenia. Brain 134(6):1751–1764.  https://doi.org/10.1093/brain/awr059 CrossRefGoogle Scholar
  17. Huys QJ, Pizzagalli DA, Bogdan R, Dayan P (2013) Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis. Biol Mood Anxiety Disord 3(1):12.  https://doi.org/10.1186/2045-5380-3-12 CrossRefGoogle Scholar
  18. Joormann J, Quinn ME (2014) Cognitive processes and emotion regulation in depression. Depress Anxiety 31(4):308–315.  https://doi.org/10.1002/da.22264 CrossRefGoogle Scholar
  19. Kempadoo KA, Mosharov EV, Choi SJ, Sulzer D, Kandel ER (2016) Dopamine release from the locus coeruleus to the dorsal hippocampus promotes spatial learning and memory. Proc Natl Acad Sci 113(51):14835–14840.  https://doi.org/10.1073/pnas.1616515114 CrossRefGoogle Scholar
  20. Knutson B, Bhanji JP, Cooney RE, Atlas LY, Gotlib IH (2008) Neural responses to monetary incentives in major depression. Biol Psychiatry 63(7):686–692.  https://doi.org/10.1016/j.biopsych.2007.07.023 CrossRefGoogle Scholar
  21. Kumar P, Goer F, Murray L, Dillon DG, Beltzer ML, Cohen AL, Brooks NH, Pizzagalli DA (2018) Impaired reward prediction error encoding and striatal-midbrain connectivity in depression. Neuropsychopharmacology 43(7):1581–1588.  https://doi.org/10.1038/s41386-018-0032-x CrossRefGoogle Scholar
  22. Leal SL, Tighe SK, Jones CK, Yassa MA (2014) Pattern separation of emotional information in hippocampal dentate and CA3. Hippocampus 24(9):1146–1155.  https://doi.org/10.1002/hipo.22298 CrossRefGoogle Scholar
  23. Matsumoto M, Hikosaka O (2007) Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447(7148):1111–1115.  https://doi.org/10.1038/nature05860 CrossRefGoogle Scholar
  24. Matt GE, Vázquez C, Campbell WK (1992) Mood-congruent recall of affectively toned stimuli: a meta-analytic review. Clin Psychol Rev 12(2):227–255.  https://doi.org/10.1016/0272-7358(92)90116-P CrossRefGoogle Scholar
  25. Mueller EM, Pechtel P, Cohen AL, Douglas SR, Pizzagalli DA (2015) Potentiated processing of negative feedback in depression is attenuated by anhedonia. Depress Anxiety 32(4):296–305.  https://doi.org/10.1002/da.22338 CrossRefGoogle Scholar
  26. Nassar MR, Rumsey KM, Wilson RC, Parikh K, Heasly B, Gold JI (2012) Rational regulation of learning dynamics by pupil-linked arousal systems. Nat Neurosci 15(7):1040–1046.  https://doi.org/10.1038/nn.3130 CrossRefGoogle Scholar
  27. Pizzagalli DA (2014) Depression, stress, and anhedonia: toward a synthesis and integrated model MDD: major depressive disorder. Annu Rev Clin Psychol 10:393–423.  https://doi.org/10.1146/annurev-clinpsy-050212-185606 CrossRefGoogle Scholar
  28. Pollack I, Norman DA (1964) A non-parametric analysis of recognition experiments. Psychon Sci 1(1–12):125–126.  https://doi.org/10.3758/BF03342823 CrossRefGoogle Scholar
  29. Read Montague P, Dayan P, Sejnowskw TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 76. Retrieved from http://www.jneurosci.org/content/jneuro/16/5/1936.full.pdf. Accessed 31 Oct 2018
  30. Roesch MR, Esber GR, Li J, Daw ND, Schoenbaum G (2012) Surprise! Neural correlates of Pearce-Hall and Rescorla-Wagner coexist within the brain. Eur J Neurosci 35(7):1190–1200.  https://doi.org/10.1111/j.1460-9568.2011.07986.x CrossRefGoogle Scholar
  31. Rouhani N, Norman KA, Niv Y (2018) Dissociable effects of surprising rewards on learning and memory. J Exp Psychol Learn Mem Cogn.  https://doi.org/10.1037/xlm0000518
  32. Rush AJ, Gullion CM, Basco MR, Jarrett RB, Trivedi MH (1996) The inventory of depressive symptomatology (IDS): psychometric properties. Psychol Med 26(03):477–486.  https://doi.org/10.1017/S0033291700035558 CrossRefGoogle Scholar
  33. Rutledge RB, Moutoussis M, Smittenaar P, Zeidman P, Taylor T, Hrynkiewicz L, Lam J, Skandali N, Siegel JZ, Ousdal OT, Prabhu G, Dayan P, Fonagy P, Dolan RJ (2017) Association of neural and emotional impacts of reward prediction errors with major depression. JAMA Psychiatry 74:10–12.  https://doi.org/10.1001/jamapsychiatry.2017.1713 CrossRefGoogle Scholar
  34. Sacchet MD, Levy BJ, Hamilton JP, Maksimovskiy A, Hertel PT, Joormann J, Anderson MC, Wagner AD, Gotlib IH (2017) Cognitive and neural consequences of memory suppression in major depressive disorder. Cogn Affect Behav Neurosci 17(1):77–93.  https://doi.org/10.3758/s13415-016-0464-x CrossRefGoogle Scholar
  35. Sara SJ (2009) The locus coeruleus and noradrenergic modulation of cognition. Nat Rev Neurosci 10:211–223.  https://doi.org/10.1038/nrn2573 CrossRefGoogle Scholar
  36. Shohamy D, Adcock RA (2010) Dopamine and adaptive memory. Trends Cogn Sci 14:464–472.  https://doi.org/10.1016/j.tics.2010.08.002 CrossRefGoogle Scholar
  37. Smoski MJ, Felder J, Bizzell J, Green SR, Ernst M, Lynch TR, Dichter GS (2009) fMRI of alterations in reward selection, anticipation, and feedback in major depressive disorder. J Affect Disord 118:69–78.  https://doi.org/10.1016/j.jad.2009.01.034 CrossRefGoogle Scholar
  38. Takeuchi T, Duszkiewicz AJ, Sonneborn A, Spooner PA, Yamasaki M, Watanabe M, Smith CC, Fernández G, Deisseroth K, Greene RW, Morris RGM (2016) Locus coeruleus and dopaminergic consolidation of everyday memory. Nature 537(7620):1–18.  https://doi.org/10.1038/nature19325 CrossRefGoogle Scholar
  39. Ubl B, Kuehner C, Kirsch P, Ruttorf M, Diener C, Flor H (2015) Altered neural reward and loss processing and prediction error signalling in depression. Soc Cogn Affect Neurosci 10(8):1102–1112.  https://doi.org/10.1093/scan/nsu158 CrossRefGoogle Scholar
  40. Walters K, Christakis DA, Wright DR (2018) Are mechanical Turk worker samples representative of health status and health behaviors in the U.S.? PLoS ONE 13(6).  https://doi.org/10.1371/journal.pone.0198835
  41. Weber EU, Blais A-R, Betz NE (2002) A domain-specific risk-attitude scale: measuring risk perceptions and risk behaviors. J Behav Decis Mak 15(4):263–290.  https://doi.org/10.1002/bdm.414 CrossRefGoogle Scholar
  42. Whitton AE, Kakani P, Foti D, Van’t Veer A, Haile A, Crowley DJ, Pizzagalli DA (2016) Blunted neural responses to reward in remitted major depression: a high-density event-related potential study. Biol Psychiatry Cogn Neurosci Neuroimaging 1(1):87–95.  https://doi.org/10.1016/j.bpsc CrossRefGoogle Scholar
  43. Wimmer GE, Shohamy D (2012) Preference by association: how memory mechanisms in the hippocampus bias decisions. Science 338:270–273.  https://doi.org/10.1126/science.1223252 CrossRefGoogle Scholar
  44. Yang Y, Cui Y, Sang K, Dong Y, Ni Z, Ma S, Hu H (2018) Ketamine blocks bursting in the lateral habenula to rapidly relieve depression. Nature 554(7692):317–322.  https://doi.org/10.1038/nature25509 CrossRefGoogle Scholar
  45. Young KD, Siegle GJ, Bodurka J, Drevets WC (2016) Amygdala activity during autobiographical memory recall in depressed and vulnerable individuals: association with symptom severity and autobiographical overgenerality. Am J Psychiatry 173:78–89.  https://doi.org/10.1176/appi.ajp.2015.15010119 CrossRefGoogle Scholar
  46. Young KD, Siegle GJ, Zotev V, Phillips R, Misaki M, Yuan H, Drevets WC, Bodurka J (2017) Randomized clinical trial of real-time fMRI amygdala neurofeedback for major depressive disorder: effects on symptoms and autobiographical memory recall. Am J Psychiatry 174:748–755.  https://doi.org/10.1176/appi.ajp.2017.16060637 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of PsychologyPrinceton UniversityPrincetonUSA
  2. 2.Princeton Neuroscience InstitutePrinceton UniversityPrincetonUSA

Personalised recommendations