Depressive symptoms bias the prediction-error enhancement of memory towards negative events in reinforcement learning
Depression is a disorder characterized by sustained negative affect and blunted positive affect, suggesting potential abnormalities in reward learning and its interaction with episodic memory.
This study investigated how reward prediction errors experienced during learning modulate memory for rewarding events in individuals with depressive and non-depressive symptoms.
Across three experiments, participants learned the average values of two scene categories in two learning contexts. Each learning context had either high or low outcome variance, allowing us to test the effects of small and large prediction errors on learning and memory. Participants were later tested for their memory of trial-unique scenes that appeared alongside outcomes. We compared learning and memory performance of individuals with self-reported depressive symptoms (N = 101) to those without (N = 184).
Although there were no overall differences in reward learning between the depressive and non-depressive group, depression severity within the depressive group predicted greater error in estimating the values of the scene categories. Similarly, there were no overall differences in memory performance. However, in depressive participants, negative prediction errors enhanced episodic memory more so than did positive prediction errors, and vice versa for non-depressive participants who showed a larger effect of positive prediction errors on memory. These results reflected differences in memory both within group and across groups.
Individuals with self-reported depressive symptoms showed relatively intact reinforcement learning, but demonstrated a bias for encoding events that accompanied surprising negative outcomes versus surprising positive ones. We discuss a potential neural mechanism supporting these effects, which may underlie or contribute to the excessive negative affect observed in depression.
KeywordsDepression Memory Reinforcement learning Reward Predictions errors Surprise
This work was supported by grant W911NF-14-1-0101 from the Army Research Office (Y.N.), the Ellison Foundation (Y.N.), grant R01MH098861 from the National Institute for Mental Health (Y.N.), and the National Science Foundation’s Graduate Research Fellowship Program (N.R.).
Compliance with ethical standards
Participants completed informed consent online and were required to correctly answer questions checking for their understanding of the task before proceeding; procedures were approved by Princeton University’s Institutional Review Board.
Conflict of interest
The authors declare that they have no conflict of interest.
- Admon R, Nickerson LD, Dillon DG, Holmes AJ, Bogdan R, Kumar P, Dougherty DD, Iosifescu DV, Mischoulon D, Fava M, Pizzagalli DA (2015) Dissociable cortico-striatal connectivity abnormalities in major depression in response to monetary gains and penalties. Psychol Med 45(1):121–131. https://doi.org/10.1017/S0033291714001123 CrossRefGoogle Scholar
- Barto AG (1995) Adaptive critic and the basal ganglia. In: Houk JC, Davis JL, Beiser DG (eds) Models of information processing in the basal ganglia. MIT press, p 215Google Scholar
- Bates D, Maechler M, Bolker B, Walker S, Christensen RHB, Singmann H, … Grothendieck G (2015) Fitting linear mixed-effects models using lme4. J Stat Softw 67(1):1–48. https://doi.org/10.18637/jss.v067.i01
- Clewett D, Huang R, Velasco R, Lee T-H, Mather M (2018) Locus coeruleus activity strengthens prioritized memories under arousal. J Neurosci 2097–17. https://doi.org/10.1523/JNEUROSCI.2097-17.2017
- Duncan KD, Shohamy D (2016) Memory states influence value-based decisions. J Exp Psychol Gen 145(9):3–9Google Scholar
- Duszkiewicz AJ, Mcnamara CG, Takeuchi T, Genzel L (2018) Novelty and dopaminergic modulation of memory persistence: a tale of two systems. Trends Neurosci. https://doi.org/10.1016/j.tins.2018.10.002
- Gershman SJ, Daw ND (2017) Reinforcement learning and episodic memory in humans and animals: an integrative framework. Annu Rev Psychol 68:101–128. https://doi.org/10.1146/annurev-psych-122414-033625 CrossRefGoogle Scholar
- Gotlib IH, Joormann J (2010) Cognition and depression: current status and future directions. SSRN. https://doi.org/10.1146/annurev.clinpsy.121208.131305
- Pizzagalli DA (2014) Depression, stress, and anhedonia: toward a synthesis and integrated model MDD: major depressive disorder. Annu Rev Clin Psychol 10:393–423. https://doi.org/10.1146/annurev-clinpsy-050212-185606 CrossRefGoogle Scholar
- Read Montague P, Dayan P, Sejnowskw TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 76. Retrieved from http://www.jneurosci.org/content/jneuro/16/5/1936.full.pdf. Accessed 31 Oct 2018
- Rouhani N, Norman KA, Niv Y (2018) Dissociable effects of surprising rewards on learning and memory. J Exp Psychol Learn Mem Cogn. https://doi.org/10.1037/xlm0000518
- Rutledge RB, Moutoussis M, Smittenaar P, Zeidman P, Taylor T, Hrynkiewicz L, Lam J, Skandali N, Siegel JZ, Ousdal OT, Prabhu G, Dayan P, Fonagy P, Dolan RJ (2017) Association of neural and emotional impacts of reward prediction errors with major depression. JAMA Psychiatry 74:10–12. https://doi.org/10.1001/jamapsychiatry.2017.1713 CrossRefGoogle Scholar
- Sacchet MD, Levy BJ, Hamilton JP, Maksimovskiy A, Hertel PT, Joormann J, Anderson MC, Wagner AD, Gotlib IH (2017) Cognitive and neural consequences of memory suppression in major depressive disorder. Cogn Affect Behav Neurosci 17(1):77–93. https://doi.org/10.3758/s13415-016-0464-x CrossRefGoogle Scholar
- Walters K, Christakis DA, Wright DR (2018) Are mechanical Turk worker samples representative of health status and health behaviors in the U.S.? PLoS ONE 13(6). https://doi.org/10.1371/journal.pone.0198835
- Whitton AE, Kakani P, Foti D, Van’t Veer A, Haile A, Crowley DJ, Pizzagalli DA (2016) Blunted neural responses to reward in remitted major depression: a high-density event-related potential study. Biol Psychiatry Cogn Neurosci Neuroimaging 1(1):87–95. https://doi.org/10.1016/j.bpsc CrossRefGoogle Scholar
- Young KD, Siegle GJ, Bodurka J, Drevets WC (2016) Amygdala activity during autobiographical memory recall in depressed and vulnerable individuals: association with symptom severity and autobiographical overgenerality. Am J Psychiatry 173:78–89. https://doi.org/10.1176/appi.ajp.2015.15010119 CrossRefGoogle Scholar
- Young KD, Siegle GJ, Zotev V, Phillips R, Misaki M, Yuan H, Drevets WC, Bodurka J (2017) Randomized clinical trial of real-time fMRI amygdala neurofeedback for major depressive disorder: effects on symptoms and autobiographical memory recall. Am J Psychiatry 174:748–755. https://doi.org/10.1176/appi.ajp.2017.16060637 CrossRefGoogle Scholar