Skip to main content
SpringerLink
Log in

Feedback Discounting in Probabilistic Categorization: Converging Evidence from EEG and Cognitive Modeling

  • Published:
Computational Brain & Behavior Aims and scope Submit manuscript

Abstract

In simple probabilistic learning environments, the informational value of corrective feedback gradually declines over time. This is because prediction errors persist despite learners acquiring the contingencies between stimuli and outcomes. An adaptive solution to the problem of unavoidable prediction error is to discount feedback from the learning environment. We provide novel neural evidence of feedback discounting using a combination of behavioral modeling and electroencephalography (EEG). Participants completed a probabilistic categorization task while EEG activity was recorded. We used a model-based analysis of choice behavior to identify individuals that did and did not discount feedback. We then contrasted changes in the feedback-related negativity (FRN) for these two groups. For individuals who did not discount feedback, we observed learning-related reductions in the FRN that reflected incremental changes in choice behavior. By contrast, for individuals who discounted feedback, we found that the FRN was effectively eliminated due to the rapid onset of feedback discounting. The use of a feedback discounting strategy was linked to superior performance on the task, highlighting the adaptive nature of discounting when trial-to-trial outcomes are variable, but the long-term contingencies relating cues and outcomes are stable.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Throughout this article, we follow convention and use the term “amplitude” as a shorthand to describe the signed voltage difference relative to some reference level.

References

  • Arbel, Y., & Wu, H. (2016). A neurophysiological examination of quality of learning in a feedback-based learning task. Neuropsychologia, 93, 13–20.

    Article  Google Scholar 

  • Bellebaum, C., & Daum, I. (2008). Learning-related changes in reward expectancy are reflected in the feedback-related negativity. European Journal of Neuroscience, 27, 1823–1835.

    Article  Google Scholar 

  • Bellebaum, C., Kobza, S., Thiele, S., & Daum, I. (2010). It was not MY fault: event-related brain potentials in active and observational learning from feedback. Cerebral Cortex, 20, 2874–2883.

    Article  Google Scholar 

  • Blair, M., & Homa, D. L. (2005). Integrating novel dimensions to eliminate category exceptions: when more is less. Journal of Experimental Psychology: Learning, Memory, & Cognition, 31, 258–271.

    Google Scholar 

  • Bland, A. R., & Schaefer, A. (2011). Electrophysiological correlates of decision making under varying levels of uncertainty. Brain Research, 1417, 55–66.

    Article  Google Scholar 

  • Bode, S., Bennett, D., Stahl, J., & Murawski, C. (2014). Distributed patterns of event-related potentials predict subsequent ratings of abstract stimulus attributes. PLoS One, 9, e109070.

    Article  Google Scholar 

  • Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 433–436.

    Article  Google Scholar 

  • Castellan, N. J. (1973). Multiple-cue probability learning with irrelevant cues. Organizational Behavior & Human Performance, 9, 16–29.

    Article  Google Scholar 

  • Chase, H. W., Swainson, R., Durham, L., Benham, L., & Cools, R. (2011). Feedback-related negativity codes prediction error but not behavioral adjustment during probabilistic reversal learning. Journal of Cognitive Neuroscience, 23, 936–946.

    Article  Google Scholar 

  • Cohen, M. X., Elger, C. E., & Ranganath, C. (2007). Reward expectation modulates feedback-related negativity and EEG spectra. NeuroImage, 35(2), 968–978.

    Article  Google Scholar 

  • Cohen, M. X., Wilmes, K. A., & van de Vijver, I. (2011). Cortical electrophysiological network dynamics of feedback learning. Trends in Cognitive Sciences, 15, 558–566.

    Article  Google Scholar 

  • Craig, S., & Lewandowsky, S. (2012). Whichever way you choose to categorize, working memory helps you learn. Quarterly Journal of Experimental Psychology, 65, 439–464.

    Article  Google Scholar 

  • Craig, S., & Lewandowsky, S. (2013). Working memory supports inference learning just like classification learning. Quarterly Journal of Experimental Psychology, 66, 1493–1503.

    Article  Google Scholar 

  • Craig, S., Lewandowsky, S., & Little, D. R. (2011). Error discounting in probabilistic category learning. Journal of Experimental Psychology: Learning, Memory, & Cognition, 37, 673–687.

    Google Scholar 

  • Delorme, A., & Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134, 9–21.

    Article  Google Scholar 

  • Edgell, S. E. (1983). Delayed exposure to configural information in nonmetric multiple-cue probability learning. Organizational Behavior & Human Decision Processes, 32, 55–65.

    Article  Google Scholar 

  • Edgell, S. E., & Morrissey, J. M. (1987). Delayed exposure to additional relevant information in nonmetric multiple-cue probability learning. Organizational Behavior & Human Decision Processes, 40, 22–38.

    Article  Google Scholar 

  • Edwards, W. (1961). Probability learning in 1000 trials. Journal of Experimental Psychology, 62, 385–394.

    Article  Google Scholar 

  • Eppinger, B., Kray, J., Mock, B., & Mecklinger, A. (2008). Better or worse than expected? Aging, learning, and the ERN. Neuropsychologia, 46, 521–539.

    Article  Google Scholar 

  • Erickson, M. A., & Kruschke, J. K. (1998). Rules and exemplars in category learning. Journal of Experimental Psychology: General, 127, 107–140.

    Article  Google Scholar 

  • Falkenstein, M., Hohnsbein, J., Hoormann, J., & Blanke, L. (1991). Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. Electroencephalography & Clinical Neurophysiology, 78, 447–455.

    Article  Google Scholar 

  • Forstmann, B. U., Wagenmakers, E. J., Eichele, T., Brown, S., & Serences, J. T. (2011). Reciprocal relations between cognitive neuroscience and formal cognitive models: opposites attract? Trends in Cognitive Sciences, 15, 272–279.

    Article  Google Scholar 

  • Friedman, D., & Massaro, D. W. (1998). Understanding variability in binary and continuous choice. Psychonomic Bulletin & Review, 5, 370–389.

    Article  Google Scholar 

  • Gehring, W. J., & Willoughby, A. R. (2002). The medial frontal cortex and the rapid processing of monetary gains and losses. Science, 295, 2279–2282.

    Article  Google Scholar 

  • Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E., & Donchin, E. (1993). A neural system for error detection and compensation. Psychological Science, 4, 385–390.

    Article  Google Scholar 

  • Hajcak, G., Moser, J. S., Holroyd, C. B., & Simons, R. F. (2006). The feedback-related negativity reflects the binary evaluation of good versus bad outcomes. Biological Psychology, 71, 148–154.

    Article  Google Scholar 

  • Hajcak, G., Moser, J. S., Holroyd, C. B., & Simons, R. F. (2007). It’s worse than you thought: the feedback negativity and violations of reward prediction in gambling tasks. Psychophysiology, 44, 905–912.

    Article  Google Scholar 

  • Holroyd, C. B., & Coles, M. G. H. (2002). The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679–709.

    Article  Google Scholar 

  • Holroyd, C. B., Krigolson, O. E., Baker, R., Lee, S., & Gibson, J. (2009). When is an error not a prediction error? An electrophysiological investigation. Cognitive, Affective, & Behavioral Neuroscience, 9, 59–70.

    Article  Google Scholar 

  • Ichikawa, N., Siegle, G. J., Dombrovski, A., & Ohira, H. (2010). Subjective and model-estimated reward prediction: association with the feedback-related negativity (FRN) and reward prediction error in a reinforcement learning task. International Journal of Psychophysiology, 78, 273–283.

    Article  Google Scholar 

  • Kalish, M. L., Newell, B. R., & Dunn, J. C. (2017). More is generally better: higher working memory capacity does not impair perceptual category learning. Journal of Experimental Psychology: Learning, Memory, & Cognition, 43, 503–514.

    Google Scholar 

  • Kamin, L. J. (1969). Predictability, surprise, attention, and conditioning. In R. M. Church & B. A. Campbell (Eds.), Punishment and aversive behavior (pp. 279–296). New York: Appleton-Century-Crofts.

    Google Scholar 

  • Krigolson, O. E., Pierce, L. J., Holroyd, C. B., & Tanaka, J. W. (2009). Learning to become an expert: reinforcement learning and the acquisition of perceptual expertise. Journal of Cognitive Neuroscience, 21, 1834–1841.

    Article  Google Scholar 

  • Kruschke, J. K., & Johansen, M. K. (1999). A model of probabilistic category learning. Journal of Experimental Psychology: Learning, Memory, & Cognition, 25, 1083–1119.

    Google Scholar 

  • Lagnado, D. A., Newell, B. R., Kahan, S., & Shanks, D. R. (2006). Insight and strategy in multiple-cue learning. Journal of Experimental Psychology: General, 135, 162–183.

    Article  Google Scholar 

  • Lewandowsky, S. (2011). Working memory capacity and categorization: individual differences and modeling. Journal of Experimental Psychology: Learning, Memory & Cognition, 37, 720–738.

    Google Scholar 

  • Lewandowsky, S., Yang, L.-X., Newell, B. R., & Kalish, M. L. (2012). Working memory does not dissociate between different perceptual categorization tasks. Journal of Experimental Psychology: Learning, Memory, & Cognition, 38, 881–904.

    Google Scholar 

  • Little, D. R., & Lewandowsky, S. (2012). Multiple-cue probability learning. In N. Seel (Ed.), Encyclopedia of the learning sciences (pp. 2386–2388). New York: Springer.

    Google Scholar 

  • Little, J. L., & McDaniel, M. A. (2015). Individual differences in category learning: memorization versus rule abstraction. Memory & Cognition, 43, 283–297.

    Article  Google Scholar 

  • Lopez-Calderon, J., & Luck, S. J. (2014). ERPLAB: an open-source toolbox for the analysis of event-related potentials. Frontiers in Human Neuroscience, 8, 1–14.

    Article  Google Scholar 

  • Love, B. C., & Gureckis, T. M. (2007). Models in search of a brain. Cognitive, Affective, & Behavioral Neuroscience, 7, 90–108.

    Article  Google Scholar 

  • Luce, R. D. (1963). Detection and recognition. In R. D. Luce, R. R. Bush, & E. Galanter (Eds.), Handbook of mathematical psychology (pp. 103–189). New York: Wiley.

    Google Scholar 

  • Luck, S. J. (2005). An introduction to the event-related potential technique. Cambridge: MIT Press.

    Google Scholar 

  • Luft, C. D. B. (2014). Learning from feedback: the neural mechanisms of feedback processing facilitating better performance. Behavioral Brain Research, 261, 356–368.

    Article  Google Scholar 

  • Luque, D., López, F. J., Marco-Pallares, J., Càmara, E., & Rodríguez-Fornells, A. (2012). Feedback-related brain potential activity complies with basic assumptions of associative learning theory. Journal of Cognitive Neuroscience, 24, 794–808.

    Article  Google Scholar 

  • McDaniel, M. A., Cahill, M. J., Robbins, M., & Wiener, C. (2014). Individual differences in learning and transfer: stable tendencies for learning exemplars versus abstracting rules. Journal of Experimental Psychology: General, 143, 668–693.

    Article  Google Scholar 

  • Miltner, W. H. R., Braun, C. H., & Coles, M. G. H. (1997). Event-related brain potentials following incorrect feedback in a time-estimation task: evidence for a “generic” neural system for error detection. Journal of Cognitive Neuroscience, 9, 788–798.

    Article  Google Scholar 

  • Navarro, D. J., & Newell, B. R. (2014). Information versus reward in a changing world. In P. Bello, M. Guarini, M. McShane, & B. Scassellati (Eds.), Proceedings of the 36th annual conference of the cognitive science society (pp. 1054–1059).

    Google Scholar 

  • Navarro, D. J., Newell, B. R., & Schulze, C. (2016). Learning and choosing in an uncertain world: an investigation of the explore-exploit dilemma in static and dynamic environments. Cognitive Psychology, 85, 43–77.

    Article  Google Scholar 

  • Nieuwenhuis, S., Holroyd, C. B., Mol, N., & Coles, M. G. H. (2004a). Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance. Neuroscience & Biobehavioral Reviews, 28, 441–448.

    Article  Google Scholar 

  • Nieuwenhuis, S., Yeung, N., Holroyd, C. B., Schurger, A., & Cohen, J. D. (2004b). Sensitivity of electrophysiological activity from medial frontal cortex to utilitarian and performance feedback. Cerebral Cortex, 14, 741–747.

    Article  Google Scholar 

  • Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision, 10, 437–442.

    Article  Google Scholar 

  • Pietschmann, M., Simon, K., Endrass, T., & Kathmann, N. (2008). Changes of performance monitoring with learning in older and younger adults. Psychophysiology, 45, 559–568.

    Article  Google Scholar 

  • Potts, G. F., Martin, L. E., Kamp, S.-M., & Donchin, E. (2011). Neural response to action and reward prediction errors: comparing the error-related negativity to behavioral errors and the feedback-related negativity to reward prediction violations. Psychophysiology, 48, 218–228.

    Article  Google Scholar 

  • Rakow, T., Newell, B. R., & Zougkou, K. (2010). The role of working memory in information acquisition and decision making: lessons from the binary prediction task. Quarterly Journal of Experimental Psychology, 63, 1335–1360.

    Article  Google Scholar 

  • Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: current research and theory (pp. 64–99). NewYork: Appleton-Century-Crofts.

    Google Scholar 

  • Sailer, U., Fischmeister, F. P. S., & Bauer, H. (2010). Effects of learning on feedback-related brain potentials in a decision-making task. Brain Research, 1342, 85–93.

    Article  Google Scholar 

  • Sambrook, T. D., & Goslin, J. (2015). A neural reward prediction error revealed by a meta-analysis of ERPs using great grand averages. Psychological Bulletin, 141, 213–235.

    Article  Google Scholar 

  • Schiffer, A.-M., Siletti, K., Waszak, F., & Yeung, N. (2017). Adaptive behaviour and feedback processing integrate experience and instruction in reinforcement learning. NeuroImage, 146, 626–641.

    Article  Google Scholar 

  • Schulze, C., & Newell, B. R. (2016). Taking the easy way out? Increasing implementation effort reduces probability maximizing under cognitive load. Memory & Cognition, 44, 806–818.

    Article  Google Scholar 

  • Sewell, D. K., & Lewandowsky, S. (2012). Attention and working memory capacity: insights from blocking, highlighting, and knowledge restructuring. Journal of Experimental Psychology: General, 141, 444–469.

    Article  Google Scholar 

  • Shanks, D. R., Tunney, R. J., & McCarthy, J. D. (2002). A re-examination of probability matching and rational choice. Journal of Behavioral Decision Making, 15, 233–250.

    Article  Google Scholar 

  • Stahl, J. (2010). Error detection and the use of internal and external error indicators: an investigation of the first-indicator hypothesis. International Journal of Psychophysiology, 77, 43–52.

    Article  Google Scholar 

  • Takeda, Y., Yamanaka, K., & Yamamoto, Y. (2008). Temporal decomposition of EEG during a simple reaction time task into stimulus- and response-locked components. NeuroImage, 39, 742–754.

    Article  Google Scholar 

  • Walsh, M. M., & Anderson, J. R. (2011). Modulation of the feedback-related negativity by instruction and experience. Proceedings of the National Academy of Science, 108, 19048–19053.

    Article  Google Scholar 

  • Walsh, M. M., & Anderson, J. R. (2012). Learning from experience: event-related potential correlates of reward processing, neural adaptation, and behavioral choice. Neuroscience & Biobehavioral Reviews, 36, 1870–1884.

    Article  Google Scholar 

  • Yeung, N., & Sanfey, A. G. (2004). Independent coding of reward magnitude and valence in the human brain. Journal of Neuroscience, 24, 6258–6264.

    Article  Google Scholar 

Download references

Acknowledgements

We thank Kashmira Daruwalla and Maggie Webb for assistance during data collection.

Funding

This research was supported by Australian Research Council Discovery Early Career Researcher Awards to David Sewell (DE140100772) and Stefan Bode (DE140100350).

Author information

Authors and Affiliations

  1. School of Psychology, The University of Queensland, St. Lucia, QLD, 4072, Australia

    David K. Sewell & Maxwell Lyons

  2. Melbourne School of Psychological Sciences, The University of Melbourne, Parkville, 3010, VIC, Australia

    David K. Sewell, Hayley A. Warren, Daniel Rosenblatt, Daniel Bennett & Stefan Bode

Authors
  1. David K. Sewell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hayley A. Warren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Rosenblatt
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Bennett
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Maxwell Lyons
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Stefan Bode
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David K. Sewell.

Ethics declarations

The study was approved by the Human Research Ethics Committee at the University of Melbourne and was conducted in accordance with the Declaration of Helsinki.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sewell, D.K., Warren, H.A., Rosenblatt, D. et al. Feedback Discounting in Probabilistic Categorization: Converging Evidence from EEG and Cognitive Modeling. Comput Brain Behav 1, 165–183 (2018). https://doi.org/10.1007/s42113-018-0012-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42113-018-0012-6

Keywords

Search

Navigation