Model-Based Identification of EEG Markers for Learning Opportunities in an Associative Learning Task with Delayed Feedback

  • Felix Putze
  • Daniel V. Holt
  • Tanja Schultz
  • Joachim Funke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8681)


This paper combines a reinforcement learning (RL) model and EEG data analysis to identify learning situations in a associative learning task with delayed feedback. We investigated neural correlates in occipital alpha and prefrontal theta band power of learning opportunities, identified by the RL model. We show that those parameters can also be used to differentiate between learning opportunities which lead to correct learning and those which do not. Finally, we show that learning situations can also be identified on a single trial basis.


Reinforcement Learning learning situations EEG Frequency Analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, J.R., Betts, S., Ferris, J.L., Fincham, J.M.: Neural imaging to track mental states while using an intelligent tutoring system. Proceedings of the National Academy of Sciences 107(15), 7018–7023 (2010)CrossRefGoogle Scholar
  2. 2.
    Cavanagh, J.F., Frank, M.J., Klein, T.J., Allen, J.J.B.: Frontal theta links prediction errors to behavioral adaptation in reinforcement learning. NeuroImage 49(4), 3198–3209 (2010)CrossRefGoogle Scholar
  3. 3.
    Collins, A.G.E., Frank, M.J.: How much of reinforcement learning is working memory, not reinforcement learning? a behavioral, computational, and neurogenetic analysis. European Journal of Neuroscience 35(7), 1024–1035 (2012)CrossRefGoogle Scholar
  4. 4.
    Fu, W.-T., Anderson, J.R.: From recurrent choice to skill learning: A reinforcement-learning model. Journal of Experimental Psychology: General 135(2), 184–206 (2006)CrossRefGoogle Scholar
  5. 5.
    Jensen, O., Tesche, C.D.: Frontal theta activity in humans increases with memory load in a working memory task. The European Journal of Neuroscience 15(8), 1395–1399 (2002)CrossRefGoogle Scholar
  6. 6.
    Klimesch, W., Doppelmayr, M., Schwaiger, J., Auinger, P., Winkler, T.: ‘Paradoxical’ alpha synchronization in a memory task. Cognitive Brain Research 7(4), 493–501 (1999)CrossRefGoogle Scholar
  7. 7.
    Klimesch, W.: Memory processes, brain oscillations and EEG synchronization. International Journal of Psychophysiology 24(1-2), 61–100 (1996)CrossRefGoogle Scholar
  8. 8.
    Klimesch, W.: EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Research Reviews 29(2-3), 169–195 (1999)CrossRefGoogle Scholar
  9. 9.
    Klimesch, W., Sauseng, P., Hanslmayr, S.: EEG alpha oscillations: The inhibition–timing hypothesis. Brain Research Reviews 53(1), 63–88 (2007)CrossRefGoogle Scholar
  10. 10.
    Osipova, D., Takashima, A., Oostenveld, R., Fernández, G., Maris, E., Jensen, O.: Theta and gamma oscillations predict encoding and retrieval of declarative memory. The Journal of Neuroscience 26(28), 7523–7531 (2006); PMID: 16837600Google Scholar
  11. 11.
    Sauseng, P., Klimesch, W., Doppelmayr, M., Pecherstorfer, T., Freunberger, R., Hanslmayr, S.: EEG alpha synchronization and functional coupling during top-down processing in a working memory task. Human Brain Mapping 26(2), 148–155 (2005)CrossRefGoogle Scholar
  12. 12.
    Sutton, R.S., Barto, A.G.: Introduction to Reinforcement Learning, 1st edn. MIT Press, Cambridge (1998)Google Scholar
  13. 13.
    Thomson, D.J.: Spectrum estimation and harmonic analysis. Proceedings of the IEEE 70(9), 1055–1096 (1982)CrossRefGoogle Scholar
  14. 14.
    Tuladhar, A.M., ter Huurne, N., Schoffelen, J.-M., Maris, E., Oostenveld, R., Jensen, O.: Parieto-occipital sources account for the increase in alpha activity with working memory load. Human Brain Mapping 28(8), 785–792 (2007)CrossRefGoogle Scholar
  15. 15.
    Walsh, M.M., Anderson, J.R.: Learning from delayed feedback: neural responses in temporal credit assignment. Cognitive, Affective, & Behavioral Neuroscience 11(2), 131–143 (2011)CrossRefGoogle Scholar
  16. 16.
    Weiss, S., Müller, H.M., Rappelsberger, P.: Theta synchronization predicts efficient memory encoding of concrete and abstract nouns. Neuroreport 11(11), 2357–2361 (2000)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Felix Putze
    • 1
  • Daniel V. Holt
    • 2
  • Tanja Schultz
    • 1
  • Joachim Funke
    • 2
  1. 1.Institute of Anthropomatics and RoboticsKarlsruhe Institute of TechnologyKarlsruheGermany
  2. 2.Institute of PsychologyUniversity of HeidelbergHeidelbergGermany

Personalised recommendations