Abstract
Learning in the brain is associated with changes of connection strengths between neurons. Here, we consider neural networks with output units for each possible action. Training is performed by giving rewards for correct actions. A major problem in effective learning is to assign credit to units playing a decisive role in the stimulus-response mapping. Previous work suggested an attentional feedback signal in combination with a global reinforcement signal to determine plasticity at units in earlier processing levels. However, it could not learn from delayed rewards (e.g., a robot could escape from fire but not walk through it to rescue a person). Based on the AGREL framework, we developed a new attention-gated learning scheme that makes use of delayed rewards. Finally, we show a close relation to standard error backpropagation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Brown, J., Bullock, D., Grossberg, S.: How the Basal Ganglia Use Parallel Excitatory and Inhibitory Learning Pathways to Selectively Respond to Unexpected Rewarding Cues. Journal of Neuroscience 19(22), 10502–10511 (1999)
Faußer, S., Schwenker, F.: Learning a Strategy with Neural Approximated Temporal–Difference Methods in English Draughts. In: ICPR, pp. 2925–2928. IEEE (2010)
Faußer, S., Schwenker, F.: Ensemble Methods for Reinforcement Learning with Function Approximation. In: Sansone, C., Kittler, J., Roli, F. (eds.) MCS 2011. LNCS, vol. 6713, pp. 56–65. Springer, Heidelberg (2011)
Felleman, D.J., Van Essen, D.C.: Distributed Hierarchical Processing in the Primate Cerebral Cortex. Cerebral Cortex 1(1), 1–47 (1991)
Frank, M.J., Badre, D.: Mechanisms of Hierarchical Reinforcement Learning in Corticostriatal Circuits 1: Computational Analysis. Cerebral Cortex 22, 509–526 (2011)
Gustafsson, B., Wigström, H.: Physiological mechanisms underlying long–term potentiation. Trends in Neurosciences 11(4), 156–162 (1988)
Joel, D., Niv, Y., Ruppin, E.: Actor-Critic Models of the Basal Ganglia: New Anatomical and Computational Perspectives. Neural Networks 15(4-6), 535–547 (2002)
Malinow, R., Miller, J.P.: Postsynaptic Hyperpolarization During Conditioning Reversibly Blocks Induction of Long–Term Potentiation. Nature 320, 529–530 (1986)
Pennartz, C.M.A.: Reinforcement Learning by Hebbian Synapses with Adaptive Thresholds. Neuroscience 81(2), 303–319 (1997)
Roelfsema, P.R., van Ooyen, A.: Attention–Gated Reinforcement Learning of Internal Representations for Classification. Neural Computation 17, 2176–2214 (2005)
Rombouts, J.O., Bohte, S.M., Roelfsema, P.R.: Neurally Plausible Reinforcement Learning of Working Memory Tasks. In: NIPS, pp. 1880–1888 (2012)
Salin, P.A., Bullier, J.: Corticocortical Connections in the Visual System: Structure and Function. Physiological Reviews 75(1), 107–154 (1995)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, London (1998)
Vitay, J., Hamker, F.H.: A Computational Model of Basal Ganglia and its Role in Memory Retrieval in Rewarded Visual Memory Tasks. Frontiers in Computational Neuroscience 4(13), 1–18 (2010)
Williams, R.J.: On the Use of Backpropagation in Associative Reinforcement Learning. In: ICNN, vol. 1, pp. 263–270 (1988)
Wörgötter, F., Porr, B.: Temporal Sequence Learning, Prediction, and Control: A Review of Different Models and Their Relation to Biological Mechanisms. Neural Computation 17(2), 245–319 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Brosch, T., Schwenker, F., Neumann, H. (2013). Attention-Gated Reinforcement Learning in Neural Networks—A Unified View. In: Mladenov, V., Koprinkova-Hristova, P., Palm, G., Villa, A.E.P., Appollini, B., Kasabov, N. (eds) Artificial Neural Networks and Machine Learning – ICANN 2013. ICANN 2013. Lecture Notes in Computer Science, vol 8131. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40728-4_34
Download citation
DOI: https://doi.org/10.1007/978-3-642-40728-4_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40727-7
Online ISBN: 978-3-642-40728-4
eBook Packages: Computer ScienceComputer Science (R0)