Abstract
Precise neuronal spike timing plays an important role in many aspects of cognitive processing. Here, we explore how a spiking neural network can learn to generate temporally precise spikes in response to a spatio-temporal pattern, through spike-timing-dependent plasticity modulated by a delayed reward signal. An escape noise neuron is implemented as the readout to incorporate the effect of background noise on spike timing. We compare the performance of two different escape rate functions that drive spiking in the readout neuron: the Arrhenius & Current (A&C) and Exponential (EXP) model. Our results show that the network can learn to reproduce target spike patterns containing between 1 and 10 spikes with 10 ms temporal accuracy. We also demonstrate the superior performance of the A&C model over the EXP model for the parameters we consider, especially when reproducing a large number of target spikes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barto, A., Sutton, R.: Reinforcement learning: An introduction. MIT Press, Cambridge (1998)
Bi, G., Poo, M.: Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. The Journal of Neuroscience 18(24), 10464–10472 (1998)
Bohte, S.: The evidence for neural information processing with precise spike-times: A survey. Natural Computing 3(2), 195–206 (2004)
Chance, F., Abbott, L., Reyes, A.: Gain modulation from background synaptic input. Neuron 35(4), 773–782 (2002)
El-Laithy, K., Bogdan, M.: A reinforcement learning framework for spiking networks with dynamic synapses. In: Computational Intelligence and Neuroscience 2011, vol. 4 (2011)
Farries, M., Fairhall, A.: Reinforcement learning with modulated spike timing dependent synaptic plasticity. Journal of Neurophysiology 98(6), 3648–3665 (2007)
Florian, R.: Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Computation 19(6), 1468–1502 (2007)
Frémaux, N., Sprekeler, H., Gerstner, W.: Functional requirements for reward-modulated spike-timing-dependent plasticity. The Journal of Neuroscience 30(40), 13326–13337 (2010)
Gerstner, W., Kistler, W.: Spiking neuron models: Single neurons, populations, plasticity. Cambridge University Press, Cambridge (2002)
Grüning, A., Sporea, I.: Supervised learning of logical operations in layered spiking neural networks with spike train encoding. Neural Processing Letters 36(2), 117–134 (2012)
Izhikevich, E.: Solving the distal reward problem through linkage of stdp and dopamine signaling. Cerebral Cortex 17(10), 2443–2452 (2007)
Legenstein, R., Pecevski, D., Maass, W.: A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Computational Biology 4(10), e1000180 (2008)
Morrison, A., Diesmann, M., Gerstner, W.: Phenomenological models of synaptic plasticity based on spike timing. Biological Cybernetics 98(6), 459–478 (2008)
Pfister, J., Toyoizumi, T., Barber, D., Gerstner, W.: Optimal spike-timing-dependent plasticity for precise action potential firing in supervised learning. Neural Computation 18(6), 1318–1348 (2006)
Ponulak, F., Kasinski, A.: Supervised learning in spiking neural networks with resume: Sequence learning, classification, and spike shifting. Neural Computation 22(2), 467–510 (2010)
Rossum, M.: A novel spike distance. Neural Computation 13(4), 751–763 (2001)
Sporea, I., Grüning, A.: Supervised learning in multilayer spiking neural networks. Neural Computation 25(2), 473–509 (2013)
Urbanczik, R., Senn, W.: Reinforcement learning in populations of spiking neurons. Nature Neuroscience 12(3), 250–252 (2009)
Van Rossum, M., Bi, G., Turrigiano, G.: Stable hebbian learning from spike timing-dependent plasticity. The Journal of Neuroscience 20(23), 8812–8821 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gardner, B., Grüning, A. (2013). Learning Temporally Precise Spiking Patterns through Reward Modulated Spike-Timing-Dependent Plasticity. In: Mladenov, V., Koprinkova-Hristova, P., Palm, G., Villa, A.E.P., Appollini, B., Kasabov, N. (eds) Artificial Neural Networks and Machine Learning – ICANN 2013. ICANN 2013. Lecture Notes in Computer Science, vol 8131. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40728-4_32
Download citation
DOI: https://doi.org/10.1007/978-3-642-40728-4_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40727-7
Online ISBN: 978-3-642-40728-4
eBook Packages: Computer ScienceComputer Science (R0)