Optimizing Generic Neural Microcircuits through Reward Modulated STDP

  • Prashant Joshi
  • Jochen Triesch
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5768)


How can we characterize if a given neural circuit is optimal for the class of computational operations that it has to perform on a certain input distribution? We show that modifying the efficacies of recurrent synapses in a generic neural microcircuit via spike timing dependent plasticity (STDP) can optimize the circuit in an unsupervised fashion for a particular input distribution if STDP is modulated by a global reward signal. More precisely, optimizing microcircuits through reward modulated STDP leads to a lower eigen-value spread of the cross-correlation matrix, higher entropy, highly decorrelated neural activity, and tunes the circuit dynamics to a regime that requires a large number of principal components for representing the information contained in the liquid state as compared to randomly drawn microcircuits. Another set of results show that such optimization brings the mean firing rate into a realistic regime, while increasing the sparseness and the information content of the network. We also show that the performance of optimized circuits improves for several linear and non-linear tasks.


Reward modulated STDP generic neural microcircuits optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Jaeger, H., Maass, W., Principe, J.: Special issue on Echo State Networks and Liquid State Machines. Neural Networks 20, 287–289 (2007)CrossRefGoogle Scholar
  2. 2.
    Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation 14(11), 2531–2560 (2002)CrossRefzbMATHGoogle Scholar
  3. 3.
    Jaeger, H., Haas, H.: Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004)CrossRefGoogle Scholar
  4. 4.
    Jaeger, H.: Reservoir riddles: Suggestions for Echo State Network research. In: IJCNN, pp. 1460–1462 (2005)Google Scholar
  5. 5.
    Lazar, A., Pipa, G., Triesch, J.: Fading memory and time series prediction in recurrent networks with different forms of plasticity. Neural Networks 20, 312–322 (2007)CrossRefzbMATHGoogle Scholar
  6. 6.
    Schrauwen, B., Wardermann, M., Verstraeten, D., Steil, J., Stroobandt, D.: Improving reservoirs using intrinsic plasticity. Neurocomputing 71, 1159–1171 (2008)CrossRefGoogle Scholar
  7. 7.
    Gupta, A., Wang, Y., Markram, H.: Organizing principles for a diversity of GABAergic interneurons and synapses in the neocortex. Science 287, 273–278 (2000)CrossRefGoogle Scholar
  8. 8.
    Markram, H., Wang, Y., Tsodyks, M.: Differential signaling via the same axon of neocortical pyramidal neurons. PNAS 95, 5323–5328 (1998)CrossRefGoogle Scholar
  9. 9.
    Buonomano, D.V., Merzenich, M.M.: Temporal information transformed into a spatial code by a neural network with realistic properties. Science 267, 1028–1030 (1995)CrossRefGoogle Scholar
  10. 10.
    Maass, W., Markram, H.: On the computational power of recurrent circuits of spiking neurons. Journal of Computer and System Sciences 69(4), 593–616 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Maass, W., Natschläger, T., Markram, H.: Computational models for generic cortical microcircuits. Chapman & Hall/CRC, Boca Raton (2004)Google Scholar
  12. 12.
    Maass, W., Joshi, P., Sontag, E.D.: Computational aspects of feedback in neural circuits. PLOS Computational Biology 3(1), e165, 1–20 (2007)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Joshi, P., Maass, W.: Movement generation with circuits of spiking neurons. Neural Computation 17(8), 1715–1738 (2005)CrossRefzbMATHGoogle Scholar
  14. 14.
    Joshi, P.: From memory based decisions to decision based movements: A model of interval discrimination followed by action selection. Neural Networks 20, 298–311 (2007)CrossRefzbMATHGoogle Scholar
  15. 15.
    Vinje, W.E., Gallant, J.L.: Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287, 1273–1276 (2000)CrossRefGoogle Scholar
  16. 16.
    Billimoria, C.P., Kraus, B.J., Narayan, R., Maddox, R.K., Sen, K.: Invariance and sensitivity to intensity in neural discrimination of natural sounds. Jour. of Neurosc. 28(25), 6304–6308 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Prashant Joshi
    • 1
  • Jochen Triesch
    • 1
  1. 1.Frankfurt Institute for Advanced StudiesFrankfurtGermany

Personalised recommendations