Spike-Based Image Processing: Can We Reproduce Biological Vision in Hardware?

  • Simon J. Thorpe
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7583)


Over the past 15 years, we have developed software image processing systems that attempt to reproduce the sorts of spike-based processing strategies used in biological vision. The basic idea is that sophisticated visual processing can be achieved with a single wave of spikes by using the relative timing of spikes in different neurons as an efficient code. While software simulations are certainly an option, it is now becoming clear that it may well be possible to reproduce the same sorts of ideas in specific hardware. Firstly, several groups have now developed spiking retina chips in which the pixel elements send the equivalent of spikes in response to particular events such as increases or a decreases in local luminance. Importantly, such chips are fully asynchronous, allowing image processing to break free of the standard frame based approach. We have recently shown how simple neural network architectures can use the output of such dynamic spiking retinas to perform sophisticated tasks by using a biologically inspired learning rule based on Spike-Time Dependent Plasticity (STDP). Such systems can learn to detect meaningful patterns that repeat in a purely unsupervised way. For example, after just a few minutes of training, a network composed of a first layer of 60 neurons and a second layer of 10 neurons was able to form neurons that could effectively count the number of cars going by on the different lanes of a freeway. For the moment, this work has just used simulations. However, there is a real possibility that the same processing strategies could be implemented in memristor-based hardware devices. If so, it will become possible to build intelligent image processing systems capable of learning to recognize significant events without the need for conventional computational hardware.


Memristive Device Simple Neural Network Biological Vision Biological Vision System Large Scale Computing System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bichler, O., et al.: Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity. Neural Networks 32, 339–348 (2012)CrossRefGoogle Scholar
  2. 2.
    Crouzet, S.M., Kirchner, H., Thorpe, S.J.: Fast saccades towards faces: Face detection in just 100 ms. J. Vis. 10(4), 1–17 (2010)CrossRefGoogle Scholar
  3. 3.
    Delorme, A., Thorpe, S.J.: Face identification using one spike per neuron: resistance to image degradations. Neural Netw. 14(6-7), 795–803 (2001)CrossRefGoogle Scholar
  4. 4.
    Gollisch, T., Meister, M.: Rapid neural coding in the retina with relative spike latencies. Science 319(5866), 1108–1111 (2008)CrossRefGoogle Scholar
  5. 5.
    Guyonneau, R., Vanrullen, R., Thorpe, S.J.: Neurons Tune to the Earliest Spikes Through STDP. Neural Comput. 17(4), 859–879 (2005)zbMATHCrossRefGoogle Scholar
  6. 6.
    Lichtsteiner, P., Posch, C., Delbruck, T.: A 128x128 120 dB 15 mu s latency asynchronous temporal contrast vision sensor. IEEE Journal of Solid-State Circuits 43(2), 566–576 (2008)CrossRefGoogle Scholar
  7. 7.
    Markram, H.: The blue brain project. Nat. Rev. Neurosci. 7(2), 153–160 (2006)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Masquelier, T., Guyonneau, R., Thorpe, S.J.: Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PLoS one 3(1), e1377 (2008)Google Scholar
  9. 9.
    Masquelier, T., Guyonneau, R., Thorpe, S.J.: Competitive STDP-based spike pattern learning. Neural computation 21(5), 1259–1276 (2009)zbMATHCrossRefGoogle Scholar
  10. 10.
    Plana, L.A., et al.: SpiNNaker: Design and Implementation of a GALS Multicore System-on-Chip. ACM Journal on Emerging Technologies in Computing Systems 7(4) (2011)Google Scholar
  11. 11.
    Thorpe, S., Imbert, M.: Biological constraints on connectionist modelling. In: Pfeifer, R., et al. (eds.) Connectionism in Perspective, pp. 63–92. Elsevier, Amsterdam (1989)Google Scholar
  12. 12.
    Thorpe, S.J.: Spike arrival times: A highly efficient coding scheme for neural networks. In: Eckmiller, R., Hartmann, G., Hauske, G. (eds.) Parallel Processing in Neural Systems and Computers, pp. 91–94. Elsevier, North-Holland (1990)Google Scholar
  13. 13.
    VanRullen, R., et al.: Face processing using one spike per neurone. Biosystems 48(1-3), 229–239 (1998)CrossRefGoogle Scholar
  14. 14.
    VanRullen, R., Thorpe, S.J.: Rate coding versus temporal order coding: what the retinal ganglion cells tell the visual cortex. Neural Comput. 13(6), 1255–1283 (2001)CrossRefGoogle Scholar
  15. 15.
    Zamarreno-Ramos, C., et al.: On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Frontiers in Neuroscience (May 2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Simon J. Thorpe
    • 1
  1. 1.Centre de Recherche Cerveau & CognitionToulouseFrance

Personalised recommendations