Advertisement

Event-based silicon retinas and cochleas

  • Tobi Delbruck
  • Shih-Chii Liu
Chapter

Abstract

This chapter reviews neuromorphic silicon retinas and cochleas that are based on the structure and operation of their biological counterparts. These devices are built using conventional chip fabrication technologies, using transistor circuits that emulate neural computations from biology. In first generation sensors, the analog outputs of every cell were read out serially at fixed sample rates. The new generation of sensors reports only the outputs of active cells through digital events (spikes) that are communicated asynchronously. Such sensors respond more quickly with reduced power consumption. Their digital “address-event” outputs rapidly convey precise timing information about the scene that is only attained from conventional sensors if they are continuously sampled at high rates. The sparseness, low latency, and spatio-temporal structure of this new form of sensor output data can benefit subsequent post-processing algorithms. Tradeoffs in the design of neuromorphic visual and auditory sensors are discussed. Examples are given of vision algorithms that process the address-events, using their spatio-temporal coherence, for low-level feature extraction and for object tracking.

Keywords

Basilar Membrane Scene Illumina Silicon Retina Neuromorphic Chip Silicon Cochlea 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abdalla H, Horiuchi T (2005) An ultrasonic filterbank with spiking neurons, IEEE Intl. Symp. on Circuits and Systems (ISCAS 2005), pp. 4201–4204Google Scholar
  2. Barbaro M, Burgi PY, Mortara A, Nussbaum P, Heitger F (2002) A 100 × 100 pixel silicon retina for gradient extraction with steering filter capabilities and temporal output coding. IEEE J. of Solid-State Circuits 37: 160–172CrossRefGoogle Scholar
  3. Bauer D, Belbachir AN, Donath N, Gritsch G, Kohn B, et al. (2007) Embedded vehicle speed estimation system using an asynchronous temporal contrast vision sensor. EURASIP J Embedded Syst 2007 (art. ID 82 174): 1–12Google Scholar
  4. Belbachir AN, Litzenberger M, Posch C, Schon P (2007) Real-time vision using a smart sensor system. IEEE Intl Symp Industrial Electronics 2007. ISIE 2007, pp. 1968–1973.Google Scholar
  5. Berner R, Lichtsteiner P, Delbruck T (2008) Self-timed vertacolor dichromatic vision sensor for low power face detection. IEEE Intl Symp Circuits and Systems (ISCAS 2008), pp. 1032–1035Google Scholar
  6. Boerlin M, Delbruck T, Eng K (2009) Getting to know your neighbors: Unsupervised learning of topography from real-world, event-based input. Neural Computation 21: 216–238PubMedCrossRefGoogle Scholar
  7. Chan V, Liu S-C, van Schaik A (2007) AER EAR: A matched silicon cochlea pair with address event representation interface. IEEE Trans Circuits and Systems I: Regular Papers 54: 48–59CrossRefGoogle Scholar
  8. Chicca E, Whatley AM, Lichtsteiner P, Dante V, Delbruck T, et al. (2006) A multi-chip pulse-based neuromorphic infrastructure and its application to a model of orientation selectivity. IEEE Trans Circuits and Systems I: Regular Papers 54: 981–993CrossRefGoogle Scholar
  9. Choi TYW, Merolla PA, Arthur JV, Boahen KA, Shi BE (2005) Neuromorphic implementation of orientation hypercolumns. IEEE Trans Circuits and Systems I: Regular Papers 52: 1049–1060CrossRefGoogle Scholar
  10. Conradt J, Berner C. M., Delbruck T (2009) An embedded AER dynamic vision sensor for low-latency pole balancing. 5th IEEE Workshop on Embedded Computer Vision (in conjunction with ICCV 2009), Kyoto, JapanGoogle Scholar
  11. Delbruck T, Lichtsteiner P (2007) Fast sensory motor control based on event-based hybrid neuromorphic-procedural system. IEEE Intl Symp Circuits and Systems (ISCAS 2007), pp. 845–848Google Scholar
  12. Delbruck T (2008) Frame-free dynamic digital vision. Proc Intl Symp Secure-Life Electronics, Advanced Electronics for Quality Life and Society, pp. 21–26. Tokyo: University of TokyoGoogle Scholar
  13. Douglas R, Mahowald M, Mead C (1995) Neuromorphic Analog VLSI. Ann Rev Neurosci 18: 255–281PubMedCrossRefGoogle Scholar
  14. Fasnacht D, Delbruck T (2007) Dichromatic spectral measurement circuit in vanilla CMOS. IEEE Intl Symp Circuits and Systems (ISCAS 2007), pp. 3091–3094Google Scholar
  15. Fossum ER (1997) CMOS image sensors: electronic camera-on-a-chip. IEEE Trans Electron Devices 44: 1689–1698CrossRefGoogle Scholar
  16. Fragniere E (2005) A 100-channel analog CMOS auditory filter bank for speech recognition. IEEE ISSCC Dig of Tech Papers, pp. 140–589Google Scholar
  17. Fu Z, Delbruck T, Lichtsteiner P, Culurciello E (2008) An address-event fall detector for assisted living applications. IEEE Trans Biomed Circuits and Systems 2: 88–96.CrossRefGoogle Scholar
  18. Georgiou J, Toumazou C (2005) A 126-uW cochlear chip for a totally implantable system. IEEE J. Solid-State Circuits 40: 430–443CrossRefGoogle Scholar
  19. Gold B, Morgan N (2000) Speech and audio signal processing: John Wiley and Sons, Inc. New York, NYGoogle Scholar
  20. Gritsch G, Litzenberger M, Donath N, Kohn B (2008) Real-time vehicle classification using a smart embedded device with a’ silicon retina’ optical sensor. ITSC08, pp. 534–538. Bejing, ChinaGoogle Scholar
  21. Hamilton T, Tapson J, Jin CT, van Schaik A (2008) An active 2-D silicon cochlea. IEEE Trans Biomed Circuits and Systems 2: 30–43CrossRefGoogle Scholar
  22. Indiveri G, Liu S-C, Delbruck T, Douglas R (2009) Neuromorphic systems. In: L Squire (ed) Encyclopedia of neuroscience, pp. 521–528: Academic PressGoogle Scholar
  23. jAER (2007) jAER Real time sensory-motor processing for spike based address-event representation (AER) sensors and systems available: http://jaer.wiki.sourceforge.netGoogle Scholar
  24. Katsiamis A, Drakakis E, Lyon R (2009) A biomimetic, 4.5uW, 120+ dB, log-domain cochlea channel with AGC. IEEE J Solid-State Circuits 44: 1006–1022CrossRefGoogle Scholar
  25. Kramer J (2002) An ON/OFFtransient imager with event-driven, asynchronous read-out. IEEE Intl Symp Circuits and Systems (ISCAS 2002), pp. 165–168Google Scholar
  26. Lazzaro J, Wawrzynek J, Mahowald M, Sivilotti M, Gillespie D (1993) Silicon auditory processors as computer peripherals. IEEE Trans Neural Networks 4: 523–528CrossRefGoogle Scholar
  27. Lennie P (2003) The cost of cortical computation. Current Biology 13: 493–497PubMedCrossRefGoogle Scholar
  28. Lenoro-Bardallo JA, Serrano-Gotarredona T, Linares-Barranco B (2010) A spatial calibrated contrast AER vision sensor with adjustable contrast threshold. IEEE Intl Symp Circuits and Systems (ISCAS 2010), pp. 2426–2429Google Scholar
  29. Lichtsteiner P, Posch C, Delbruck T (2006) A 128×128 120 dB 30 mW asynchronous vision sensor that responds to relative intensity change. ISSCC Dig Tech. Papers, pp. 508–509 (27.9). San FranciscoGoogle Scholar
  30. Lichtsteiner P, Posch C, Delbruck T (2008) A 128×128 120 dB 15us latency asynchronous temporal contrast vision sensor. IEEE J Solid State Circuits 43: 566–576CrossRefGoogle Scholar
  31. Linares-Barranco A, Gómez-Rodríguez F, Jiménez A, Delbruck T, Lichtsteiner P (2007) Using FPGA for visuo-motor control with a silicon retina and a humanoid robot. IEEE Intl Symp Circuits and Systems (ISCAS 2007), pp. 1192–1195Google Scholar
  32. Litzenberger M, Posch C, Bauer D, Schön P, Kohn B, et al. (2006) Embedded vision system for real-time object tracking using an asynchronous transient vision sensor. IEEE Digital Signal Proc Workshop 2006, pp. 173–178. Grand Teton, WyomingGoogle Scholar
  33. Liu SC, Kramer J, Indiveri G, Delbruck T, Douglas R (2002) Analog VLSI: circuits and principles: MIT Press, Cambridge, MAGoogle Scholar
  34. Liu SC and Delbruck T (2010) Neuromorphic sensory systems. Curr. Opin. in Neurobiol 20: 288–295CrossRefGoogle Scholar
  35. Liu SC, van Schaik A, Minch BA, Delbruck T (2010a) Event-based 64-channel binaural silicon cochlea with Q enhancement mechanisms. IEEE Intl Symp Circuits and Systems 2010 (ISCAS 2010), pp. 2027–2030Google Scholar
  36. Liu SC, Mesgarani N, Harris, J, Hermansky, H (2010b) The use of spike-based representations for hardware audition systems. IEEE Intl Symp Circuits and Systems 2010 (ISCAS 2010), pp. 505–508Google Scholar
  37. Lyon RF, Mead C (1988) An analog electronic cochlea. IEEE Trans Acoustics Speech and Signal Processing 36: 1119–1134CrossRefGoogle Scholar
  38. Mahowald MA (1992) VLSI analogs of neuronal visual processing: a synthesis of form and function. Computation and neural systems, Caltech, Pasadena, CaliforniaGoogle Scholar
  39. Mahowald MA (1994) An analog VLSI system for stereoscopic vision: Kluwer, Boston, MAGoogle Scholar
  40. Mahowald MA, Mead C (1991) The silicon retina. Sci Am 264: 76–82PubMedCrossRefGoogle Scholar
  41. Mallik U, Clapp M, Choi E, Cauwenberghs G, Etienne-Cummings R (2005) Temporal change threshold detection imager. IEEE ISSCC Dig Tech. Papers, pp. 362–363Google Scholar
  42. Martignoli S, van der Vyver J-J, Kern A, Uwate Y, Stoop R (2007) Analog electronic cochlea with mammalian hearing characteristics. Applied Physics Letters 91 (064 108)Google Scholar
  43. Massari N, Gottardi M, Jawed S (2008) A 100uW 64 × 128-pixel contrast-based asynchronous bin ary vision sensor for wireless sensor networks. IEEE ISSCC Dig Tech Papers, pp. 588–638Google Scholar
  44. Mead C (1990) Neuromorphic electronic systems. Proc IEEE 78: 1629–1636CrossRefGoogle Scholar
  45. Olsson JAM, Hafliger P (2009) Live demonstration of an asynchronous integrate-and-fire pixel-event vision sensor. IEEE Intl Symp Circuits and Systems (ISCAS 2009), pp. 774 774Google Scholar
  46. Pelgrom M, Tuinhout H, Vertregt M (1998) Transistor matching in analog CMOS applications. IEDM Tech Dig: 915–918Google Scholar
  47. Posch C, Hofstatter M, Matolin D, Vanstraelen G, Schon P, et al. (2007) A dual-line optical transient sensor with on-chip precision time-stamp generation. IEEE ISSCC Dig Tech Papers, pp. 500–618Google Scholar
  48. Posch C, Matolin D, Wohlgenannt R (2010) A QVGA 143 dB DR asynchronous address-event PWM dynamic image sensor with lossless pixel-level video compression. IEEE ISSCC Dig Tech. Papers, pp. 400–401Google Scholar
  49. Posch C, Matolin D, Wohlgenannt R, Maier T, Litzenberger M (2009) A microbolometer asynchronous dynamic vision sensor for LWIR. IEEE Sensors Journal 9: 654–664CrossRefGoogle Scholar
  50. Rodieck R (1998) The first steps in seeing: Sinauer Associates, Sunderland, MAGoogle Scholar
  51. Ruedi PF, Heim P, Gyger S, Kaess F, Arm C, et al. (2009) An SoC combining a 132 dB QVGA pixel array and a 32 b DSP/MCU processor for vision applications. IEEE ISSCC Dig Tech Papers, pp. 46–47Google Scholar
  52. Ruedi PF, Heim P, Kaess F, Grenet E, Heitger F, et al. (2003) A 128 × 128, pixel 120-dB dynamic-range vision-sensor chip for image contrast and orientation extraction. IEEE J. Solid-State Circuits 38: 2325–2333CrossRefGoogle Scholar
  53. Sarpeshkar R (1998) Analog versus digital: Extrapolating from electronics to neurobiology. Neural Computation 10: 1601–38PubMedCrossRefGoogle Scholar
  54. Sarpeshkar R, Baker MS C, Sit JJ, Turicchia L, Zhak S (2005) An analog bionic ear processor with zero-crossing detection. IEEE ISSCC Dig Tech Papers, pp. 78–79Google Scholar
  55. Sarpeshkar R, Lyon RF (1998) A low-power wide-dynamic-range analog VLSI cochlea. Analog Integrated Circuits and Signal Processing 16: 245–274CrossRefGoogle Scholar
  56. Serrano-Gotarredona R, Oster M, Lichtsteiner P, Linares-Barranco A, Paz-Vicente R, et al. (2009) CAVIAR: A 45 k neuron, 5 M synapse, 12 G connects/s AER hardware sensory-processing-learning-actuating system for high-speed visual object recognition and tracking. IEEE Trans Neur al Networks 20: 1417–1438CrossRefGoogle Scholar
  57. Uysal I, Sathyendra H, Harris JG (2006) A biologically plausible system approach for noise robust vowel recognition. IEEE Proc Midwest Symp Circuits and Systems, pp. 245–249Google Scholar
  58. van Schaik A, Shamma S (2004) A neuromorphic sound localizer. Analog Integrated Circuits and Signal Processing 39: 267–273CrossRefGoogle Scholar
  59. Watts L, Kerns DA, Lyon RF, Mead CA (1992) Improved implementation of the silicon cochlea. IEEE J. of Solid State Circuits 27: 692–700CrossRefGoogle Scholar
  60. Wen B, Boahen K (2006) A 360-channel speech preprocessor that emulates the cochlear amplifier. IEEE ISSCC Dig Tech Papers, pp. 556–557Google Scholar
  61. Zaghloul KA, Boahen K (2004) Optic nerve signals in a neuromorphic chip II: Testing and results. IEEE Trans Biomed Engineering 51: 667–675CrossRefGoogle Scholar

Copyright information

© Springer-Verlag/Wien 2012

Authors and Affiliations

  • Tobi Delbruck
    • 1
  • Shih-Chii Liu
  1. 1.Institute of NeuroinformaticsUniversity of Zürich and ETH ZürichZürichSwitzerland

Personalised recommendations

Cite chapter

Buy options