On-Chip Vision

  • Marco Ferretti

Abstract

This paper reviews the ever increasing activity in smart sensors, initiated in the 80’s by Mead’s silicon retina design. The gap between the sensors and the processors in a traditional image processing system can be partially reduced by embedding into the same silicon die both the transduction of incoming illumination and its interpretation through some processing function. If we consider the silicon die a “real estate” on which to build the smart sensor, many possible alternatives arise, which either privilege the sensing function or the processing one. On-chip-vision is the domain of integration that tries to balance the available silicon resources between these two extreme approaches. Analog processing is a viable solution for many tasks, though it leads to implementations that lack a general programmability. Application specific solutions, according to the ASIC paradigm, take a more flexible approach in mixing both analog and digital processing, according to the single task embedded in the silicon die. General purpose digital retinas, which combine an array of sensors with a programmable digital cellular array of simple processors, have also been proposed and built; the area trade-off between the sensing function and the near-neighbour connectivity suited to low level image processing is a major issue for further research.

Keywords

Silicon Area Smart Sensor Horizontal Resistor Photodetector Array Fingerprint Verification 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    J.E. Dowling, The Retina: An Approachable Part of the Brain, Belknap Press of Harvard University, Cambridge, MA (1987).Google Scholar
  2. 2.
    K. Sakakibara, H. Yamamoto, S. Maegawa, H. Kawashima, Y. Nishioka, M. Yamawaki, S. Asai, N. Tsubouchi, T. Okumura, and J. Fujino, A 1 Format 1.5M Pixel IT-CCD Image Sensor for and HDTV Camera System, IEEE Trans, on Consumer Electronics, Vol.37, No.3, pp. 487–493 (1991).CrossRefGoogle Scholar
  3. 3.
    M. Negishi, H. Yamada, K. Harada, M. Yamagishi, and K. Yonemoto, A Low Smear Structure for 2M-Pixel CCD Image Sensors, IEEE Trans. on Consumer Electronics, Vol.37, No.3, pp. 494–499 (1991).CrossRefGoogle Scholar
  4. 4.
    B.Y. Zavidovique and T. M. Bernard, Generic Functions for On-Chip Vision, Proc. IEEE ICPR-92, The Hague, The Netherlands, NL, pp. 1-10 (1992).Google Scholar
  5. 5.
    S.M. Sze, Semiconductor Devices: Physics and Technology, John Wiley and Sons, New York, NY, pp. 24, 523 (1985).Google Scholar
  6. 6.
    J.E. Hershey, R. Liberati, and D. S. Hammer, New Focal Plane Architecture and Transform for Fast Target Recognition, Applied Optics, Vol.28, No. 18, pp. 3810–3813 (1989).PubMedCrossRefGoogle Scholar
  7. 7.
    C. Mead, Analog VLSI and Neural Systems, Addison-Wesley, Reading, MA (1989).CrossRefGoogle Scholar
  8. 8.
    S.G. Chamberlain and J.P.Y. Lee, A Novel Wide Dynamic Range Silicon Photodetector and Linear Imaging Array, IEEE Journal of Solid-State Circuits, Vol.SC-19, No.1, pp. 41–48 (1984).CrossRefGoogle Scholar
  9. 9.
    H. Akimoto, H. Ando, H. Nakagawa, Y. Nakahara, M. Hikiba, and H. Ohta, A 1/3-in 410000-Pixel CCD Image Sensor with Feedback Field-Plate Amplifier, IEEE Journal of Solid-State Circuits, Vol.SC-26, No.12, pp. 1907–1913 (1991).CrossRefGoogle Scholar
  10. 10.
    E.R. Fossum, Wire Transfer of Charge Packets Using a CCD-BBD Structure for Charge-Domain Signal Processing, IEEE Trans. on Electron Devices, Vol.ED-38, No.2, pp. 291–298 (1991).CrossRefGoogle Scholar
  11. 11.
    E.G. Stevens, Photoresponse Nonlinearity of Solid-State Image Sensors with Antiblooming Protection, IEEE Trans. on Electron Devices, Vol.ED-38, No.2, pp. 299–302 (1991).CrossRefGoogle Scholar
  12. 12.
    T. Watanabe, K. Hashiguchi, T. Yamano, J. Nakai, S. Miyatake, O. Matsui, and K. Awane, A CCD Color Signal Separation IC for Single-Chip Color Imagers, IEEE Journal of Solid-State Circuits, Vol.SC-19, No.1, pp. 49–54 (1984).CrossRefGoogle Scholar
  13. 13.
    S. Kawamoto, Y. Watanabe, Y. Otsuka, and T. Narabu, A CCD Color Linear Image Sensor Employing New Transfer Method, IEEE Trans. on Consumer Electronics, Vol.37, No.3, pp. 481–486 (1991).CrossRefGoogle Scholar
  14. 14.
    J. Van der Spiegel, G. Kreider, C. Claeys, I. Debusschere, G. Sandini, P. Dario, F. Fantini, P. Bellutti, and G. Soncini, A Foveated-Like Sensor Using CCD Technology, in Analog VLSI Implementation of Neural Systems, C. Mead and M. Ismail eds., Kluwer Academic Publisher, Norwell, MA, pp. 189–211 (1989).CrossRefGoogle Scholar
  15. 15.
    S.S. Bencuya and A.J. Steckl, Charge-Packet Splitting in Charge-Domain Devices, IEEE Trans. on Electron Devices, Vol.ED-31, No.10, pp. 1494–1501 (1984).CrossRefGoogle Scholar
  16. 16.
    J. Hynecek, CCM-A New Low-Noise Charge Carrier Multiplier Suitable for Detection of Charge in Small Pixel CCD Image Sensors, IEEE Trans. on Electron Devices, Vol.ED-39, No.8, pp. 1972–1975 (1992).CrossRefGoogle Scholar
  17. 17.
    E.R. Fossum and R.C. Barker, A Linear and Compact Charge-Coupled Charge Packet Differencer/Replicator, IEEE Trans. on Electron Devices, Vol.ED-31, No.12, pp. 1784–1789 (1984).CrossRefGoogle Scholar
  18. 18.
    C. Sin, A. Kramer, V. Hu, R.R. Chu, and P. K. Ko, EEPROM as an Analog Storage Device, with Particular Applications in Neural Networks, IEEE Trans. on Electron Devices, Vol.ED-39, No.6, pp. 1410–1419 (1992).CrossRefGoogle Scholar
  19. 19.
    J. Harris, C. Koch, J. Luo, and J. Wyatt, Resitive Fuses: Analog Hardware for Detecting Discontinuities in Early Vision, in Analog VLSI Implementation of Neural Systems, C. Mead and M. Ismail eds., Kluwer Academic Publisher, Norwell, MA, pp. 27–56 (1989).CrossRefGoogle Scholar
  20. 20.
    A.M. Chiang and M.L. Chuang, A CCD Programmable Image Processor and its Neural Network Applications, IEEE Journal of Solid-State Circuits, Vol.SC-26, No.12, pp. 1894–1901 (1991).CrossRefGoogle Scholar
  21. 21.
    O. Yadid-Pecht, R. Ginosar, and Y. Shacham Diamand, A Random Access Photodiode Array for Intelligent Image Capture, IEEE Trans. on Electron Devices, Vol.ED-38, No.8, pp. 1772–1780 (1991).CrossRefGoogle Scholar
  22. 22.
    M. Tremblay, D. Larendeau, and D. Poussait, Multi module focal plane processing sensor with parallel analog support for computer vision, Proc. Symposium on Integrated Systerna, Seattle, WA (1993).Google Scholar
  23. 23.
    M. Gottardi, A. Sartori, and A. Simoni, POLIFEMO-An Addressable CMOS 128x128-Pixel Image Sensor with Digital Interface, IRST TR (1992).Google Scholar
  24. 24.
    A. Sartori, A Smart Camera, in FPGAs, Moore and Luks eds., Abingdon EE&CS Books, Abingdon, pp. 353–362 (1991).Google Scholar
  25. 25.
    K. Chen, A. Aström, and P.E. Danielsson, PASIC a smart sensor for computer vision, Proc. ICPR, Atlantic City, NJ, pp. 286-291 (1990).Google Scholar
  26. 26.
    C. Jansson, P. Ingelhag, C. Svensson, and R. Forchheimer, An Addressable 256x256 Photodiode Image Sensor Array with an 8-bit Digital Output, Proc. ESSCIRC-92, pp. 151-154 (1992).Google Scholar
  27. 27.
    A. Aström, Smart Image Sensors, Linköping Studies in Science and Technology, Dissertation No.319, Linköping, S (1993).Google Scholar
  28. 28.
    R. Forchheimer, P. Ingelhag, and C. Jansson, MAPP2200: a second generation smart optical sensor, SPIE 92, Vol. 1659, pp. 2–11(1992).CrossRefGoogle Scholar
  29. 29.
    IVP Integrated Vision Products AB, MAPP2200 Technical Description, Linköping, S (1991).Google Scholar
  30. 30.
    C. Mead, Adaptive retina, in Analog VLSI Implementation of Neural Systems, C. Mead and M. Ismail eds., Kluwer Academic Publisher, Norwell, MA, pp. 239–246 (1989).CrossRefGoogle Scholar
  31. 31.
    D.L. Standley and J.L. Jr. Wyatt, Stability Criterion for Lateral Inhibition and Related Networks that is Robust in the Presence of Integrated Circuits Parasitics, IEEE Trans. on Circuits and Systems, Vol.CAS-36, No.5, pp. 675–681 (1989).CrossRefGoogle Scholar
  32. 32.
    H. Kobayashi, J. White, and A.A. Abidi, An active resistor network for gaussian filtering of images, IEEE Journal of Solid-State Circuits, Vol.26, No.5, pp. 738–748 (1991).CrossRefGoogle Scholar
  33. 33.
    P.C. Yu and H. Lee, A CMOS resitive-fuse processor for 2-D image acquisition, smoothing and segmentation, Proc. ESSCIRC 92, pp. 147-150 (1992).Google Scholar
  34. 34.
    T. Shimmi, H. Kobayashi, T. Yagi, T. Sawaji, T. Matsumoto, and A.A. Abidi, A parallel analog CMOS signal processor for image contrast enhancement, Proc. ESSCIRC 92, pp. 163–166 (1992).Google Scholar
  35. 35.
    J. Tanner and C. Mead, Optical motion sensor, in VLSI Signal Processing, Kung, Owen and Nash eds., IEEE Press, pp. 59-76 (1987).Google Scholar
  36. 36.
    J.L. Wyatt Jr., D.L. Standley, and W. Yang, The MIT vision chip project: analog VSLI systems for fast image acquisition and early vision processing, Proc. IEEE Intern. Conference on Robotics and Automation, Sacramento, CA, pp. 1330-1335 (1991).Google Scholar
  37. 37.
    C. Chong, C. Andre, T. Salama, and K.C. Smith, Image-Motion Detection Using Analog VLSI, IEEE Journal of Solid-State Circuits, Vol.27, No.1, pp. 93–96 (1992).CrossRefGoogle Scholar
  38. 38.
    M. Gottardi and W. Yang, A CCD/CMOS Image Motion Sensor, Proc. 1993 IEEE ISSCC, pp. 194-195 (1993).Google Scholar
  39. 39.
    R. Etienne-Cummings, S. Fernando, N. Takahashi, V. Shtonov, J. Van der Spiegel, and P. Mueller, A new temporal domain optical flow measurement technique for focal plane VLSI implementation, Proc. CAMP93, New Orleans, LA, pp. 241-250 (1993).Google Scholar
  40. 40.
    A. Yakovleff, A. Moini, A. Bouzerdoum, X.T. Nguyen, R.E. Bogner, K. Eshraghian, and D. Abbott, A microsensor based on insect vision, Proc. CAMP93, New Orleans, LA, pp. 137-146 (1993).Google Scholar
  41. 41.
    A. Andreou, K. Strhben, and R.E. Jenkins, Silicon retina for motion computation, Proc. ISCAS 91, Singapore, pp. 1373-1376 (1991).Google Scholar
  42. 42.
    J.M. Hakkarainen and H.S. Lee, A 40x40 CCD/CMOS AVD Processor for Use in a Stereo Vision System, Proc. ESSC1RC-92, pp. 155-158 (1992).Google Scholar
  43. 43.
    A. Gruss, L.R. Carley, and T. Kanade, Integrated sensor and range-finding analog signal processor, IEEE Journal of Solid-state Circuits, Vol.26, No.3, pp. 184–191 (1991).CrossRefGoogle Scholar
  44. 44.
    S. Anderson, W.H. Bruce, P.B. Denyer, D. Renshaw, and G. Wang, A Single Chip Sensor and Image Processor for Fingerprint Verification, Proc. IEEE 1991 ICCD, pp. 12.1.1-12.1.4 (1991).Google Scholar
  45. 45.
    D. Renshaw, P.B. Denyer, G. Wang, and M. Lu, ASIC Image Sensor, Proc. ISCAS-90, pp. 3038-3041 (1990).Google Scholar
  46. 46.
    D. Standley, An Object Position and Orientation IC with Embedded Imager, IEEE Journal of Solid-State Circuits, Vol.26, No.12, pp. 1853–1859 (1991).CrossRefGoogle Scholar
  47. 47.
    D.J. Friedman and J.J. Clark, VLSI Sensor/processor Circuitry for Autonomous Robots, Optics, Illumination, and Image Sensing for Machine Vision, SPIE, Vol.1614, pp. 99–110 (1991).CrossRefGoogle Scholar
  48. 48.
    S. Kemeny, H. Torby, H. Meadows, R. Bredthauer, M. LaShell, and E.R. Fossum, CCD image sensor with differential pyramidal output for lossless image compression, Proc. IEEE CICC 91, pp. 12.6.1–12.6.4 (1991).Google Scholar
  49. 49.
    H. Lin, W. Sah, and S. Lee, The Vertical Integration of Christalline NMOS and Amorphous Orientational Edge Detector, IEEE Trans, on Electron Devices, Vol.ED-39, No.12, pp. 2810–2812 (1992).CrossRefGoogle Scholar
  50. 50.
    V. Cantoni, V. Di Gesù, M. Ferretti, S. Levialdi, R. Negrini, and R. Stefanelli, The PAPIA System, Journal of VLSI Signal Processing, Vol.2, pp. 195–217 (1991).CrossRefGoogle Scholar
  51. 51.
    M.J.B. Duff, Propagation in cellular logic arrays, in Proc. Workshop on Picture Data Description and Management, pp. 259-262 (1980).Google Scholar
  52. 52.
    T. Bernard, B. Zavidovique, and F. Devos, A programmable artificial retina, IEEE Journal of Solid-State Circuits, Vol.28, pp. 789–798 (1993).CrossRefGoogle Scholar
  53. 53.
    T.M. Bernard, B. Zavidovique, and F. Devos, The NCP retina: an imager, a halftoner and a micro-grained array processor on the same chip, Proc. ESSCIRC 92, Copenhagen, DK, pp. 159-162 (1992).Google Scholar
  54. 54.
    B.E. Bayer, An optimum medium for two-level rendition of continuos-tone pictures, IEEE Intern. Conf. on Communications, pp. 2611-2615 (1973).Google Scholar
  55. 55.
    M. Ishikawa, A. Morita, and N. Takayanagi, High speed vision system using massively parallel processing, Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Raleigh, NC, pp. 373-377 (1992).Google Scholar
  56. 56.
    A. Aström, R. Forchheimer, and P. Ingelhag, An integrated sensor/processor architecture based on nearsensor image processing, Proc. CAMP93, New Orleans, LA, pp. 147-154 (1993).Google Scholar
  57. 57.
    IVP Integrated Vision Products AB, LAPP1100 Product Description, Linköping, S (1989).Google Scholar

Copyright information

© Springer Science+Business Media New York 1994

Authors and Affiliations

  • Marco Ferretti
    • 1
  1. 1.Dipartimento di Informatica e SistemisticaUniversità di PaviaPaviaItaly

Personalised recommendations