Advertisement

Computing Image and Motion with 3-D Memristive Grids

  • Chuan Kai Kenneth Lim
  • A. Gelencser
  • T. ProdromakisEmail author
Chapter

Abstract

In this paper, we first present a biorealistic model for the first part of early vision processing by incorporating memristive nanodevices. The architecture of the proposed network is based on the organisation and functioning of the Outer Plexiform Layer (OPL) and Inner Plexiform Layer (IPL) in the vertebrate retina. The non-linear and adaptive response of memristive devices make them excellent building blocks for realizing complex synaptic- like architectures that are common in the human retina. We particularly show how that hexagonal memristive grids can be employed for faithfully emulating the smoothing effect occuring in the OPL to enhance the dynamic range of the system. A memristor-based thresholding scheme is employed for detecting the edges of grayscale images, while evaluating the proposed system’s adaptability to different lighting conditions and fault tolerance capacity. We then extend our work to computing relative motion of objects, which is an important navigation task that vertebrates routinely perform by relying on inherently unreliable biological cells in the retina. Here, a novel memristive thresholding scheme that facilitates the detection of moving edges is introduced. In addition, a double-layered 3-D memristive network is employed for modeling the motion computations that take place in both the OPL and IPL that enables the detection of on-center and off-center transient responses. Applying the transient detection results, it is shown that it is possible to generate an estimation of the speed and direction a moving object.

Notes

Acknowledgements

The authors wish to acknowledge the financial support of the CHIST-ERA ERAnet EPSRC EP/J00801X/1 and EP/K017829/1.

References

  1. 1.
    Baccus, S.A., Olveczky, B.P., Manu, M., Meister, M.: A retinal circuit that computes object motion. J. Neurosci.: Official J. Soc. Neurosci. 28(27), 6807–6817 (2008). (JID: 8102140; ppublish)CrossRefGoogle Scholar
  2. 2.
    Barranco, B.L., Gotarredona, T.S., Mesa, L.A.C., Carrasco, J.A.P., Ramos, C.Z., Masquelier, T.: On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front. Neurosci. 5, 26 (2011)Google Scholar
  3. 3.
    Barron, J.L., Thacker, N.A.: Tutorial: computing 2D and 3D optical flow. In: Tina Memo (2005)Google Scholar
  4. 4.
    Benson, R.G., Delbruck, T.: Direction selective silicon retina that uses null inhibition. In: Advances in Neural Information Processing Systems, vol. 4, pp. 756–763. Morgan Kaufmann, Los Altos (1991)Google Scholar
  5. 5.
    Bhushan, B.: Biomimetics: lessons from nature- an overview. Philos. Trans. R. Soc. A: Math. Phys. Eng. Sci. 367(1893), 1445–1486 (2009)CrossRefGoogle Scholar
  6. 6.
    Biolek, Z., Biolek, D., Biolkova, V.: Spice model of memristor with nonlinear dopant drift. Radioengineering 18, 210–214 (2009)zbMATHGoogle Scholar
  7. 7.
    Boahen, K.: A retinomorphic chip with parallel pathways: encoding. In: Increasing, on, Decreasing, and Off Visual Signals. Analog Integrated Circuits and Signal Processing. 2002 Feb 01; 30(2), 121–135 (2002). https://doi.org/10.1023/A:1013751627357CrossRefGoogle Scholar
  8. 8.
    Borst, A.: Fly vision: moving into the motion detection circuit. Current biology : CB 21(24), R990–2, Dec 20, 2011. CI: Copyright (c) 2011; JID: 9107782; CON: Curr Biol. 2011 Dec 20; 21(24), 2077–84. PMID: 22137471; publishCrossRefGoogle Scholar
  9. 9.
    Canals, R., Roussel, A., Famechon, J.L., Treuillet, S.: A biprocessor-oriented vision-based target tracking system. IEEE Trans. Industr. Electron. 49(2), 500–506 (2002)CrossRefGoogle Scholar
  10. 10.
    Capson, D.W., Maludzinski, R.A., Feuerstein, I.A.: Microcomputer-based interactive tracking of blood cells at biomaterial surfaces. IEEE Trans. Biomed. Eng. 36(8), 860–864 (1989)CrossRefGoogle Scholar
  11. 11.
    Liu, S.C., Usseglio-Viretta, A.: Visuo-motor fly-like responses of a robot using a VLSI motion-sensitive chips. In: 2nd International ICSC Symposium on Neural Computation, pp. 23–26 (2000)Google Scholar
  12. 12.
    Chiu, C.-F., Wu, C.-Y.: Design of CMOS elementary-motion-flow-selective image detector using the BJT-based silicon retina. In: Proceedings of 1997 IEEE International Symposium on Circuits and Systems, 1997. ISCAS ’97, vol. 1, pp. 717–720 (1997)Google Scholar
  13. 13.
    Chua, L.: Memristor-the missing circuit element. IEEE Trans. Circuit Theory 18(5), 507–519 (1971)CrossRefGoogle Scholar
  14. 14.
    Chua, L., Kang, S.M.: Memristive devices and systems. Proc. IEEE 64(2), 209–223 (1976)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Clark, J.J.: Authenticating edges produced by zero-crossing algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 11(1), 43 (1989)zbMATHCrossRefGoogle Scholar
  16. 16.
    Curcio, C.A., Allen, K.A.: Topography of ganglion cells in human retina. J. Comp. Neurol. 300(1), 5–25 (1990)CrossRefGoogle Scholar
  17. 17.
    Curcio, C.A., Sloan, K.R., Kalina, R.E., Hendrickson, A.E.: Human photoreceptor topography. J. Comp. Neurol. 292(4), 497–523 (1990)CrossRefGoogle Scholar
  18. 18.
    Davies, E.R.: Machine Vision: Theory, Algorithms, Practicalities. Morgan Kaufmann Publishers Inc., San Francisco (2004)Google Scholar
  19. 19.
    Delbruck, T.: Silicon retina with correlation-based, velocity-tuned pixels. IEEE Trans. Neural Networks 4(3), 529–541 (1993)CrossRefGoogle Scholar
  20. 20.
    Dowling, J.E.: The Retina - An Approachable Part of The Brain. The Belknap press of Harvard University Press, Cambridge (1987)Google Scholar
  21. 21.
    Ebong, I.E., Mazumder, P.: Cmos and memristor-based neural network design for position detection. Proc. IEEE 100(6), 2050–2060 (2012)CrossRefGoogle Scholar
  22. 22.
    Eeckman, F.H., Colvin, M.E., Axelrod, T.S.: A retina-like model for motion detection. In: International Joint Conference on Neural Networks, 1989. IJCNN, vol. 2, pp. 247–249 (1989)Google Scholar
  23. 23.
    Wang, F.Y.: Memristor for introductory physics. ArXiv e-prints: 0808.0286 Aug 2008, Provided by the SAO/NASA Astrophysics Data SystemGoogle Scholar
  24. 24.
    Gelencsér, A., Prodromakis, T., Toumazou, C., Roska, T.: Biomimetic model of the outer plexiform layer by incorporating memristive devices. Phys. Rev. E 85, 041918 (2012)CrossRefGoogle Scholar
  25. 25.
    Gomez, G.: Local smoothness in terms of variance: the adaptive Gaussian filter, pp. 815–824 (2000)Google Scholar
  26. 26.
    Horn, B.K.P., Schunck, B.G.: Determining optical flow (1981)Google Scholar
  27. 27.
    Huber, Susanne A., Franz, Matthias O., Blthoff, Heinrich H.: On robots and flies: modeling the visual orientation behavior of flies. Rob. Auton. Syst. 29, 227–242 (1999)CrossRefGoogle Scholar
  28. 28.
    Hutchinson, J., Koch, C., Luo, J., Mead, C.: Computing motion using analog and binary resistive networks. Computer 21(3), 52–63 (1988)CrossRefGoogle Scholar
  29. 29.
    Jiang, F., Shi, B.E.: The memristive grid outperforms the resistive grid for edge preserving smoothing. In: European Conference on Circuit Theory and Design, 2009. ECCTD 2009, pp. 181–184 (2009)Google Scholar
  30. 30.
    Kameda, S., Yagi, T.: Calculating direction of motion with sustained and transient responses of silicon retina. In: SICE 2002. Proceedings of the 41st SICE Annual Conference, vol. 3, pp. 1853–1858 (2002)Google Scholar
  31. 31.
    Kameda, S., Yagi, T.: A silicon retina system that calculates direction of motion. In: Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS ’03, vol. 4, pp. IV–792–IV–795 (2003)Google Scholar
  32. 32.
    Kolb, H.: Organization of the outer plexiform layer of the primate retina: electron microscopy of Golgi impregnated cells. Philos. Trans. R. Soc. Lond. Ser. B, Biol. Sci. 258(823), 261–283 (1970)CrossRefGoogle Scholar
  33. 33.
    Lee, J.W., Chae, S.P., Kim, M.N., Kim, S.Y., Cho, J.H.: A moving detectable retina model considering the mechanism of an amacrine cell for vision. In: IEEE International Symposium on Industrial Electronics, 2001. Proceedings. ISIE 2001, vol. 1, pp. 106–109 (2001)Google Scholar
  34. 34.
    Li, H., Lin, J.C.: Using fuzzy logic to detect dimple defects of polished wafer surfaces. IEEE Trans. Ind. Appl. 30(2), 317–323 (1994)CrossRefGoogle Scholar
  35. 35.
    Lichtsteiner, P., Posch, C., Delbruck, T.: A \(128 \times 128\) 120 dB 15 s latency asynchronous temporal contrast vision sensor. IEEE J. Solid-State Circuits 43(2), 566–576 (2008)CrossRefGoogle Scholar
  36. 36.
    Likharev, K.K., Mayr, A., Muckra, I., Trel, O.: Crossnets: high-performance neuromorphic architectures for CMOL circuits. Ann. N. Y. Acad. Sci. 1006(1), 146–163 (2003)CrossRefGoogle Scholar
  37. 37.
    Linares-Barranco, B., Serrano-Gotarredona, T., Camuas-Mesa, L.A., Perez-Carrasco, J.A., Zamarre-Ramos, C., Masquelier, T.: On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front. Neurosci. 5(26) (2011)Google Scholar
  38. 38.
    Lu, Y., Jain, R.C.: Behavior of edges in scale space. IEEE Trans. Pattern Anal. Mach. Intell. 11(4), 337–356 (1989)CrossRefGoogle Scholar
  39. 39.
    Mahowald, M.: An Analog VLSI System for Stereoscopic Vision. Kluwer Academic Publishers, Norwell (1994)CrossRefGoogle Scholar
  40. 40.
    Maini, R., Aggarwal, H.: Study and comparison of various image edge detection techniques. Int. J. Image Process. 3(1), 1–12 (2009)CrossRefGoogle Scholar
  41. 41.
    Malamas, E.N., Petrakis, E.G.M., Zervakis, M., Petit, L., Legat, J.-D.: A survey on industrial vision systems, applications and tools. Image Vis. Comput. 21, 171–188 (2003)CrossRefGoogle Scholar
  42. 42.
    Masaki, I.: Machine-vision systems for intelligent transportation systems. Intell. Syst. Appl. 13(6), 24–31 (1998)CrossRefGoogle Scholar
  43. 43.
    Mead, C.: Neuromorphic electronic systems. Proc. IEEE 78(10), 1629–1636 (1990)CrossRefGoogle Scholar
  44. 44.
    Mead, C.: Analog VLSI and Neural Systems. Addison-Wesley Longman Publishing Co., Inc., Boston (1989)zbMATHGoogle Scholar
  45. 45.
    Meitzler, R.C., Strohbehn, K., Andreou, A.G.: A silicon retina for 2-D position and motion computation. In: 1995 IEEE International Symposium on Circuits and Systems, 1995. ISCAS ’95, vol. 3, pp. 2096–2099 (1995)Google Scholar
  46. 46.
    Mhani, A., Bouvier, G., Herault, J.: A contrast and motion-sensitive silicon retina. In: Solid-State Circuits Conference, 1995. ESSCIRC ’95. Twenty-first European, pp. 326–329 (1995)Google Scholar
  47. 47.
    Netravali, A.N.: Motion: computation, synthesis and applications. In: Electrical and Electronics Engineers in Israel, 1989. The Sixteenth Conference of, pp. 0–18 (1989)Google Scholar
  48. 48.
    Prodromakis, T., Bhushan, B.: Encyclopedia of Nanotechnology: Biomimetic Products. Springer, Berlin (2012)Google Scholar
  49. 49.
    Prodromakis, T., Peh, B.P., Papavassiliou, C., Toumazou, C.: A versatile memristor model with nonlinear dopant kinetics. IEEE Trans. Electron Devices 58(9), 3099–3105 (2011)CrossRefGoogle Scholar
  50. 50.
    Ray, N., Acton, S.T., Ley, K.: Tracking leukocytes in vivo with shape and size constrained active contours. IEEE Trans. Med. Imaging 21(10), 1222–1235 (2002)CrossRefGoogle Scholar
  51. 51.
    Regan, D.: Human Perception of Objects. Sinauer Associates Inc, Publishers, USA (2000)Google Scholar
  52. 52.
    Roska, B., Werblin, F.: Vertical interactions across ten parallel, stacked representations in the mammalian retina. Nature 410, 583–587 (2001)CrossRefGoogle Scholar
  53. 53.
    Salaoru, I., Prodromakis, T., Khiat, A., Toumazou, C.: Resistive switching of oxygen enhanced \(\text{ TiO }_{2}\) thin-film devices. Appl. Phys. Lett. 102(1), 013506 (2013)CrossRefGoogle Scholar
  54. 54.
    Sanz, J.L.C., Petkovic, D.: Machine vision algorithms for automated inspection thin-film disk heads. IEEE Trans. Pattern Anal. Mach. Intell. 10(6), 830–848 (1988)CrossRefGoogle Scholar
  55. 55.
    Saripalli, S., Montgomery, J.F., Sukhatme, G.S.: Visually guided landing of an unmanned aerial vehicle. IEEE Trans. Rob. Autom. 19(3), 371–380 (2003)CrossRefGoogle Scholar
  56. 56.
    Serrano-Gotarredona, T., Masquelier, T., Prodromakis, T., Indiveri, G., Linares-Barranco, B.: STDP and STDP variations with memristors for spiking neuromorphic learning systems. Front. Neurosci. 7(2), (2013)Google Scholar
  57. 57.
    Shafarenko, L., Petrou, M., Kittler, J.: Automatic watershed segmentation of randomly textured color images. Trans. Img. Proc. 6(11), 1530–1544 (1997)CrossRefGoogle Scholar
  58. 58.
    Sotelo, M.A., Rodriguez, F.J., Magdalena, L.: Virtuous: vision-based road transportation for unmanned operation on urban-like scenarios. IEEE Trans. Intell. Transp. Syst. 5(2), 69–83 (2004)CrossRefGoogle Scholar
  59. 59.
    Steger, C., MUlrich, C., Wiedemann, C.: Machine Vision Algorithms and Applications (2008)Google Scholar
  60. 60.
    Strukov, D.B., Likharev, K.K.: Cmol fpga: a reconfigurable architecture for hybrid digital circuits with two-terminal nanodevices. Nanotechnology 16(6), 888 (2005)CrossRefGoogle Scholar
  61. 61.
    Strukov, D.B., Snider, G.S., Stewart, D.R., Williams, R.S.: The missing memristor found. Nature 453(7191), 80–83 (2008)CrossRefGoogle Scholar
  62. 62.
    Inc The MathWorks. MATLAB Image Processing Toolbox - User’s Guide R2012b. The MathWorks, Inc, 3 Apple Hill Drive, Natick, MA 01760-2098 (2012)Google Scholar
  63. 63.
    Toumazou, C.: Bio-inspired technologies. Ingenia, 37, Dec 2008Google Scholar
  64. 64.
    Turel, Ö., Lee, J.H., Ma, X., Likharev, K.K.: Neuromorphic architectures for nanoelectronic circuits: research articles. Int. J. Circuit Theory Appl. 32(5), 277–302 (2004)CrossRefGoogle Scholar
  65. 65.
    Ullman, S.: Analysis of visual motion by biological and computer systems. Computer 14(8), 57–69 (1981)CrossRefGoogle Scholar
  66. 66.
    Wandell, B.A.: Foundation of Vision, 1st edn. Sinauer Associates Inc, USA (1955)Google Scholar
  67. 67.
    Williams, R.: How we found the missing memristor. Spectrum, IEEE 45(12), 28–35 (2008)CrossRefGoogle Scholar
  68. 68.
    Yang, W.-C., Lin, L.-J., Chiueh, H., Wu, C.-Y.: A CMOS bio-inspired 2-D motion direction sensor based on a direction computation method derived from the directionally selective ganglion cells in the retina. Sens. J. 11(12), 3341–3351 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Chuan Kai Kenneth Lim
    • 1
  • A. Gelencser
    • 2
  • T. Prodromakis
    • 3
    Email author
  1. 1.Centre for Bio-inspired Technology, Department of Electrical and Electronic EngineeringImperial College LondonLondonUK
  2. 2.Interdisciplinary Technical Sciences Doctoral SchoolPázmány Péter Catholic UniversityBudapestHungary
  3. 3.Zepler Institute for Photonics and NanoelectronicsUniversity of SouthamptonSouthamptonUK

Personalised recommendations