Abstract
Depth perception through stereo vision is an important feature of biological and artificial vision systems. While biological systems can compute disparities effortlessly, it requires intensive processing for artificial vision systems. The computing complexity resides in solving the correspondence problem – finding matching pairs of points in the two eyes. Inspired by the retina, event-based vision sensors allow a new constraint to solve the correspondence problem: time. Relying on precise spike-time, spiking neural networks can take advantage of this constraint. However, disparities can only be computed from dynamic environments since event-based vision sensors only report local changes in light intensity. In this paper, we show how microsaccadic eye movements can be used to compute disparities from static environments. To this end, we built a robotic head supporting two Dynamic Vision Sensors (DVS) capable of independent panning and simultaneous tilting. We evaluate the method on both static and dynamic scenes perceived through microsaccades. This paper demonstrates the complementarity of event-based vision sensors and active perception leading to more biologically inspired robots.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Davies, E.R.: Computer and Machine Vision: Theory, Algorithms, Practicalities. Academic Press, Cambridge (2012)
Davison, A.P.: PyNN: a common interface for neuronal network simulators. Front. Neuroinform. 2, 11 (2008)
Dikov, G., Mohsen, F., Röhrbein, F., Conradt, J., Richter, C.: Spiking cooperative stereo-matching at 2 ms latency with neuromorphic hardware. Front. Neurosci. (2017)
Dodgson, N.A.: Variation and extrema of human interpupillary distance. Proc. Soc. Photo-Opt. Instrum. Eng. 12(8), 36–46 (2004)
Furber, S., Temple, S., Brown, A.: On-chip and inter-chip networks for modelling large-scare neural systems, pp. 6–9 (2006)
Furber, S.B., Galluppi, F., Temple, S., Plana, L.A.: The spinnaker project. Proc. IEEE 102(5), 652–665 (2014)
Gerstner, W., Kistler, W.M.: Spiking Neuron Models: Single Neurons, Populations. Plasticity. Cambridge University Press, Cambridge (2002)
Gewaltig, M.O., Diesmann, M.: Nest (neural simulation tool). Scholarpedia 2(4), 1430 (2007)
Hermann, A., et al.: Hardware and software architecture of the bimanual mobile manipulation robot HoLLiE and its actuated upper body. In: 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics: Mechatronics for Human Wellbeing, AIM 2013, pp. 286–292, July 2013
Kaiser, J., et al.: Benchmarking microsaccades for feature extraction with spiking neural networks on continuous event streams. In: International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob) (2018, submitted)
Lichtsteiner, P., Posch, C., Delbruck, T.: A \(128\, \times \,128\) 120 db 15 \(\mu \)s latency asynchronous temporal contrast vision sensor. IEEE J. Solid-State Circuits 43(2), 566–576 (2008)
Maass, W.: Networks of spiking neurons: the third generation of neural network models. Neural Netw. 10(9), 1659–1671 (1997)
Marr, D.: Vision: a computational investigation into the human representation and processing of visual information. W.H. Freeman and Company, San Francisco (1982)
Marr, D., Poggio, T.: A theory of human stereo vision. Proc. Roy. Soc. Lond. B Biol. Sci. 204, 301–328 (1977)
Martinez-Conde, S., Macknik, S.L., Hubel, D.H.: The role of fixational eye movements in visual perception. Nat. Rev. Neurosci. 5(3), 229–240 (2004)
Masquelier, T., Portelli, G., Kornprobst, P.: Microsaccades enable efficient synchrony-based coding in the retina: a simulation study. Sci. Rep. 6, 24086 (2016)
Mueggler, E., Huber, B., Scaramuzza, D.: Event-based, 6-DOF pose tracking for high-speed maneuvers. In: International Conference on Intelligent Robots and Systems. IEEE (2014)
Orchard, G., Jayawant, A., Cohen, G., Thakor, N.: Converting static image datasets to spiking neuromorphic datasets using saccades. arXiv preprint arXiv:1507.07629 (2015)
Osswald, M., Ieng, S.H., Benosman, R., Indiveri, G.: A Spiking Neural Network Model of 3D Perception For Event-Based Neuromorphic Stereo Vision Systems, pp. 1–11. Nature Publishing Group, London (2017)
Osswald, M., Ieng, S.H., Benosman, R., Indiveri, G.: Supplementary Material: A Spiking Neural Network Model of 3D Perception for Event-Based Neuromorphic Stereo Vision Systems, pp. 1–14 (2017)
Rucci, M., Victor, J.D.: The unsteady eye: an information-processing stage, not a bug. Trends Neurosci. 38(4), 195–206 (2015)
Acknowledgments
This research has received funding from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 720270 (Human Brain Project SGA1) and No. 785907 (Human Brain Project SGA2).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Kaiser, J. et al. (2018). Microsaccades for Neuromorphic Stereo Vision. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds) Artificial Neural Networks and Machine Learning – ICANN 2018. ICANN 2018. Lecture Notes in Computer Science(), vol 11139. Springer, Cham. https://doi.org/10.1007/978-3-030-01418-6_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-01418-6_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01417-9
Online ISBN: 978-3-030-01418-6
eBook Packages: Computer ScienceComputer Science (R0)