Abstract
Event-based gaze detection is a modern problem having several applications and advantages over frame-based techniques. Retinomorphic Event data is logged at a time resolution of microseconds that makes them suitable for the detection of saccadic eye movements. We recorded a new and compact event-based dataset for gaze detection under varying conditions of illumination using a DVS camera. The recorded dataset involved subjects tracking a circle displayed on a screen within a very short duration of time. We propose a novel event encoding technique for encoding event logs resulting from saccadic motion into six channel images. We design a Convolutional Neural Network for the gaze prediction using the encoded events obtained from the retinomorphic sensor. We use multiple evaluation metrics like average distance, average angle, and pixel radius accuracy to validate the reliability of our approach. The recorded dataset will be made available as per request.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Rayner, K., Castelhano, M.: Eye movements. Scholarpedia 2(10), 3649 (2007)
Findlay, J., Walker, R.: Human saccadic eye movements. Scholarpedia 7(7), 5095 (2012)
Cheng, W., Luo, H., Yang, W., Yu, L., Chen, S., Li, W.: Det: a high-resolution dvs dataset for lane extraction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (2019)
Gallego, G., et al.: Event-based vision: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(1), 154–180 (2020)
Dynamic vision sensor (2022). https://inivation.com/products/customsolutions/videos/ Accessed 13 Apr 2022
Baby, S.A., Vinod, B., Chinni, C., Mitra, K.: Dynamic vision sensors for human activity recognition. In: 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), pp. 316–321. IEEE (2017)
Wan, J., et al.: Event-based pedestrian detection using dynamic vision sensors. Electronics 10(8), 888 (2021)
Liao, F., Zhou, F., Chai, Y.: Neuromorphic vision sensors: principle, progress and perspectives. J. Semicond. 42(1), 013105 (2021)
Köles, M.: A review of pupillometry for human-computer interaction studies. Periodica Polytechnica Electr. Eng. Comput. Sci. 61(4), 320–326 (2017)
Lukander, K.: A short review and primer on eye tracking in human computer interaction applications (2016)
Corcoran, P.M., Nanu, F., Petrescu, S., Bigioi, P.: Real-time eye gaze tracking for gaming design and consumer electronics systems. IEEE Trans. Cons. Electron. 58(2), 347–355 (2012)
Brunyé, T.T., Drew, T., Weaver, D.L., Elmore, J.G.: A review of eye tracking for understanding and improving diagnostic interpretation. Cogn. Res.: Principles Impl. 4 (2019)
Young, L.R., Sheena, D.: Survey of eye movement recording methods. Behav. Res. Methods Instr. 7, 397–429 (1975)
Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: Appearance-based gaze estimation in the wild. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4511–4520 (2015)
Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: Mpiigaze: real-world dataset and deep appearance-based gaze estimation. IEEE Trans. Pattern Anal. Mach. Intell. 41(1), 162–175 (2019)
Cornsweet, T.N., Crane, H.D.: Accurate two-dimensional eye tracker using first and fourth purkinje images. J. Opt. Soc. Am. 63(8), 921–928 (1973)
Crane, H.D., Steele, C.M.: Generation-v dual-purkinje-image eyetracker. Appl. Opt. 24(4), 527–537 (1985)
Li, Y., Wang, S., Ding, X.: Eye/eyes tracking based on a unified deformable template and particle filtering. Pattern Recogn. Lett. 31(11), 1377–1387 (2010)
Wang, K., Ji, Q.: Real time eye gaze tracking with 3D deformable eye-face model. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 1003–1011 (2017)
Topal, C., Gerek, Ö.N., Doğan, A.: A head-mounted sensor-based eye tracking device: eye touch system. In: ETRA 2008 (2008)
Akşit, K., Kautz, J., Luebke, D.: Gaze-sensing leds for head mounted displays (2020)
Vogel, U., et al.: Bidirectional oled microdisplay for interactive see-through hmds: study toward integration of eye-tracking and informational facilities. J. Soc. Inf. Disp. 17, 03 (2009)
Angelopoulos, A.N., Martel, J.N.P., Kohli, A.P., Conradt, J., Wetzstein, G.: Event-based near-eye gaze tracking beyond 10,000 hz. IEEE Trans. Vis. Comput. Graph. 27(5), 2577–2586 (2021)
Stoffregen, T., Daraei, H., Robinson, C., Fix, A.: Event-based kilohertz eye tracking using coded differential lighting. In: 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 3937–3945 (2022)
Mueggler, E., Forster, C., Baumli, N., Gallego, G., Scaramuzza, D.: Lifetime estimation of events from dynamic vision sensors. In: 2015 IEEE international conference on Robotics and Automation (ICRA), pp. 4874–4881. IEEE (2015)
Acknowledgements
All computations were performed using the GPU resources provided by the AI Computing Facility, CSIR-CEERI. The authors sincerely appreciate the willingness of the contributing subjects.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Banerjee, A., Prasad, S.S., Mehta, N.K., Kumar, H., Saurav, S., Singh, S. (2023). Gaze Detection Using Encoded Retinomorphic Events. In: Zaynidinov, H., Singh, M., Tiwary, U.S., Singh, D. (eds) Intelligent Human Computer Interaction. IHCI 2022. Lecture Notes in Computer Science, vol 13741. Springer, Cham. https://doi.org/10.1007/978-3-031-27199-1_44
Download citation
DOI: https://doi.org/10.1007/978-3-031-27199-1_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-27198-4
Online ISBN: 978-3-031-27199-1
eBook Packages: Computer ScienceComputer Science (R0)