Advertisement

Appearance-Based Gaze Estimation via Evaluation-Guided Asymmetric Regression

  • Yihua Cheng
  • Feng LuEmail author
  • Xucong Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11218)

Abstract

Eye gaze estimation has been increasingly demanded by recent intelligent systems to accomplish a range of interaction-related tasks, by using simple eye images as input. However, learning the highly complex regression between eye images and gaze directions is nontrivial, and thus the problem is yet to be solved efficiently. In this paper, we propose the Asymmetric Regression-Evaluation Network (ARE-Net), and try to improve the gaze estimation performance to its full extent. At the core of our method is the notion of “two eye asymmetry” observed during gaze estimation for the left and right eyes. Inspired by this, we design the multi-stream ARE-Net; one asymmetric regression network (AR-Net) predicts 3D gaze directions for both eyes with a novel asymmetric strategy, and the evaluation network (E-Net) adaptively adjusts the strategy by evaluating the two eyes in terms of their performance during optimization. By training the whole network, our method achieves promising results and surpasses the state-of-the-art methods on multiple public datasets.

Keywords

Gaze estimation Eye appearance Asymmetric regression 

References

  1. 1.
    Zhang, X., Sugano, Y., Bulling, A.: Everyday eye contact detection using unsupervised gaze target discovery. In: Proceedings of the ACM Symposium on User Interface Software and Technology (UIST), pp. 193–203 (2017)Google Scholar
  2. 2.
    Sugano, Y., Zhang, X., Bulling, A.: Aggregaze: collective estimation of audience attention on public displays. In: Proceedings of the ACM Symposium on User Interface Software and Technology (UIST), pp. 821–831 (2016)Google Scholar
  3. 3.
    Sun, X., Yao, H., Ji, R., Liu, X.M.: Toward statistical modeling of saccadic eye-movement and visual saliency. IEEE Trans. Image Process. 23(11), 4649 (2014)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Cheng, Q., Agrafiotis, D., Achim, A., Bull, D.: Gaze location prediction for broadcast football video. IEEE Trans. Image Process. 22(12), 4918–4929 (2013)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Hansen, D., Ji, Q.: In the eye of the beholder: A survey of models for eyes and gaze. IEEE Trans. PAMI 32(3), 478–500 (2010)CrossRefGoogle Scholar
  6. 6.
    Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: Appearance-based gaze estimation in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4511–4520 (2015)Google Scholar
  7. 7.
    Zhu, W., Deng, H.: Monocular free-head 3D gaze tracking with deep learning and geometry constraints. In: The IEEE International Conference on Computer Vision (ICCV) (2017)Google Scholar
  8. 8.
    Lepetit, V., Moreno-Noguer, F., Fua, P.: EPNP: an accurate o(n) solution to the pnp problem. Int. J. Comput. Vis. 81(2), 155 (2008)CrossRefGoogle Scholar
  9. 9.
    Morimoto, C., Mimica, M.: Eye gaze tracking techniques for interactive applications. CVIU 98(1), 4–24 (2005)Google Scholar
  10. 10.
    Guestrin, E., Eizenman, M.: General theory of remote gaze estimation using the pupil center and corneal reflections. IEEE Trans. Biomed. Eng. 53(6), 1124–1133 (2006)CrossRefGoogle Scholar
  11. 11.
    Zhu, Z., Ji, Q.: Novel eye gaze tracking techniques under natural head movement. IEEE Trans. Biomed. Eng. J. 54(12), 2246–2260 (2007)CrossRefGoogle Scholar
  12. 12.
    Nakazawa, A., Nitschke, C.: Point of gaze estimation through corneal surface reflection in an active illumination environment. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, pp. 159–172. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33709-3_12CrossRefGoogle Scholar
  13. 13.
    Valenti, R., Sebe, N., Gevers, T.: Combining head pose and eye location information for gaze estimation. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 21(2), 802–815 (2012)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Jeni, L.A., Cohn, J.F.: Person-independent 3d gaze estimation using face frontalization. In: Computer Vision and Pattern Recognition Workshops, pp. 792–800 (2016)Google Scholar
  15. 15.
    Funes Mora, K.A., Odobez, J.M.: Geometric generative gaze estimation (g3e) for remote RGB-D cameras. In: IEEE Computer Vision and Pattern Recognition Conference, pp. 1773–1780 (2014)Google Scholar
  16. 16.
    Xiong, X., Liu, Z., Cai, Q., Zhang, Z.: Eye gaze tracking using an RGBD camera: a comparison with a RGB solution. The 4th International Workshop on Pervasive Eye Tracking and Mobile Eye-Based Interaction (PETMEI 2014), pp. 1113–1121 (2014)Google Scholar
  17. 17.
    Wang, K., Ji, Q.: Real time eye gaze tracking with 3d deformable eye-face model. In: The IEEE International Conference on Computer Vision (ICCV) (2017)Google Scholar
  18. 18.
    Duchowski, A.T.: A breadth-first survey of eye-tracking applications. Behav. Res. Methods Instrum. Comput. 34(4), 455–470 (2002)CrossRefGoogle Scholar
  19. 19.
    Tan, K., Kriegman, D., Ahuja, N.: Appearance-based eye gaze estimation. In: WACV, pp. 191–195 (2002)Google Scholar
  20. 20.
    Baluja, S., Pomerleau, D.: Non-Intrusive Gaze Tracking Using Artificial Neural Networks. Carnegie Mellon University (1994)Google Scholar
  21. 21.
    Xu, L.Q., Machin, D., Sheppard, P.: A novel approach to real-time non-intrusive gaze finding. In: BMVC, pp. 428–437 (1998)Google Scholar
  22. 22.
    Lu, F., Sugano, Y., Okabe, T., Sato, Y.: Adaptive linear regression for appearance-based gaze estimation. IEEE Trans. Pattern Anal. Mach. Intell. 36(10), 2033–2046 (2014)CrossRefGoogle Scholar
  23. 23.
    Williams, O., Blake, A., Cipolla, R.: Sparse and semi-supervised visual mapping with the S3GP. In: CVPR, pp. 230–237(2006)Google Scholar
  24. 24.
    Schneider, T., Schauerte, B., Stiefelhagen, R.: Manifold alignment for person independent appearance-based gaze estimation. In: International Conference on Pattern Recognition (ICPR), pp. 1167–1172 (2014)Google Scholar
  25. 25.
    Lu, F., Chen, X., Sato, Y.: Appearance-based gaze estimation via uncalibrated gaze pattern recovery. IEEE Trans. Image Process. 26(4), 1543–1553 (2017)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Sugano, Y., Matsushita, Y., Sato, Y., Koike, H.: Appearance-based gaze estimation with online calibration from mouse operations. IEEE Trans. Hum. Mach. Syst. 45(6), 750–760 (2015)CrossRefGoogle Scholar
  27. 27.
    Mora, K.A.F., Monay, F., Odobez, J.M.: Eyediap:a database for the development and evaluation of gaze estimation algorithms from RGB and RGB-D cameras. In: Symposium on Eye Tracking Research and Applications, pp. 255–258 (2014)Google Scholar
  28. 28.
    Wood, E., Morency, L.P., Robinson, P., Bulling, A.: Learning an appearance-based gaze estimator from one million synthesised images. In: Biennial ACM Symposium on Eye Tracking Research & Applications, pp. 131–138 (2016)Google Scholar
  29. 29.
    Krafka, K., et al.: Eye tracking for everyone. In: Computer Vision and Pattern Recognition, pp. 2176–2184 (2016)Google Scholar
  30. 30.
    Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: It’s written all over your face: Full-face appearance-based gaze estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2017)Google Scholar
  31. 31.
    Lu, F., Sugano, Y., Okabe, T., Sato, Y.: Head pose-free appearance-based gaze sensing via eye image synthesis. In: International Conference on Pattern Recognition, pp. 1008–1011 (2012)Google Scholar
  32. 32.
    Sugano, Y., Matsushita, Y., Sato, Y.: Appearance-based gaze estimation using visual saliency. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 329 (2013)CrossRefGoogle Scholar
  33. 33.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012)Google Scholar
  34. 34.
    Sugano, Y., Matsushita, Y., Sato, Y.: Learning-by-synthesis for appearance-based 3D gaze estimation. In: Computer Vision and Pattern Recognition, pp. 1821–1828 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and EngineeringBeihang UniversityBeijingChina
  2. 2.Beijing Advanced Innovation Center for Big Data-Based Precision MedicineBeihang UniversityBeijingChina
  3. 3.Max Planck Institute for Informatics, Saarland Informatics CampusSaarbrückenGermany

Personalised recommendations