Skip to main content
SpringerLink
Log in

Appearance-Based Gaze Estimation Using Dilated-Convolutions

  • Conference paper
  • First Online:
Computer Vision – ACCV 2018 (ACCV 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11366))

Included in the following conference series:

Abstract

Appearance-based gaze estimation has attracted more and more attention because of its wide range of applications. The use of deep convolutional neural networks has improved the accuracy significantly. In order to improve the estimation accuracy further, we focus on extracting better features from eye images. Relatively large changes in gaze angles may result in relatively small changes in eye appearance. We argue that current architectures for gaze estimation may not be able to capture such small changes, as they apply multiple pooling layers or other downsampling layers so that the spatial resolution of the high-level layers is reduced significantly. To evaluate whether the use of features extracted at high resolution can benefit gaze estimation, we adopt dilated-convolutions to extract high-level features without reducing spatial resolution. In cross-subject experiments on the Columbia Gaze dataset for eye contact detection and the MPIIGaze dataset for 3D gaze vector regression, the resulting Dilated-Nets achieve significant (up to 20.8%) gains when compared to similar networks without dilated-convolutions. Our proposed Dilated-Net achieves state-of-the-art results on both the Columbia Gaze and the MPIIGaze datasets.

Supported by the Innovation and Technology Fund of Hong Kong under grant ITS/406/16FP.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baltrusaitis, T., Zadeh, A., Lim, Y.C., Morency, L.P.: OpenFace 2.0: facial behavior analysis toolkit. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 59–66. IEEE (2018)

    Google Scholar 

  2. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27 (2011)

    Article  Google Scholar 

  3. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv preprint arXiv:1412.7062 (2014)

  4. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)

    Article  Google Scholar 

  5. Chen, Z., Shi, B.E.: Using variable dwell time to accelerate gaze-based web browsing with two-step selection. Int. J. Hum.-Comput. Interact. 35, 240–255 (2018)

    Article  Google Scholar 

  6. Deng, H., Zhu, W.: Monocular free-head 3D gaze tracking with deep learning and geometry constraints. In: IEEE International Conference on Computer Vision, pp. 3162–3171. IEEE (2017)

    Google Scholar 

  7. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  8. Funes Mora, K.A., Monay, F., Odobez, J.M.: EYEDIAP: a database for the development and evaluation of gaze estimation algorithms from RGB and RGB-D cameras. In: ACM Symposium on Eye Tracking Research & Applications, pp. 255–258. ACM (2014)

    Google Scholar 

  9. George, A., Routray, A.: Real-time eye gaze direction classification using convolutional neural network. In: International Conference on Signal Processing and Communications, pp. 1–5. IEEE (2016)

    Google Scholar 

  10. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385 (2015)

  11. Hoppe, S., Loetscher, T., Morey, S.A., Bulling, A.: Eye movements during everyday behavior predict personality traits. Front. Hum. Neurosci. 12, 105 (2018)

    Article  Google Scholar 

  12. Huang, C.M., Mutlu, B.: Anticipatory robot control for efficient human-robot collaboration. In: ACM/IEEE International Conference on Human Robot Interaction, pp. 83–90. IEEE (2016)

    Google Scholar 

  13. Ioffe, S.: Batch renormalization: towards reducing minibatch dependence in batch-normalized models. In: Advances in Neural Information Processing Systems, pp. 1942–1950. MIT Press (2017)

    Google Scholar 

  14. King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10(Jul), 1755–1758 (2009)

    Google Scholar 

  15. Krafka, K., et al.: Eye tracking for everyone. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2176–2184. IEEE (2016)

    Google Scholar 

  16. Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)

  17. van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov), 2579–2605 (2008)

    Google Scholar 

  18. Parekh, V., Subramanian, R., Jawahar, C.V.: Eye contact detection via deep neural networks. In: Stephanidis, C. (ed.) HCI 2017. CCIS, vol. 713, pp. 366–374. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58750-9_51

    Chapter  Google Scholar 

  19. Patney, A., et al.: Towards foveated rendering for gaze-tracked virtual reality. ACM Trans. Graph. 35(6), 179 (2016)

    Article  MathSciNet  Google Scholar 

  20. Pi, J., Shi, B.E.: Probabilistic adjustment of dwell time for eye typing. In: International Conference on Human System Interactions, pp. 251–257. IEEE (2017)

    Google Scholar 

  21. Ranjan, R., De Mello, S., Kautz, J.: Light-weight head pose invariant gaze tracking. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2156–2164. IEEE (2018)

    Google Scholar 

  22. Schneider, T., Schauerte, B., Stiefelhagen, R.: Manifold alignment for person independent appearance-based gaze estimation. In: International Conference on Pattern Recognition, pp. 1167–1172. IEEE (2014)

    Google Scholar 

  23. Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., Webb, R.: Learning from simulated and unsupervised images through adversarial training. In: IEEE International Conference on Computer Vision, pp. 2242–2251. IEEE (2017)

    Google Scholar 

  24. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  25. Smith, B.A., Yin, Q., Feiner, S.K., Nayar, S.K.: Gaze locking: passive eye contact detection for human-object interaction. In: ACM Symposium on User Interface Software and Technology, pp. 271–280. ACM (2013)

    Google Scholar 

  26. Su, H., Qi, C.R., Li, Y., Guibas, L.J.: Render for CNN: viewpoint estimation in images using CNNs trained with rendered 3D model views. In: IEEE International Conference on Computer Vision, pp. 2686–2694 (2015)

    Google Scholar 

  27. Sugano, Y., Matsushita, Y., Sato, Y.: Learning-by-synthesis for appearance-based 3D gaze estimation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1821–1828. IEEE (2014)

    Google Scholar 

  28. Wang, H., Pi, J., Qin, T., Shen, S., Shi, B.E.: SLAM-based localization of 3D gaze using a mobile eye tracker. In: ACM Symposium on Eye Tracking Research & Applications, p. 65. ACM (2018)

    Google Scholar 

  29. Wang, P., et al.: Understanding convolution for semantic segmentation. arXiv preprint arXiv:1702.08502 (2017)

  30. Wood, E., Baltrusaitis, T., Zhang, X., Sugano, Y., Robinson, P., Bulling, A.: Rendering of eyes for eye-shape registration and gaze estimation. In: IEEE International Conference on Computer Vision, pp. 3756–3764. IEEE (2015)

    Google Scholar 

  31. Ye, Z., Li, Y., Liu, Y., Bridges, C., Rozga, A., Rehg, J.M.: Detecting bids for eye contact using a wearable camera. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 1–8. IEEE (2015)

    Google Scholar 

  32. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122 (2015)

  33. Yu, F., Koltun, V., Funkhouser, T.: Dilated residual networks. In: IEEE International Conference on Computer Vision, pp. 636–644. IEEE (2017)

    Google Scholar 

  34. Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: Appearance-based gaze estimation in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4511–4520. IEEE (2015)

    Google Scholar 

  35. Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: It’s written all over your face: full-face appearance-based gaze estimation. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2299–2308. IEEE (2017)

    Google Scholar 

  36. Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: MPIIGaze: real-world dataset and deep appearance-based gaze estimation. IEEE Trans. Pattern Anal. Mach. Intell. 41, 162–175 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Hong Kong University of Science and Technology, Kowloon, Hong Kong SAR

    Zhaokang Chen & Bertram E. Shi

Authors
  1. Zhaokang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bertram E. Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhaokang Chen .

Editor information

Editors and Affiliations

  1. IIIT Hyderabad, Hyderabad, India

    C.V. Jawahar

  2. ANU, Canberra, ACT, Australia

    Hongdong Li

  3. Simon Fraser University, Burnaby, BC, Canada

    Greg Mori

  4. ETH Zurich, Zurich, Zürich, Switzerland

    Konrad Schindler

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, Z., Shi, B.E. (2019). Appearance-Based Gaze Estimation Using Dilated-Convolutions. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11366. Springer, Cham. https://doi.org/10.1007/978-3-030-20876-9_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-20876-9_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20875-2

  • Online ISBN: 978-3-030-20876-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation