Skip to main content
SpringerLink
Log in

Remote gaze estimation based on 3D face structure and iris centers under natural light

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Remote gaze estimation under natural light is still a challenging problem. Appearance based methods are seriously sensitive to illumination variation in the visual spectrum and usually can hardly handle the problem of head movements. And most existing feature-based gaze estimation methods strongly rely on cornea reflections, which are unstable to glasses, head movements and especially useless for natural light condition. In this paper, we propose a novel feature based gaze estimation method without use of cornea reflections. A stereo camera system is built for the proposed method. Firstly, 3D Active Shape Models (ASM) is reconstructed using stereo vision to represent 3D face structure. Then, without use of cornea reflections, a 3D Iris-Eye-Contours based descriptor is proposed to represent human gaze information. Iris centers are used in natural light just like the pupil centers in condition of near-infrared light. What’s more, precise estimation of head poses based on 3D face structure is employed to rectify the 3D iris centers and eye contours for improving the ability of tolerance to head movements. Experiments on several subjects show that the system is accurate and allows natural head movements under natural light.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Baluja S, Pomerleau D (1994) Non-intrusive gaze tracking using artificial neural networks. In: Adv Neural Inf Process Syst, pp 753–760

  2. Cerrolaza JJ, Villanueva A, Cabeza R (2008) Taxonomic study of polynomial regressions applied to the calibration of video-oculographic systems. In: Proceedings eye tracking research and applications (ETRA), pp 259–266

  3. Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models-their training and application. Comput Vis and Image Underst 61(1):38–59

    Article  Google Scholar 

  4. Duchowski AT (2002) A breadth-first survey of eye-tracking applications. Behav Res Methods Instrum Comput 34(4):455–470

    Article  Google Scholar 

  5. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. ACM Commun 24(6):381–395

    Article  MathSciNet  Google Scholar 

  6. Funes Mora K, Odobez JM (2012) Gaze estimation from multimodal kinect data. In: IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 25–30

  7. Golub GH, Reinsch C (1970) Singular value decomposition and least squares solutions. Numerische Mathematik 14(5):403–420

    Article  MathSciNet  MATH  Google Scholar 

  8. Group T (2015) Tobii technology products. Accessed 5 January 2014. http://www.tobii.com

  9. Hansen DW, Ji Q (2010) In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans Pattern Anal Mach Intell 32(3):478–500

    Article  Google Scholar 

  10. Jafari R, Ziou D (2015) Eye-gaze estimation under various head positions and iris states. Expert Syst Appl 42(1):510–518

    Article  Google Scholar 

  11. Lin YT, Lin R Y, Lin YC, Lee GC (2013) Real-time eye-gaze estimation using a low-resolution webcam. Multimed Tools Appl 65(3):543–568

    Article  Google Scholar 

  12. Loy G, Zelinsky A (2003) Fast radial symmetry for detecting points of interest. IEEE Trans Pattern Anal Mach Intell 25(8):959–973

    Article  MATH  Google Scholar 

  13. Lu F, Okabe T, Sugano Y, Sato Y (2011a) A head pose-free approach for appearance-based gaze estimation. In: Proceedings British machine vision conference (BMVC), pp 1–11

  14. Lu F, Sugano Y, Okabe T, Sato Y (2011b) Inferring human gaze from appearance via adaptive linear regression. In: Proceedings IEEE Int Conf Comput Vis (ICCV), pp 153–160

  15. Martinez F, Carbone A, Pissaloux E (2012) Gaze estimation using local features and non-linear regression. In: IEEE international conference on image processing (ICIP), pp 1961–1964

  16. Mora KAF, Odobez JM (2014) Geometric generative gaze estimation for remote rgb-d cameras. In: IEEE Comput Soc Conf Comput Vis Pattern Recognit (CVPR), pp 1773–1780

  17. Morimoto CH, Koons D, Amir A, Flickner M (2000) Pupil detection and tracking using multiple light sources. Image Vis Comput 18(4):331–335

    Article  Google Scholar 

  18. Schneider T, Schauerte B, Stiefelhagen R (2014) Manifold alignment for person independent appearance-based gaze estimation. In: IEEE international conference on pattern recognition (ICPR)

  19. Sesma L, Villanueva A, Cabeza R (2012) Evaluation of pupil center-eye corner vector for gaze estimation using a web cam. In: Proceedings eye tracking research and applications (ETRA), pp 217–220

  20. Sesma-Sanchez L, Villanueva A, Cabeza R (2012) Gaze estimation interpolation methods based on binocular data. IEEE Trans Biomed Eng 59(8):2235–2243

    Article  Google Scholar 

  21. Sugano Y, Matsushita Y, Sato Y, Koike H (2008) An incremental learning method for unconstrained gaze estimation. In: Proceedings European conference on computer vision (ECCV), pp 656–667

  22. Sugano Y, Matsushita Y, Sato Y (2010) Calibration-free gaze sensing using saliency maps. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2667–2674

  23. Sugano Y, Matsushita Y, Sato Y (2013) Appearance-based gaze estimation using visual saliency. IEEE Trans Pattern Anal Mach Intell 35(2):329–341

    Article  Google Scholar 

  24. Sugano Y, Matsushita Y, Sato Y (2014) Learning-by-synthesis for appearance-based 3d gaze estimation. In: IEEE international conference on computer vision and pattern recognition (CVPR)

  25. Tropp J A, Gilbert A C (2007) Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans Inf Theory 53(12):4655–4666

    Article  MathSciNet  MATH  Google Scholar 

  26. Valenti R, Sebe N, Gevers T (2012) Combining head pose and eye location information for gaze estimation. IEEE Trans Image Process 21(2):802–815

    Article  MathSciNet  Google Scholar 

  27. Williams O, Blake A, Cipolla R (2006) Sparse and semi-supervised visual mapping with the sˆ 3gp. In: IEEE conference on computer vision and pattern recognition (CVPR), vol 1, pp 230–237

  28. Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Elsevier Chemom Intell Lab Syst 2(1):37–52

    Article  Google Scholar 

  29. Xiong C, Huang L, Liu C (2014) Gaze estimation based on 3d face structure and pupil centers. In: IEEE international conference on pattern recognition (ICPR), pp 1156–1161

  30. Xiong P, Huang L, Liu C (2010) Initialization and pose alignment in active shape model. In: IEEE international conference on pattern recognition (ICPR), pp 3971–3974

  31. Zhu Z W, Ji Q (2004) Eye and gaze tracking for interactive graphic display. Mach Vis Appl 15(3):139–148

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank all the subjects who highly cooperate with the test process. The authors would also like to thank the Data Research Group at Hanvon Technology for helping to prepare the dataset.

Author information

Authors and Affiliations

  1. Institute of Automation, Chinese Academy of Sciences, No. 95, Zhongguancun East Road, Haidian District, Beijing, 100190, China

    Chunshui Xiong

  2. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Lei Huang & Changping Liu

Authors
  1. Chunshui Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Changping Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunshui Xiong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiong, C., Huang, L. & Liu, C. Remote gaze estimation based on 3D face structure and iris centers under natural light. Multimed Tools Appl 75, 11785–11799 (2016). https://doi.org/10.1007/s11042-015-2600-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2600-y

Keywords

Search

Navigation