Pose Normalization for Eye Gaze Estimation and Facial Attribute Description from Still Images

  • Bernhard EggerEmail author
  • Sandro Schönborn
  • Andreas Forster
  • Thomas Vetter
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8753)


Our goal is to obtain an eye gaze estimation and a face description based on attributes (e.g. glasses, beard or thick lips) from still images. An attribute-based face description reflects human vocabulary and is therefore adequate as face description. Head pose and eye gaze play an important role in human interaction and are a key element to extract interaction information from still images. Pose variation is a major challenge when analyzing them. Most current approaches for facial image analysis are not explicitly pose-invariant. To obtain a pose-invariant representation, we have to account the three dimensional nature of a face. A 3D Morphable Model (3DMM) of faces is used to obtain a dense 3D reconstruction of the face in the image. This Analysis-by-Synthesis approach provides model parameters which contain an explicit face description and a dense model to image correspondence. However, the fit is restricted to the model space and cannot explain all variations. Our model only contains straight gaze directions and lacks high detail textural features. To overcome this limitations, we use the obtained correspondence in a discriminative approach. The dense correspondence is used to extract a pose-normalized version of the input image. The warped image contains all information from the original image and preserves gaze and detailed textural information. On the pose-normalized representation we train a regression function to obtain gaze estimation and attribute description. We provide results for pose-invariant gaze estimation on still images on the UUlm Head Pose and Gaze Database and attribute description on the Multi-PIE database. To the best of our knowledge, this is the first pose-invariant approach to estimate gaze from unconstrained still images.



This work has been partially founded by the Swiss National Science Foundation.


  1. 1.
    Amberg, B., Paysan, P., Vetter, T.: Weight, sex, and facial expressions: on the manipulation of attributes in generative 3D face models. In: Bebis, G. (ed.) ISVC 2009, Part I. LNCS, vol. 5875, pp. 875–885. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  2. 2.
    Blanz, V., Grother, P., Phillips, P.J., Vetter, T.: Face recognition based on frontal views generated from non-frontal images. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 454–461. IEEE (2005)Google Scholar
  3. 3.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: SIGGRAPH’99 Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194. ACM Press (1999)Google Scholar
  4. 4.
    Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1063–1074 (2003)CrossRefGoogle Scholar
  5. 5.
    Bradski, G.: The opencv library. Dr. Dobb’s J. Softw. Tools 25, 120–126 (2000)Google Scholar
  6. 6.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  7. 7.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  8. 8.
    Florea, L., Florea, C., Vrânceanu, R., Vertan, C.: Can your eyes tell me how you think? a gaze directed estimation of the mental activity (2013)Google Scholar
  9. 9.
    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image Vis. Comput. 28(5), 807–813 (2010)CrossRefGoogle Scholar
  10. 10.
    Hansen, D.W., Ji, Q.: In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 478–500 (2010)CrossRefGoogle Scholar
  11. 11.
    Kharevych, L., Springborn, B., Schröder, P.: Discrete conformal mappings via circle patterns. ACM Trans. Graph. (TOG) 25(2), 412–438 (2006)CrossRefGoogle Scholar
  12. 12.
    Kumar, N., Berg, A., Belhumeur, P., Nayar, S.: Describable visual attributes for face verification and image search. IEEE Trans. Pattern Anal. Mach. Intell. 33(10), 1962–1977 (2011)CrossRefGoogle Scholar
  13. 13.
    Marku, N., Frljak, M., Pandi, I.S., Ahlberg, J., Forchheimer, R.: Eye pupil localization with an ensemble of randomized trees. Pattern Recogn. 47(2), 578–587 (2014)CrossRefGoogle Scholar
  14. 14.
    Paysan, P.: Statistical modeling of facial aging based on 3D scans. Ph.D. thesis, University of Basel, Switzerland (2010)Google Scholar
  15. 15.
    Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T.: A 3D face model for pose and illumination invariant face recognition. In: Proceedings of the 6th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 296–301. IEEE (2009)Google Scholar
  16. 16.
    Prabhu, U., Heo, J., Savvides, M.: Unconstrained pose-invariant face recognition using 3D generic elastic models. IEEE Trans. Pattern Anal. Mach. Intell. 33(10), 1952–1961 (2011)CrossRefGoogle Scholar
  17. 17.
    Schönborn, S., Forster, A., Egger, B., Vetter, T.: A monte carlo strategy to integrate detection and model-based face analysis. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 101–110. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  18. 18.
    Weidenbacher, U., Layher, G., Strauss, P.M., Neumann, H.: A comprehensive head pose and gaze database (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Bernhard Egger
    • 1
    Email author
  • Sandro Schönborn
    • 1
  • Andreas Forster
    • 1
  • Thomas Vetter
    • 1
  1. 1.Department for Mathematics and Computer ScienceUniversity of BaselBaselSwitzerland

Personalised recommendations