Abstract
In this chapter we focus on the eye landmarking and eye components identification in the framework of emerging psychology-related eye tracking applications. Traditional eye landmarking separates the identification of eye centers and of eye corners and margins, while here we discuss their joint use for face expression analysis in unconstrained environments and precise estimation of non-visual gaze directions, as suggested by the Eye Accessing Cues (EAC) of the Neuro-Linguistic Programming (NLP). Such a system involves a combination of low-level feature extraction, heuristic pre-processing and trained classifiers. The approach is extensively tested across several classical image databases and compared with state of the art traditional methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
vision.ucsd.edu/~leekc/ExtYaleDatabase/ExtYaleB.html.
- 4.
The database is available at http://vis-www.cs.umass.edu/lfw/.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
References
R. Bandler, J. Grinder, Frogs into Princes: Neuro Linguistic Programming (Real People Press, Moab, 1979)
P. Tsiamyrtzis, J. Dowdall, D. Shastri, I.T. Pavlidis, M.G. Frank, P. Ekman, Imaging facial physiology for the detection of deceit. Int. J. Comput. Vis. 71, 197–214 (2007)
A.B. Ashraf, S. Lucey, J.F. Cohn, T. Chen, Z. Ambadar, K.M. Prkachin, P. Solomon, The painful face – pain expression recognition using active appearance models. Image Vis. Comput. 27, 1788–1796 (2009)
C. Florea, L. Florea, C. Vertan, Learning pain from emotion: transferred hot data representation for pain intensity estimation, in Proceedings of European Conference on Computer Vision Workshop on ACVR (2014)
D.S. Messinger, M.H. Mahoor, S.M. Chow, J. Cohn, Automated measurement of facial expression in infant-mother interaction: a pilot study. Infancy 14(3), 285–305 (2009)
D. McDuff, R.E. Kaliouby, R. Picard, Predicting online media effectiveness based on smile responses gathered over the internet, in IEEE Face and Gesture (2013), pp. 1–8
J. Rehg, G. Abowd, A. Rozga et al., Decoding children’s social behavior, in Proceedings of Computer Vision and Pattern Recognition (2013), pp. 3414–3421
J.F. Cohn, F. De la Torre, Automated face analysis for affective computing, in The Oxford Handbook of Affective Computing (Oxford University Press, Oxford, 2014)
A. Frischen, A.P. Bayliss, S.P. Tipper, Gaze cueing of attention. Psychol. Bull. 133, 694–724 (2007)
R. Vranceanu, C. Florea, L. Florea, C. Vertan, Gaze direction estimation by component separation for recognition of eye accessing cues. Mach. Vis. Appl. 26(2–3), 267–278 (2015)
W. James, The Principles of Psychology (Harvard University Press, Cambridge, 1890)
L. Nummenmaa, A. Calder, Neural mechanisms of social attention. Trends Cogn. Sci. 13, 135–43 (2009)
S. Liversedge, J. Findlay, Saccadic eye movements and cognition. Trends Cogn. Sci. 4(1), 6–14 (2000)
R. Adams, R.E. Kleck, Effects of direct and averted gaze on the perception of facially communicated emotion. Emotion 5, 3–11 (2005)
H. Joseph, K. Nation, S.P. Liversedge, Using eye movements to investigate word frequency effects in children’s sentence reading. Sch. Psychol. Rev. 42, 207–222 (2013)
A. Godfroid, F. Boers, A. Housen, An eye for words: gauging the role of attention in incidental l2 vocabulary acquisition by means of eye-tracking. Stud. Second Lang. Acquis. 35, 483–517 (2013)
K. Rayner, T.J. Slattery, D. Drieghe, S.P. Liversedge, Eye movements and word skipping during reading: effects of word length and predictability. J. Exp. Psychol. Hum. Percept. Perform. 37, 514–528 (2011)
K. Rayner, B.R. Foorman, C.A. Perfetti, D. Pesetsky, M.S. Seidenberg, How psychological science informs the teaching of reading. Psychol. Sci. Public Interest 2, 31–74 (2001)
M.M. Chun, Contextual cueing of visual attention. Trends Cogn. Sci. 4, 170–178 (2000)
A. Bulling, T. Zander, Cognition-aware computing. IEEE Trans. Pervasive Comput. 13, 80–83 (2014)
B. Meijering, H. van Rijn, N.A. Taatgen, R. Verbrugge, What eye movements can tell about theory of mind in a strategic game. PLoS One 7(9) (2012) doi:10.1371/journal.pone.0045961
K. Krejtz, C. Biele, D. Chrzastowski, A. Kopacz, A. Niedzielska, P. Toczyski, A. Duchowski, Gaze-controlled gaming: immersive and difficult but not cognitively overloading, in Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (2014), pp. 1123–1129
J. Sturt, S. Ali, W. Robertson, D. Metcalfe, A. Grove, C. Bourne, C. Bridle, Neurolinguistic programming: systematic review of the effects on health outcomes. Br. J. Gen. Pract. 62, 757–764 (2012)
R. Vranceanu, C. Florea, L. Florea, C. Vertan, NLP EAC recognition by component separation in the eye region, in Proceedings of Computer Analysis and Image Processing (2013), pp. 225–232
B. Laeng, D.S. Teodorescu, Eye scanpaths during visual imagery reenact those of perception of the same visual scene. Cogn. Sci. 26, 207–231 (2002)
T. Kanade, J.F. Cohn, Y. Tian, Comprehensive database for facial expression analysis, in IEEE Face and Gesture (2000), pp. 46–53
K. Lee, J. Ho, D. Kriegman, Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intell. 27, 684–698 (2005)
P. Belhumeur, D. Jacobs, D. Kriegman, N. Kumar, Localizing parts of faces using a consensus of exemplars, in Proceedings of Computer Vision and Pattern Recognition (2011), pp. 545–552
G. Huang, M. Ramesh, T. Berg, E. Learned-Miller, Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report, University of Massachusetts, 2007
L. Florea, C. Florea, R. Vranceanu, C. Vertan, Can your eyes tell me how you think? A gaze directed estimation of the mental activity, in Proceedings of British Machine Vision Conference (2013)
S. Asteriadis, D. Soufleros, K. Karpouzis, S. Kollias, A natural head pose and eye gaze dataset, in ACM Workshop on Affective Interaction in Natural Environments (2009), pp. 1–4
U. Weidenbacher, G. Layher, P. Strauss, H. Neumann, A comprehensive head pose and gaze database, in IET International Conference on Intelligent Environments (2007), pp. 455–458
A. Kasinśki, A. Florek, A. Schmidt, The PUT face database. Image Process. Commun. 13, 59–64 (2008)
L. Wolf, Z. Freund, S. Avidan, An eye for an eye: a single camera gaze-replacement method, in Proceedings of Computer Vision and Pattern Recognition (2010), pp. 817–824
K. Radlak, M. Kawulok, B. Smolka, N. Radlak, Gaze direction estimation from static images, in Proceedings of IEEE Multimedia Signal Processing (2014), pp. 1–4
P. Viola, M. Jones, Robust real-time face detection. Int. J. Comput. Vis. 57, 137–154 (2004)
M. Mathias, R. Benenson, M. Pedersoli, L.V. Gool, Face detection without bells and whistles, in Proceedings of the European Conference on Computer Vision, vol. 8692 (2014), pp. 720–735
P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan, Object detection with discriminatively trained part-based models. Pattern Recogn. Lett. 19, 899–906 (2010)
F. Song, X. Tan, S. Chen, Z. Zhoub, A literature survey on robust and efficient eye localization in real-life scenarios. Br. J. Gen. Pract. 46, 3157–3173 (2013)
M. Hamouz, J. Kittlerand, J.K. Kamarainen, P. Paalanen, H. Kalviainen, J. Matas, Feature-based affine-invariant localization of faces. IEEE Trans. Pattern Anal. Mach. Intell. 27, 643–660 (2005)
S. Asteriadis, N. Nikolaidis, I. Pitas, Facial feature detection using distance vector fields. Pattern Recogn. 42, 1388–1398 (2009)
J. Wu, Z.H. Zhou, Efficient face candidates selector for face detection. Pattern Recogn. 36, 1175–1186 (2003)
R. Valenti, T. Gevers, Accurate eye center location and tracking using isophote curvature, in Proceedings of Computer Vision and Pattern Recognition (2008), pp. 1–8
O. Jesorsky, K. Kirchberg, R. Frischholz, Robust face detection using the Hausdorff distance, in Proceedings of International Conference on Audio- and Video-Based Biometric Person Authentication (2001), pp. 90–95
T. Kanade, Picture processing by computer complex and recognition of human faces. Technical Report, Kyoto University, Department of Information Science, 1973
G.C. Feng, P.C. Yuen, Variance projection function and its application to eye detection for human face recognition. Pattern Recogn. Lett. 19, 899–906 (1998)
Z. Zhou, Projection functions for eye detection. Pattern Recogn. 37, 1049–1056 (2004)
M. Turkan, M. Pardas, A.E. Cetin, Edge projections for eye localization. Opt. Eng. 47, 047–054 (2008)
M. Verjak, M. Stephancic, An anthropological model for automatic recognition of the male human face. Ann. Hum. Biol. 21, 363–380 (1994)
D. Cristinacce, T. Cootes, I. Scott, A multi-stage approach to facial feature detection, in Proceedings of British Machine Vision Conference (2004), pp. 277–286
P. Campadelli, R. Lanzarotti, G. Lipori, Precise eye localization through a general-to-specific model definition, in Proceedings of British Machine Vision Conference, I, 187–196 (2006)
Z. Niu, S. Shan, S. Yan, X. Chen, W. Gao, 2D cascaded adaboost for eye localization, in Proceedings of International Conference of Pattern Recognition (2006), pp. 1216–1219
S. Kim, S.T. Chung, S. Jung, D. Oh, J. Kim, S. Cho, World Academy of Science, Engineering and Technology, in WASET, vol. 21 (World Academy of Science, Engineering and Technology, 2007), pp. 483–487
M. Asadifard, J. Shanbezadeh, Automatic adaptive center pupil detection using face detection and CDF analysis, in Proceedings of International Multimedia Conference of Engineers and Computer Scientist (2010), pp. 130–133
L. Ding, A.M. Martinez, Features versus context: an approach for precise and detailed detection and delineation of faces and facial features. IEEE Trans. Pattern Anal. Mach. Intell. 32, 2022–2038 (2010)
F. Timm, E. Barth, Accurate eye centre localisation by means of gradients, in Proceedings of International Conference on Computer Theory and Applications (2011), pp. 125–130
M. Kawulok, J. Szymanek, Precise multi-level face detector for advanced analysis of facial images. IET Image Process. 6, 95–103 (2012)
C. Florea, L. Florea, C. Vertan, Robust eye centers localization with zero-crossing encoded image projections. Pattern Anal. Applic. 1–17 (2015), DOI:10.1007/s10044-015-0479-x, http://dx.doi.org/10.1007/s10044-015-0479-x
R. Valenti, T. Gevers, Accurate eye center location through invariant isocentric patterns. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1785–1798 (2012)
H.C. Becker, W.J. Nettleton, P.H. Meyers, J.W. Sweeney, C.M. Nice, Digital computer determination of a medical diagnostic index directly from chest X-ray images. IEEE Trans. Biomed. Eng. 11, 62–72 (1964)
F. Crow, Summed-area tables for texture mapping. Proc. SIGGRAPH 18, 207–212 (1984)
G.E. Blelloch, Prefix sums and their applications. synthesis of parallel algorithms. Technical report, University of Massachusetts, 1990
R.A. King, T.C. Phipps, Shannon, TESPAR and approximation strategies. Comput. Secur. 18, 445–453 (1999)
X. Chen, H. Wu, X. Jin, Q. Zhao, Face illumination manipulation using a single reference image by adaptive layer decomposition. IEEE Trans. Image Processing 22(11), 4249–4259 (2013)
B. Kroon, A. Hanjalic, S.M. Maas, Eye localization for face matching: is it always useful and under what conditions, in Proceedings of International Conference on Content-Based Image and Video Retrieval (2008), pp. 379–387
M. Ciesla, P. Koziol, Eye pupil location using webcam. CoRR, (2012) http://arxiv.org/abs/1202.6517
M. Dantone, J. Gall, G. Fanelli, L.V. Gool, Real-time facial feature detection using conditional regression forests, in Proceedings of Computer Vision and Pattern Recognition (2012), pp. 2578–2585
Y. Sun, X. Wang, X. Tang, Deep convolutional network cascade for facial point detection, in Proceedings of Computer Vision and Pattern Recognition (2013), pp. 3476–3483
T. Cootes, C. Taylor, D. Cooper, J. Graham, Active shape models - their training and application. Comput. Vis. Image Underst. 61, 38–59 (1995)
T.F. Cootes, G.J. Edwards, C.J. Taylor, Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 681–685 (2001)
T. Leung, M. Burl, P. Perona, Finding faces in cluttered scenes using random labeled graph matching, in Proceedings of International Conference on Computer Vision (1995), pp. 637–644
S. Milborrow, F. Nicolls, Locating facial features with an extended active shape model, in Proceedings of European Conference on Computer Vision (2008), pp. 504–513
V. Le, J. Brandt, Z. Lin, L. Bourdev, T.S. Huang, Interactive facial feature localization, in Proceedings of European Conference on Computer Vision (2012), pp. 679–692
D. Cristinacce, T. Cootes, Feature detection and tracking with constrained local models, in Proceedings of British Machine Vision Conference (2006), pp. 929–938
P. Tresadern, H. Bhaskar, S. Adeshina, C. Taylor, T. Cootes, Combining local and global shape models for deformable object matching, in Proceedings of British Machine Vision Conference (2009)
T. Cootes, M.C. Ionita, C. Lindner, P. Sauer, Robust and accurate shape model fitting using random forest regression voting, in Proceedings of European Conference on Computer Vision (2012)
J. Saragih, S. Lucey, J. Cohn, Deformable model fitting by regularized landmark mean-shift. Int. J. Comput. Vis. 91, 200–215 (2011)
M. Valstar, T. Martinez, X. Binefa, M. Pantic, Facial point detection using boosted regression and graph models, in Proceedings of Computer Vision and Pattern Recognition (2010), pp. 2729–2736
X. Zhu, D. Ramanan, Face detection, pose estimation, and landmark localization in the wild, in Proceedings of Computer Vision and Pattern Recognition (2012), pp. 2879–2886
X. Yu, J. Huang, S. Zhang, W. Yan, D.N. Metaxas, Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model, in Proceedings of International Conference on Computer Vision (2013), pp. 1944–1951
B. Martinez, M.F. Valstar, X. Binefa, M. Pantic, Local evidence aggregation for regression based facial point detection. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1149–1163 (2013)
P. Wang, M.B. Green, Q. Ji, J. Wayman, Automatic eye detection and its validation, in IEEE Workshop on FRGC, Computer Vision and Pattern Recognition (2005), p. 164
A. Duchowski, Eye Tracking Methodology: Theory and Practice (Springer, Berlin, 2007)
D. Hansen, J. Qiang, In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32, 478–500 (2010)
D. Yoo, M. Chung, A novel non-intrusive eye gaze estimation using cross-ratio under large head motion. Comput. Vis. Image Underst. 98, 25–51 (2005)
B. Pires, M. Hwangbo, M. Devyver, T. Kanade, Visible-spectrum gaze tracking for sports, in WACV (2013)
D. Hansen, A. Pece, Eye tracking in the wild. Comput. Vis. Image Underst. 98, 182–210 (2005)
S. Cadavid, M. Mahoor, D. Messinger, J. Cohn, Automated classification of gaze direction using spectral regression and support vector machine, in Proceedings of Affective Computing and Intelligent Interaction (2009), pp. 1–6
T. Heyman, V. Spruyt, A. Ledda, 3d face tracking and gaze estimation using a monocular camera, in Proceedings of International Conference on Positioning and Context-Awareness (2011), pp. 23–28
M. Everingham, A. Zisserman, Regression and classification approaches to eye localization in face images, in IEEE Face and Gesture (2006), pp. 441–446
G. Diamantopoulos, Novel eye feature extraction and tracking for non-visual eye-movement applications. Ph.D. thesis, University of Birmingham, 2010
S. le Cessie, J. van Houwelingen, Ridge estimators in logistic regression. Appl. Stat. 41, 191–201 (1992)
Acknowledgements
This work was partially supported by the Romanian Sectoral Operational Programme Human Resources Development 2007–2013 through the European Social Fund Financial Agreements POSDRU/159/1.5/S/134398 (Knowledge).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Florea, L., Florea, C., Vertan, C. (2016). Extended Eye Landmarks Detection for Emerging Applications. In: Kawulok, M., Celebi, M., Smolka, B. (eds) Advances in Face Detection and Facial Image Analysis. Springer, Cham. https://doi.org/10.1007/978-3-319-25958-1_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-25958-1_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25956-7
Online ISBN: 978-3-319-25958-1
eBook Packages: EngineeringEngineering (R0)