Abstract
Automated analysis of facial expressions is a well-investigated research area in the field of computer vision, with impending applications such as human-computer interaction (HCI). The conducted work proposes new methods for the automated evaluation of facial expression in image sequences of color and depth data. In particular, we present the main components of our system, i.e. accurate estimation of the observed person’s head pose, followed by facial feature extraction and, third, by classification. Through the application of dimensional affect models, we overcome the use of strict categories, i.e. basic emotions, which are focused on by most state-of-the-art facial expression recognition techniques. This is of importance as in most HCI applications classical basic emotions are only occurring sparsely, and hence are often inadequate to guide the dialog with the user. To resolve this issue we suggest the mapping to the so-called “Circumplex model of affect”, which enables us to determine the current affective state of the user, which can then be used in the interaction. Especially, the output of the proposed machine vision-based recognition method gives insight to the observed person’s arousal and valence states. In this chapter, we give comprehensive information on the approach and experimental evaluation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We use here the dot notation to denote temporal derivatives of a function with time as the independent variable.
References
Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975). http://doi.acm.org/10.1145/361002.361007
Besl, P.J., McKay, N.D.: A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992). doi:10.1109/34.121791. http://doi.ieeecomputersociety.org/10.1109/34.121791
Bradski, G.: The OpenCV library. Dr. Dobb’s J. Softw. Tools 25(11), 120–126 (2000)
Calder, A.J., Lawrence, A.D., Young, A.W.: Neuropsychology of fear and loathing. Nat. Rev. Neurosci. 2(5), 352–363 (2001). doi:10.1038/35072584. http://dx.doi.org/10.1038/35072584
Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models - their training and application. Comput. Vis. Image Underst. 61, 38–59 (1995)
Ekman, P.: Strong evidence for universals in facial expressions: a reply to Russell’s mistaken critique. Psychol. Bull. 115, 268–287 (1994)
Facegen modeller. http://facegen.com/modeller.htm (June 2017)
Fanelli, G., Gall, J., Gool, L.J.V.: Real time head pose estimation with random regression forests. In: The 24th IEEE Conference on Computer Vision and Pattern Recognition, pp. 617–624 (2011)
Foley, J.D., van Dam, A., Feiner, S., Hughes, J.: Computer Graphics: Principles and Practice, 3rd edn. Addison-Wesley, Boston (2013)
GFMesstechnik GmbH Germany, T.: Facescan 3d. http://www.gfm3d.com (2015)
Haykin, S.: Neural Networks: A Comprehensive Foundation, 3rd edn. Prentice-Hall, Upper Saddle River, NJ (2008)
McGlone, C., Mikhail, E., Bethe, J.: Manual of Photogrammetry, 5th edn. ASPRS, ISBN: 1-57083-071-1 (2004)
Niese, R., Al-Hamadi, A., Panning, A., Brammen, D.G., Ebmeyer, U., Michaelis, B.: Towards pain recognition in post-operative phases using 3d-based features from video and support vector machines. Int. J. Digit. Content Technol. Appl. 3(4), 21–33 (2009)
Niese, R., Al-Hamadi, A., Heuer, M., Michaelis, B., Matuszewski, B.: Machine vision based recognition of emotions using the circumplex model of affect. In: International Conference on Multimedia Technology (ICMT), pp. 6424–6427 (2011)
Niese, R., Werner, P., Al-Hamadi, A.: Accurate, fast and robust realtime face pose estimation using kinect camera. In: IEEE SMC International Conference, pp. 487–490 (2013)
Pandzic, I.S., Forchheimer, R.: MPEG-4 Facial Animation: The Standard, Implementation and Applications, 1st edn. Wiley, New York (2002). ISBN: 0-470-84465-5
Pantic, M., Pentland, A., Nijholt, A., Huang, T.S.: Human computing and machine understanding of human behavior: a survey. In: Artificial Intelligence for Human Computing, ICMI, pp. 47–71 (2007)
Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 FPS via regressing local binary features. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, OH, 23–28 June 2014, pp. 1685–1692 (2014)
Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39, 1161–1178 (1980)
Saeed, A., Al-Hamadi, A., Niese, R., Elzobi, M.: Frame-based facial expression recognition using geometrical features. Hindawi Adv. Hum.-Comput. Interaction (2014). doi:10.1155/2014/408953
Viola, P.A., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)
Wang, J., Yin, L., Wei, X., Sun, Y.: 3d facial expression recognition based on primitive surface feature distribution. In: IEEE International Conference on Computer Vision and Pattern Recognition, CVPR06, pp. 1399–1406 (2006)
Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)
Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high-resolution 3d dynamic facial expression database. In: IEEE International Conference on Automatic Face and Gesture Recognition (FG’08), pp. 1–6 (2008)
Acknowledgements
This work was done within the Transregional Collaborative Research Centre SFB/TRR 62 “Companion-Technology for Cognitive Technical Systems” funded by the German Research Foundation (DFG).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Niese, R., Al-Hamadi, A., Neumann, H. (2017). Automated Analysis of Head Pose, Facial Expression and Affect. In: Biundo, S., Wendemuth, A. (eds) Companion Technology. Cognitive Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-43665-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-43665-4_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-43664-7
Online ISBN: 978-3-319-43665-4
eBook Packages: Computer ScienceComputer Science (R0)