Abstract
Facial expression recognition has become a very active research in computer vision, behavior interpretation of emotions, human computer interaction, cognitive science and intelligent control. Traditional facial expression analysis methods mainly focuses on the facial muscle movement and basic expression features of face image. In this paper, we propose a novel method for facial expression recognition based on ensemble extreme learning machine with eye movements information. Here, the eye movements information is regarded as explicit clue to improve the performance of facial expression recognition. Firstly, we extract eye movements features from eye movements information which recorded by Tobii eye tracker. The histogram of orientation gradient (HOG) features are simultaneously obtained from the face images by dividing it into a number of small cells. Secondly, we combine the eye movements features together with the HOG features of face images by using a tensor kernel. Finally, the fusion features are trained by ensemble extreme learning machine and a bagging algorithm is explored for producing the results. Extensive experiment on the two widely available datasets of facial expressions demonstrate that our proposal effectively improves the accuracy and efficiency of face expression recognition and achieve performance at extremely high speed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tian, Y., Brown, L., Hampapur, A., Pankanti, S., Senior, A., Bolle, R.: Real world real-time automatic recognition of facial expression. IEEE PETS, Australia (2003)
Whitehill, J., Bartlett, M.S., Movellan, J.: Automatic facial expression recognition. In: Social Emotions in Nature and Artifact. London: Oxford University Press (2013)
Hardoon, D., Kaski, K.: Information retrieval by inferring implicit queries from eye movements. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, pp. 29–46 (2007)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Nuerocomputing 70(1–3): 489–501 (2006)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Etreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of International Joint Conference on Neural Networks, vol. 2, pp. 985–990 (2004)
Schurgin, M.W., Nelson, J., Lida, S., Franconeri, S.L.: Eye movements during emotion recognition in faces. J. Vis. 14(13), 1–16 (2014)
Chen, J.J., Yan, H.X., Xiang, J.: Study of decoding mental state based on eye tracks using SVM. Comput. Eng. Appl. 47(11), 39–42 (2011)
Pulmannova, S.: Tensor products of hilbert space effect algebras. Rep. Math. Phys. 53(2), 301–316 (2004)
Huang, G.B., Chen, L.: Convex incremental extreme learning machine. Neurocomputing 70(1), 3056–3062 (2007)
Huang, G.B., Zhu, Q.Y., Siew, K.Z.: Can threshold networks be trained directly? IEEE Trans. Circuits Syst. II 53(3), 187–191 (2006)
Huang, G.B., Ding, X.J., Zhou, H.: Optimization method based extreme learning machine for classification. Neurocomputing (2010)
Serre, D.: Matrieces: Theory and Applications. Springer, New York (2002)
Lyons, M., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding facial expressions with Gabor wavelets. In: Proceedings of the IEEE International Conference on Face and Gesture Recognition, pp. 200–205 (1998)
Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Matthews, I.: The extended cohn-kanade dataset (CK+): a complete dataset for action unit and emotion-specific expressions. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 94–101 (2010)
Acknowledgments
This work was supported by the National Natural Science Foundation of China under Grant No. 61370146; National Natural Science Foundation of China under Grant No. 2013405003; Fundamental Research Funds for the Central Universities No. DC201501030401, DC201502030203.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Lu, B., Duan, X., Yuan, Y. (2016). Facial Expression Recognition Based on Ensemble Extreme Learning Machine with Eye Movements Information. In: Cao, J., Mao, K., Wu, J., Lendasse, A. (eds) Proceedings of ELM-2015 Volume 2. Proceedings in Adaptation, Learning and Optimization, vol 7. Springer, Cham. https://doi.org/10.1007/978-3-319-28373-9_25
Download citation
DOI: https://doi.org/10.1007/978-3-319-28373-9_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-28372-2
Online ISBN: 978-3-319-28373-9
eBook Packages: EngineeringEngineering (R0)