Abstract
This paper presents a novel method for facial expression recognition based on semi random patches sampling. Different from most of the facial expression methods that use spatio expression descriptor, temporal expression descriptor or both, we extract spatio-temporal expression information by a technology of semi random patches sampling. In the facial feature extraction, expression salient features are first determined; face images are second normalized and warped to a standard face; then temporal expression information is obtained by the differences between the warped expression and its corresponding reference expression model; thirdly, a small set of patches is extracted from both spatio expression information and temporal expression information around each expression salient features. The semi random patches are used to perform facial expression recognition. The proposed semi random patches extraction is simple, yet by leveraging the sparse nature of expression images. Experimental results demonstrate that our approach is able to capture subtle texture variations caused by expression, even under occlusion and corruption.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans. PAMI 31(1), 39–58 (2009)
Ekman, P., Friesen, W.V.: Pictures of facial affect. In: Human Interaction Laboratory. University of California Medical Center, San Francisco (1976)
Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recogn. 36(1), 259–275 (2003)
Tong, Y., Liao, W., Ji, Q.: Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Trans. Pattern Anal. Mach. Intell. 29(10), 1683–1699 (2007)
Kim, D.H., Jung, S.U., et al.: Extension of cascaded simple feature based face detection to facial expression recognition. Pattern Recogn. Lett. 29(11), 1621–1631 (2008)
Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary pattern with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 2(6), 915–928 (2007)
Jiang, B., Valstar, M., Martinez, B., Pantic, M.: A dynamic appearance descriptor approach to facial actions temporal modeling. IEEE Trans. Cybern. 44(2), 161–174 (2014)
Feng, X., Hadid, A., Pietikainen, M.: Facial expression recognition with local binary patterns and linear programming. Pattern Recogn. Image Anal. 15(2), 546–554 (2005)
Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27, 803–816 (2009)
Yan, T., Jixu, C., Qiang, J.: A unified probabilistic framework for spontaneous facial action modeling and understanding. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 258–273 (2010)
Aleksic, P.S., Katsaggelos, A.K.: Automatic facial expression recognition using facial animation parameters and multistream HMMs. IEEE Trans. Inf. Forensics Secur. 1(1), 3–11 (2006)
Hamdi, D., Roberto, V., Ali, S.A., Theo, G.: Eyes do not lie: spontaneous versus posed smiles. In: Proceedings of the international conference on Multimedia, pp. 703–706 (2010)
Wang, T.-H., Lien, J.-J.J.: Facial expression recognition system based on rigid and non-rigid motion separation and 3D pose estimation. J. Pattern Recogn. 42(5), 962–977 (2009)
Peng, L., Prince, S.J.D.: Joint and implicit registration for face recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1510–1517 (2009)
Yeasin, M., Bullot, B., Sharma, R.: Recognition of facial expressions and measurement of levels of interest from video. IEEE Trans. Multimedia 8(3), 500–508 (2006)
Cheon, Y., Kim, D.: Natural facial expression recognition using differential-AAM and manifold learning. Pattern Recogn. 42, 1340–1350 (2009)
Tsalakanidou, F., Malassiotis, S.: Real-time 2D+3D facial action and expression recognition. Pattern Recogn. 43, 1763–1775 (2010)
Cohen, I., Sebe, N., Garg, A., Chen, L.S., Huang, T.S.: Facial expression recognition from video sequences: temporal and static modeling. Comput. Vis. Image Underst. 91, 160–187 (2003)
Zhao, G., Pietikainen, M.: Boosted multi-resolution spatio-temporal descriptors for facial expression recognition. Pattern Recogn. Lett. 30, 1117–1127 (2009)
Qiao, L., Chen, S., Tan, X.: Sparsity preserving projections with applications to face recognition. Pattern Recogn. 43(1), 331–341 (2010)
Lai, Z., Jin, Z., Yang, J.: Global sparse representation projections for feature extraction and classification. In: Proceedings of Chinese Conference on Pattern Recognition, pp. 1–5, November 2009
Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27, 803–816 (2009)
Aharon, M., Elad, M.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Sig. Process. 54(11), 4311–4323 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Xia, H. (2015). Semi Random Patches Sampling Based on Spatio-temporal Information for Facial Expression Recognition. In: He, X., et al. Intelligence Science and Big Data Engineering. Image and Video Data Engineering. IScIDE 2015. Lecture Notes in Computer Science(), vol 9242. Springer, Cham. https://doi.org/10.1007/978-3-319-23989-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-23989-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23987-3
Online ISBN: 978-3-319-23989-7
eBook Packages: Computer ScienceComputer Science (R0)