Abstract
In this paper, we propose a novel idea for automatic facial expression analysis with the aim of resolving the existing challenges in 2D images. The subtle combination of the geometry-based method with the appearance-based features in depth and color images contributes to increasing in distinguishable features among various facial expressions. Particular functions are utilised to calculate the correlation between expressions in order to determine the exact facial expression. Our approach consists of a sequence of steps including estimating the normal vector of facial surface, then extracting the geometric features such as the orientation of normal vector in the point cloud. The useful color information is known as LBP. According to the result of the experiment, we demonstrate that the effective fusion scheme of texture and shape feature on color and depth images. In comparison with the non fusion scheme, our fusion scheme has resulted in the increase of recognition under low and high illuminated light, about 19.84 % and 1.59 %, respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Berretti, S., Amor, B.B., Daoudi, M., Del Bimbo, A.: 3D facial expression recognition using sift descriptors of automatically detected keypoints. Int. J. Vis. Comput. 27(11), 1021–1036 (2011)
Cao, C., Weng, Y., Zhou, S., Tong, Y., Zhou, K.: Facewarehouse: a 3D facial expression database for visual computing. IEEE Trans. J. Vis. Comput. Graph. 20(3), 413–425 (2014)
Tim, C.: Model-based methods in analysis of biomedical images. In: Baldock, R., Graham, J. (eds.) Image Processing and Analysis, pp. 223-247. Oxford University Press (2000)
Fehr, D.A.: Covariance based point cloud descriptors for object detection and classification. University of Minnosita, August 2013
Förstner, W., Moonen, B.: A metric for covariance matrices. In: Grafarend, E.W., Krumm, F.W., Schwarze, V.S. (eds.) Journal of Geodesy-The Challenge of the 3rd Millennium, pp. 299–309. Springer, Heidelberg (2003)
Guo, S., Ruan, Q.: Facial expression recognition using local binary covariance matrices. In: Proceedings of 4th IET International Conference on Wireless, Mobile and Multimedia Networks (ICWMMN 2011) (2011)
Li, B.Y., Mian, A.S., Liu, W., Krishna, A.: Using kinect for face recognition under varying poses, expressions, illumination and disguise. In: 2013 IEEE Workshop on Proceedings of Applications of Computer Vision (WACV), pp. 186–192. IEEE (2013)
Li, H., Chen, L., Huang, D., Wang, Y., Morvan, J.: 3D facial expression recognition via multiple kernel learning of multi-scale local normal patterns. In: 2012 21st International Conference on Proceedings of Pattern Recognition (ICPR), pp. 2577–2580. IEEE (2012)
Li, H., Morvan, J.-M., Chen, L.: 3D facial expression recognition based on histograms of surface differential quantities. In: Blanc-Talon, J., Kleihorst, R., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2011. LNCS, vol. 6915, pp. 483–494. Springer, Heidelberg (2011)
Liu, M., Shan, S., Wang, R., Chen, X.: Learning expressionlets on spatio-temporal manifold for dynamic facial expression recognition. In: IEEE Conference on Proceedings of Computer Vision and Pattern Recognition, pp. 1749–1756. IEEE (2014)
Malawski, F., Kwolek, B., Sako, S.: Using kinect for facial expression recognition under varying poses and illumination. In: Ślȩzak, D., Schaefer, G., Vuong, S.T., Kim, Y.-S. (eds.) AMT 2014. LNCS, vol. 8610, pp. 395–406. Springer, Heidelberg (2014)
Mao, Q., Pan, X., Zhan, Y., Shen, X.: Using kinect for real-time emotion recognition via facial expressions. J. Front. Inf. Tech. Electr. Eng. 16, 272–282 (2015)
Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. J. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)
Pang, Y., Yuan, Y., Li, X.: Gabor-based region covariance matrices for face recognition. J. IEEE Trans. Circuits Syst. Video Technol. 18(7), 989–993 (2008)
Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE International Conference on Proceedings of Robotics and Automation (ICRA), Shanghai, China, 9–13 May 2011, pp. 1–4. IEEE (2011)
Savran, A., Gur, R., Verma, R.: Automatic detection of emotion valence on faces using consumer depth cameras. In: 2013 IEEE International Conference on Proceedings of Computer Vision Workshops (ICCVW), pp. 75–82. IEEE (2013)
Shbib, R., Zhou, S.: Facial expression analysis using active shape model. J. Signal Process. Image Process. Pattern Recogn. 8(1), 9–22 (2015)
Acknowledgments
This research is funded by Vietnam National University Ho Chi Minh City (VNU-HCM) under grant number B2014-18-02.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Truong, T., Ly, N. (2016). Building the Facial Expressions Recognition System Based on RGB-D Images in High Performance. In: Nguyen, N.T., Trawiński, B., Fujita, H., Hong, TP. (eds) Intelligent Information and Database Systems. ACIIDS 2016. Lecture Notes in Computer Science(), vol 9622. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-49390-8_37
Download citation
DOI: https://doi.org/10.1007/978-3-662-49390-8_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-49389-2
Online ISBN: 978-3-662-49390-8
eBook Packages: Computer ScienceComputer Science (R0)