Abstract
Facial expressions are generated by contractions of facial muscles. The contractions lead to variations in the appearance of facial parts. It has been proved that the features from different facial parts can improve the accuracy of facial expression recognition. In this paper, we propose a bilinear encoding model for facial expression recognition. Our system uses the still facial expression images as inputs and employs the bilinear convolutional networks to capture the features in the appearance of facial parts. It detects crucial facial parts and extracts the appearance features simultaneously with end-to-end learning. To verify the performance of our system, we have made experiments on two popular expression databases: CK+ and Oulu-CASIA. The experimental results show that the proposed method achieves comparable or better performance compared with the state-of-the-art methods for facial expression recognition.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., Metaxas, D.N.: Learning active facial patches for expression analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2562–2569 (2012)
Happy, S.L., Routray, A.: Automatic facial expression recognition using features of salient facial patches. IEEE Trans. Affect. Comput. 6(1), 1–12 (2015)
Liu, M., Li, S., Shan, S., Wang, R., Chen, X.: Deeply learning deformable facial action parts model for dynamic expression analysis. In: Proceedings of the IEEE Asian Conference on Computer Vision, pp. 143–157 (2014)
Perveen, N., Singh, D., Mohan, C.K.: Spontaneous facial expression recognition: a part based approach. In: Proceedings of the IEEE Conference on Machine Learning and Applications, pp. 819–824 (2016)
Lin, T.Y., RoyChowdhury, A., Maji, S.: Bilinear convolutional neural networks for fine-grained visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1309–1322 (2017)
Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 94–101 (2010)
Zhao, G., Huang, X., Taini, M., Li, S.Z., PietikaInen, M.: Facial expression recognition from near-infrared videos. Image Vis. Comput. 29(9), 607–619 (2011)
Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)
Goodfellow, I.J., et al.: Challenges in representation learning: a report on three machine learning contests. In: Proceedings of the IEEE International Conference on Neural Information Processing, pp. 117–124 (2013)
Klaser, A., Marszalek, M., Schmid, C.: A spatio-temporal descriptor based on 3d-gradients. In: BMVC 2008–19th British Machine Vision Conference, pp. 275–1 (2008)
Jain, S., Hu, C., Aggarwal, J.K.: Facial expression recognition with temporal modeling of shapes. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1642–1649 (2011)
Mollahosseini, A., Chan, D., Mahoor, M.H.: Going deeper in facial expression recognition using deep neural networks. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pp. 1–10 (2016)
Liu, M., Shan, S., Wang, R., Chen, X.: Learning expressionlets on spatio-temporal manifold for dynamic facial expression recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1749–1756 (2014)
Meng, Z., Liu, P., Cai, J., Han, S., Tong, Y.: Identity-aware convolutional neural network for facial expression recognition. In: Proceedings of the IEEE International Conference on Automatic Face & Gesture Recognition, pp. 558–565 (2017)
Jung, H., Lee, S., Yim, J., Park, S., Kim, J.: Joint fine-tuning in deep neural networks for facial expression recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2983–2991 (2015)
Zhao, X., et al.: Peak-piloted deep network for facial expression recognition. In: Proceedings of the IEEE European Conference on Computer Vision, pp. 425–442 (2016)
Zhang, K., Huang, Y., Du, Y., Wang, L.: Facial expression recognition based on deep evolutional spatial-temporal networks. IEEE Trans. Image Process. 26(9), 4193–4203 (2017)
Cai, J., Meng, Z., Khan, A.S., Li, Z., Tong, Y.: Island loss for learning discriminative features in facial expression recognition. arXiv:1710.03144
Acknowledgements
This work was supported by the National Natural Science Foundation of China (No. 61472393).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, H., Su, W., Wang, Z. (2018). Facial Expression Bilinear Encoding Model. In: Zhou, J., et al. Biometric Recognition. CCBR 2018. Lecture Notes in Computer Science(), vol 10996. Springer, Cham. https://doi.org/10.1007/978-3-319-97909-0_33
Download citation
DOI: https://doi.org/10.1007/978-3-319-97909-0_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97908-3
Online ISBN: 978-3-319-97909-0
eBook Packages: Computer ScienceComputer Science (R0)