Advertisement

Facial Expression Bilinear Encoding Model

  • Haifeng Zhang
  • Wen Su
  • Zengfu Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10996)

Abstract

Facial expressions are generated by contractions of facial muscles. The contractions lead to variations in the appearance of facial parts. It has been proved that the features from different facial parts can improve the accuracy of facial expression recognition. In this paper, we propose a bilinear encoding model for facial expression recognition. Our system uses the still facial expression images as inputs and employs the bilinear convolutional networks to capture the features in the appearance of facial parts. It detects crucial facial parts and extracts the appearance features simultaneously with end-to-end learning. To verify the performance of our system, we have made experiments on two popular expression databases: CK+ and Oulu-CASIA. The experimental results show that the proposed method achieves comparable or better performance compared with the state-of-the-art methods for facial expression recognition.

Keywords

Facial expression recognition Facial parts Appearance features Bilinear encoding 

Notes

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61472393).

References

  1. 1.
    Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., Metaxas, D.N.: Learning active facial patches for expression analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2562–2569 (2012)Google Scholar
  2. 2.
    Happy, S.L., Routray, A.: Automatic facial expression recognition using features of salient facial patches. IEEE Trans. Affect. Comput. 6(1), 1–12 (2015)CrossRefGoogle Scholar
  3. 3.
    Liu, M., Li, S., Shan, S., Wang, R., Chen, X.: Deeply learning deformable facial action parts model for dynamic expression analysis. In: Proceedings of the IEEE Asian Conference on Computer Vision, pp. 143–157 (2014)Google Scholar
  4. 4.
    Perveen, N., Singh, D., Mohan, C.K.: Spontaneous facial expression recognition: a part based approach. In: Proceedings of the IEEE Conference on Machine Learning and Applications, pp. 819–824 (2016)Google Scholar
  5. 5.
    Lin, T.Y., RoyChowdhury, A., Maji, S.: Bilinear convolutional neural networks for fine-grained visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1309–1322 (2017)CrossRefGoogle Scholar
  6. 6.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 94–101 (2010)Google Scholar
  7. 7.
    Zhao, G., Huang, X., Taini, M., Li, S.Z., PietikaInen, M.: Facial expression recognition from near-infrared videos. Image Vis. Comput. 29(9), 607–619 (2011)CrossRefGoogle Scholar
  8. 8.
    Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)Google Scholar
  9. 9.
    Goodfellow, I.J., et al.: Challenges in representation learning: a report on three machine learning contests. In: Proceedings of the IEEE International Conference on Neural Information Processing, pp. 117–124 (2013)Google Scholar
  10. 10.
    Klaser, A., Marszalek, M., Schmid, C.: A spatio-temporal descriptor based on 3d-gradients. In: BMVC 2008–19th British Machine Vision Conference, pp. 275–1 (2008)Google Scholar
  11. 11.
    Jain, S., Hu, C., Aggarwal, J.K.: Facial expression recognition with temporal modeling of shapes. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1642–1649 (2011)Google Scholar
  12. 12.
    Mollahosseini, A., Chan, D., Mahoor, M.H.: Going deeper in facial expression recognition using deep neural networks. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pp. 1–10 (2016)Google Scholar
  13. 13.
    Liu, M., Shan, S., Wang, R., Chen, X.: Learning expressionlets on spatio-temporal manifold for dynamic facial expression recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1749–1756 (2014)Google Scholar
  14. 14.
    Meng, Z., Liu, P., Cai, J., Han, S., Tong, Y.: Identity-aware convolutional neural network for facial expression recognition. In: Proceedings of the IEEE International Conference on Automatic Face & Gesture Recognition, pp. 558–565 (2017)Google Scholar
  15. 15.
    Jung, H., Lee, S., Yim, J., Park, S., Kim, J.: Joint fine-tuning in deep neural networks for facial expression recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2983–2991 (2015)Google Scholar
  16. 16.
    Zhao, X., et al.: Peak-piloted deep network for facial expression recognition. In: Proceedings of the IEEE European Conference on Computer Vision, pp. 425–442 (2016)Google Scholar
  17. 17.
    Zhang, K., Huang, Y., Du, Y., Wang, L.: Facial expression recognition based on deep evolutional spatial-temporal networks. IEEE Trans. Image Process. 26(9), 4193–4203 (2017)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Cai, J., Meng, Z., Khan, A.S., Li, Z., Tong, Y.: Island loss for learning discriminative features in facial expression recognition. arXiv:1710.03144

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.University of Science and Technology of ChinaHefeiChina
  2. 2.Institute of Intelligent MachinesChinese Academy of SciencesHefeiChina
  3. 3.National Engineering Laboratory for Speech and Language Information ProcessingHefeiChina

Personalised recommendations