Cross-Database Facial Expression Recognition with Domain Alignment and Compact Feature Learning

  • Lan Wang
  • Jianbo SuEmail author
  • Kejun Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11555)


Expression recognition has achieved impressive success in recent years. Most methods are based on the assumption that the training and testing databases follow the same feature distribution. However, distribution discrepancy among datasets is pretty common in practical scenarios. Thus, the performance of these methods may drop sharply on target datasets. To address this issue, we aim to learn a facial expression classification model from several labeled source databases and generalize it to target databases. This is achieved by integrating domain alignment and class-compact features learning across source domains. Domain alignment paves the way to involve more expression-related representations. Learning compact features can signicantly diminish the intra-class divergence, which is beneficial to both domain alignment and expression recognition. Experimental results demonstrate that the proposed model has a more promising performance compared with other cross-database expression recognition methods.


Facial expression recognition Domain alignment Class-compact feature learning 



This work was partially financially supported by National Natural Science Foundation of China under grant 61533012 and 91748120.


  1. 1.
    Ding, H., Zhou, S.K., Chellappa, R.: FaceNet2ExpNet: regularizing a deep face recognition net for expression recognition. In: 12th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 118–126 (2017)Google Scholar
  2. 2.
    Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(59), 1–35 (2016)Google Scholar
  3. 3.
    Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1735–1742 (2006)Google Scholar
  4. 4.
    Hasani, B., Mahoor, M.H.: Spatio-temporal facial expression recognition using convolutional neural networks and conditional random fields. In: 12th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 790–795 (2017)Google Scholar
  5. 5.
    Jian, M., Lam, K.M.: Face-image retrieval based on singular values and potential-field representation. Signal process. 100, 9–15 (2014)Google Scholar
  6. 6.
    Jian, M., Lam, K.M.: Simultaneous hallucination and recognition of low-resolution faces based on singular value decomposition. IEEE Trans. Circ. Syst. Video Technol. 25(11), 1761–1772 (2015)Google Scholar
  7. 7.
    Jian, M., Lam, K.M., Dong, J.: A novel face-hallucination scheme based on singular value decomposition. Pattern Recognit. 46(11), 3091–3102 (2013)Google Scholar
  8. 8.
    Jian, M., Lam, K.M., Dong, J.: Facial-feature detection and localization based on a hierarchical scheme. Inform. Sci. 262, 1–14 (2014)Google Scholar
  9. 9.
    Liu, N., et al.: Super wide regression network for unsupervised cross-database facial expression recognition. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1897–1901 (2018)Google Scholar
  10. 10.
    Liu, X., Kumar, B.V., You, J., Jia, P.: Adaptive deep metric learning for identity-aware facial expression recognition. In: CVPR Workshops, pp. 522–531 (2017)Google Scholar
  11. 11.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 94–101 (2010)Google Scholar
  12. 12.
    Lundqvist, D., Flykt, A., Öhman, A.: The Karolinska Directed Emotional Faces (KDEF). CD ROM from Department of Clinical Neuroscience, Psychology section, Karolinska Institutet 91, p. 630 (1998)Google Scholar
  13. 13.
    Lyons, M., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding facial expressions with Gabor wavelets. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, pp. 200–205 (1998)Google Scholar
  14. 14.
    Meng, Z., Liu, P., Cai, J., Han, S., Tong, Y.: Identity-aware convolutional neural network for facial expression recognition. In: 12th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 558–565 (2017)Google Scholar
  15. 15.
    Pantic, M., Valstar, M., Rademaker, R., Maat, L.: Web-based database for facial expression analysis. In: 2005 IEEE International Conference on Multimedia and Expo, pp. 317–321 (2005)Google Scholar
  16. 16.
    Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)Google Scholar
  17. 17.
    da Silva, F.A.M., Pedrini, H.: Effects of cultural characteristics on building an emotion classifier through facial expression analysis. J. Electron. Imaging 24(2), 1–9 (2015)Google Scholar
  18. 18.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  19. 19.
    Yan, K., Zheng, W., Cui, Z., Zong, Y.: Cross-database facial expression recognition via unsupervised domain adaptive dictionary learning. In: International Conference on Neural Information Processing, pp. 427–434 (2016)Google Scholar
  20. 20.
    Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. arXiv preprint arXiv:1411.7923 (2014)
  21. 21.
    Zavarez, M.V., Berriel, R.F., Oliveira-Santos, T.: Cross-database facial expression recognition based on fine-tuned deep convolutional network. In: 30th SIBGRAPI Conference on Graphics, Patterns and Images, pp. 405–412 (2017)Google Scholar
  22. 22.
    Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 23(10), 1499–1503 (2016)Google Scholar
  23. 23.
    Zhang, X., Mahoor, M.H., Mavadati, S.M.: Facial expression recognition using \(l_{p}\)-norm MKL multiclass-SVM. Mach. Vis. Appl. 26(4), 467–483 (2015)Google Scholar
  24. 24.
    Zhao, G., Huang, X., Taini, M., Li, S.Z., PietikäInen, M.: Facial expression recognition from near-infrared videos. Image Vis. Comput. 29(9), 607–619 (2011)Google Scholar
  25. 25.
    Zheng, W., Zong, Y., Zhou, X., Xin, M.: Cross-domain color facial expression recognition using transductive transfer subspace learning. IEEE Trans. Affect. Comput. 9(1), 21–37 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Shanghai Jiao Tong UniversityShanghaiChina
  2. 2.Shanghai Lingzhi Hi-Tech CorporationShanghaiChina

Personalised recommendations