Brief Technical Analysis of Facial Expression Recognition

  • Lei Xu
  • Aolei YangEmail author
  • Minrui Fei
  • Wenju Zhou
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 924)


Facial expression recognition (FER) is the current hot research topic, and it is widely used in the fields of pattern recognition, computer vision and artificial intelligence. As it is an important part of intelligent human-computer interaction technology, the FER has received widespread attention in recent years, and researchers in different fields have proposed many approaches for it. This paper reviews recent developments on FER approaches and the key technologies involved in the FER system: face detection and preprocessing, facial expression feature extraction and facial expression classification, which are analyzed and summarized in detail. Finally, the state-of-the-art of the FER is summarized, and its future development direction is pointed out.


Computer vision Artificial intelligence Facial expression Feature extraction Classification 



This work is supported by Natural Science Foundation of Shanghai (No. 18ZR1415100), National Science Foundation of China (61473182, 61773253), Science and Technology Commission of Shanghai Municipality (15JC1401900) and Key research and development project of Yantai (2017ZH061).


  1. 1.
    Mehrabian, A.: Communication without words. Psychol. Today 2, 53–55 (1968)Google Scholar
  2. 2.
    Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17, 124–129 (1971)CrossRefGoogle Scholar
  3. 3.
    Ekman, P.: Facial Action Coding System. A Technique for the Measurement of Facial Action (1978)Google Scholar
  4. 4.
    Hu, P.: Application research on face detection technology based on OpenCV in mobile augmented reality. Int. J. Sign. Process. Image Process. Pattern Recogn. 8, 249–256 (2015)Google Scholar
  5. 5.
    Shbib, R., Zhou, S.: Facial expression analysis using active shape model. Int. J. Sign. Process. Image Process. Pattern Recogn. 8, 9–22 (2016)Google Scholar
  6. 6.
    Li, Y., Liu, W., Li, X., Huang, Q., Li, X.: GA-SIFT: a new scale invariant feature transform for multispectral image using geometric algebra. Inf. Sci. 281, 559–572 (2014)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Chao, W.L., Ding, J.J., Liu, J.Z.: Facial expression recognition based on improved local binary pattern and class-regularized locality preserving projection. Signal Process. 117, 1–10 (2015)CrossRefGoogle Scholar
  8. 8.
    Ghimire, D.: Recognition of facial expressions based on tracking and selection of discriminative geometric features. Int. J. Multimed. Ubiquitous Eng. 10, 35–44 (2015)CrossRefGoogle Scholar
  9. 9.
    Dewan, M.A.A., Granger, E., Marcialis, G.L., Sabourin, R., Roli, F.: Adaptive appearance model tracking for still-to-video face recognition. Pattern Recogn. 49, 129–151 (2016)CrossRefGoogle Scholar
  10. 10.
    Zafeiriou, S., Pitas, I.: Discriminant graph structures for facial expression recognition. IEEE Trans. Multimed. 10, 1528–1540 (2008)CrossRefGoogle Scholar
  11. 11.
    Siddiqi, M.H., Ali, R., Khan, A.M., Park, Y.T., Lee, S.: Human facial expression recognition using stepwise linear discriminant analysis and hidden conditional random fields. IEEE Trans. Image Process. 24, 1386–1398 (2015)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Zhao, X.: Facial expression recognition using local binary patterns and discriminant kernel locally linear embedding. EURASIP J. Adv. Signal Process. 2012, 20 (2012)CrossRefGoogle Scholar
  13. 13.
    Wang, S., Yang, H., Li, H.: Facial expression recognition based on incremental isomap with expression weighted distance. J. Comput. 8, 2051–2058 (2013)Google Scholar
  14. 14.
    Taheri, S., Qiu, Q., Chellappa, R.: Structure-preserving sparse decomposition for facial expression analysis. IEEE Trans. Image Process. 23, 3590–3603 (2014)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Wu, C., Wang, S., Ji, Q.: Multi-instance hidden Markov model for facial expression recognition. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 1–6 (2015)Google Scholar
  16. 16.
    Bahari, F., Janghorbani, A.: EEG-based emotion recognition using recurrence plot analysis and K nearest neighbor classifier. In: Biomedical Engineering, pp. 228–233 (2013)Google Scholar
  17. 17.
    Wang, F., He, K., Liu, Y., Li, L., Hu, X.: Research on the selection of kernel function in SVM based facial expression recognition. In: Industrial Electronics and Applications, pp. 1404–1408 (2013)Google Scholar
  18. 18.
    Schapire, R.E.: Explaining AdaBoost. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  19. 19.
    Owusu, E., Zhan, Y., Mao, Q.R.: A neural-AdaBoost based facial expression recognition system. Expert Syst. Appl. 41, 3383–3390 (2014)CrossRefGoogle Scholar
  20. 20.
    Lopes, A.T., Aguiar, E.D., Souza, A.F.D., Oliveira-Santos, T.: Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn. 61, 610–628 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Shanghai Key Laboratory of Power Station Automation Technology, School of Mechatronic Engineering and AutomationShanghai UniversityShanghaiChina

Personalised recommendations