Skip to main content
SpringerLink
Log in

Comparative Study and Analysis of Various Facial Emotion Recognition Techniques

  • Chapter
  • First Online:
Decision Analytics Applications in Industry

Part of the book series: Asset Analytics ((ASAN))

Abstract

Facial emotions are important factors in human communication that help us understand the behavior of others. Emotion recognition has become a progressive research area and it plays a major role in human–computer interaction. Emotion is a physiological state which involves a lot of actions, thoughts, feelings, and behavior of an individual. Facial expression is a fundamental means to communicate with people. The ability to understand the facial emotion of others is a key to successful communication. Recognition of facial emotions by computer with high recognition rate is still a challenging task. The human emotions that are recognized universally are as follows: anger, disgust, happiness, fear, surprise, and sadness. Facial emotion recognition is basically performed in three phases: image pre-processing, feature extraction, and finally classification. This paper presents a survey of the current techniques that are used in the field of facial emotion recognition. The goal of this paper is to present a comparative study of various recognition techniques using different approaches. A summary of some of the papers is given in the tabular form to help researchers have a quick glance in this arena. A list of few challenges in this field is given at the end along with the possible future advancements.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Fasel B, Luettin J (2003) Automatic facial expression analysis a survey. Pattern Recognit 36(1):259–275

    Article  Google Scholar 

  2. Happy SL, Routray A (2015) Automatic facial expression recognition using features of salient facial patches. IEEE Trans Affect Comput 6(1):1–12

    Article  Google Scholar 

  3. Bai Y, Tang X, Zhu Y, Cai D (2014) A sentiment analysis method for facial expression generation in human-robot interactive communication. In: IEEE International Conference on Virtual Reality and Visualization (ICVRV), Shenyang, China, pp 97–102

    Google Scholar 

  4. Szwoch M, Pieniazek P (2015) Facial emotion recognition using depth data. In: IEEE 8th international conference on human system interactions (HSI), Warsaw, Poland, pp 271–277

    Google Scholar 

  5. Majumdar J, Avabhrith R (2014) Human face expression recognition. Int J Emerg Technol Adv Eng (IJETAE) 4(7):559–565

    Google Scholar 

  6. Khan NU (2013) A comparative analysis of facial expression recognition techniques. In: 3rd IEEE international advance computing conference (IACC), pp 1262–1268. https://doi.org/10.1109/iadcc.2013.6514409

  7. Anitha C, Venkatesha MK, Narayana Adiga BS (2010) A survey on facial expression databases. Int J Eng Sci Technol 2(10):5158–5174

    Google Scholar 

  8. Huang Hung-Fu, Tai Shen-Chuan (2012) Facial expression recognition using new feature extraction algorithm. ELCVIA: Electron Lett Comput Vis Image Anal 11(1):0041–54

    Article  Google Scholar 

  9. Hsu R-L, Abdel-Mottaleb M, Jain AK (2002) Face detection in color images. IEEE Trans Pattern Anal Mach Intell 24(5):696–706

    Google Scholar 

  10. Grgic M, Delac K, Grgic S (2011) SCface–surveillance cameras face database. In: Multimedia tools and applications, vol 51, no 3, pp 863–879

    Google Scholar 

  11. Priyanka MS, Jalan A, Nimbarte MS (2014) A survey on facial feature extraction to recognize facial expressions. Int J Eng Res Technol 3(2):539–544

    Google Scholar 

  12. Jabid T, Kabir MH, Chae O (2010) Robust facial expression recognition based on local directional pattern. ETRI J 32(5):784–794

    Article  Google Scholar 

  13. Baker S, Sim T, Bsat M (2003) The CMU pose, illumination, and expression database. IEEE Trans Pattern Anal Mach Intell 25(12):1615–1618

    Article  Google Scholar 

  14. He X, Yan S, Yuxiao H, Niyogi P, Zhang H-J (2005) Face recognition using laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340

    Article  Google Scholar 

  15. Mehta N, Jadhav S (2016) Facial emotion recognition using log gabor filter and PCA. In: International conference on computing communication control and automation (ICCUBEA), Pune, pp 1–5

    Google Scholar 

  16. Kamal S, Sayeed F, Rafeeq M (2016) Facial emotion recognition for human-computer interactions using hybrid feature extraction technique. In: IEEE international conference on data mining and advanced computing, Ernakulam, India, pp 180–184

    Google Scholar 

  17. Jun Ou (2012) Classification algorithms research on facial expression recognition. Phys Procedia 25:1241–1244

    Article  Google Scholar 

  18. Geng C, Jian X (2009) Face recognition using sift features. In: IEEE international conference on image processing (ICIP), Cairo, Egypt, pp 3313–3316

    Google Scholar 

  19. Kumari J, Rajesh R, Pooja KM (2015) Facial expression recognition: a survey. Procedia Comput Sci 58:486–491

    Article  Google Scholar 

  20. Lorincz A, Jeni L, Szabo Z, Cohn J, Kanade T (2013) Emotional expression classification using time-series kernels. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, Portland, OR, USA, pp 889–895

    Google Scholar 

  21. Agrawal DD, Dubey SR, Jalal AS (2014) Emotion recognition from facial expressions based on multi-level classification. Int J Comput Vis Robot 4(4):365–389

    Article  Google Scholar 

  22. Chang J, Ryoo S (2018) Implementation of an improved facial emotion retrieval method in multimedia system. In: Multimedia tools and applications, vol 77, no 4, pp 5059–5065

    Article  Google Scholar 

  23. Jain N, Kumar S, Kumar A, Shamsolmoali P, Zareapoor M (2018) Hybrid deep neural networks for face emotion recognition. Pattern Recognit Lett. https://doi.org/10.1016/j.patrec.2018.04.010

    Article  Google Scholar 

  24. Nigam S, Singh R, Misra1 AK (2018) Efficient facial expression recognition using histogram of oriented gradients in wavelet domain. In: Multimedia tools and applications, pp 1–23. https://doi.org/10.1007/s11042-018-6040-3

    Article  Google Scholar 

  25. Majumder Anima, Behera Laxmidhar, Subramanian Venkatesh K (2018) Automatic facial expression recognition system using deep network-based data fusion. IEEE Trans Cybernet 48(1):103–114

    Article  Google Scholar 

  26. Priya V, Muralidhar (2017) Facial emotion recognition using eye. Int J Appl Eng Res 12(16):5655–5659

    Google Scholar 

  27. Tarnowski P, Kolodziej M, Majkowski A, Rak RJ (2017) Emotion recognition using facial expressions. Procedia Comput Sci (Zurich, Switzerland) 108C:1175–1184

    Article  Google Scholar 

  28. Shan K, Guo J, You W, Lu D, Bie R (2017) Automatic facial expression recognition based on a deep convolutional-neural-network structure. In: IEEE 15th international conference on software engineering research, management and applications (SERA), London, UK, pp 123–128

    Google Scholar 

  29. Qayyum H, Majid M, Anwar SM, Khan B (2017) Facial expression recognition using stationary wavelet transform features. Math Probl Eng 2017:1–9, Article ID 9854050

    Article  Google Scholar 

  30. Yanga D, Alsadoona A, Prasad PWC, Singh AK, Elchouemic A (2017) An emotion recognition model based on facial recognition in virtual learning environment. In: 6th international conference on smart computing and communications (ICSCC), vol 125. Procedia Computer Science, Kurukshetra, pp 1–10

    Google Scholar 

  31. Wang S-H, Phillips P, Dong Z-C, Zhang Y-D (2018) Intelligent facial emotion recognition based on stationary wavelet entropy and Jaya algorithm. Neurocomputing 272:668–676. https://doi.org/10.1016/j.neucom.2017.08.015

    Article  Google Scholar 

  32. Al-Sumaidaee S, Abdullah MAM, Al-Nima R, Dlay SS, Chamber JA (2017) Multigradient features and elongated quinary pattern encoding for image-based facial expression recognition. Pattern Recognit. https://doi.org/10.1016/j.patcog.2017.06.007

    Article  Google Scholar 

  33. Suja SP, Tripathi S (2016) Real-time emotion recognition from facial images using raspberry Pi II. In: IEEE 3rd international conference on signal processing and integrated networks (SPIN), Noida, India, pp 666–670

    Google Scholar 

  34. Mert A, Akan A (2016) Emotion recognition from EEG signals by using multivariate empirical mode decomposition. Springer, London. https://doi.org/10.1007/s10044-016-0567-6

    Article  Google Scholar 

  35. Dixit BA, Gaikwad AN (2015) Statistical moments based facial expression analysis. In: IEEE international advance computing conference (IACC), Banglore, India, pp 552–557

    Google Scholar 

  36. Basu A, Routray A, Shit S, Deb AK (2015) Human emotion recognition from facial thermal image based on fused statistical feature and multi-class SVM. In: Annual IEEE India conference (INDICON), New Delhi, pp 1–5

    Google Scholar 

  37. Hsieh C-C, Hsih M-H, Jiang M-K, Cheng Y-M, Liang E-H (2015) Effective semantic features for facial expressions recognition using SVM. In: Multimedia tools and applications, vol 75, no 11, New York, pp 6663–6682

    Google Scholar 

  38. De A, Saha A, Pal MC (2015) A human facial expression recognition model based on Eigen face approach. In: Procedia computer science, international conference on advanced computing technologies and applications (ICACTA), vol 45, pp 282–289

    Article  Google Scholar 

  39. Owusu E, Zhan Y, Mao QR (2014) A neural-AdaBoost based facial expression recognition system. Expert Syst Appl 41(7):3383–3390

    Article  Google Scholar 

  40. Gu W, Xiang C, Venkatesh YV, Huang D, Lin H (2012) Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Elsevier J Pattern Recognit, vol 45, pp 80–91

    Article  Google Scholar 

  41. Liu S, Wang W (2010) The application study of learner’s face detection and location in the teaching network system based on emotion recognition. Second Int Conf Netw Secur Wirel Commun Trust Comput 01:394–397

    Google Scholar 

  42. Bashyal S, Venayagamoorthy GK (2008) Recognition of facial expressions using Gabor wavelets and learning vector quantization. Eng Appl Artif Intell 21:1056–1064

    Article  Google Scholar 

  43. Zhang Z, Luo P, Chen CL, Tang X (2018) From facial expression recognition to interpersonal relation prediction. Int J Comput Vision 126(5):1–20

    Article  Google Scholar 

  44. Li S, Deng W, Du J (2017) Reliable crowdsourcing and deep localitypreserving learning for expression recognition in the wild. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2584–2593

    Google Scholar 

  45. Pramerdorfer C, Kampel M (2016) Facial expression recognition using convolutional neural networks: State of the art. arXiv preprint arXiv:1612.02903

  46. Huynh X-P, Tran T-D et al (2016) Convolutional neural network models for facial expression recognition using BU-3DFE database. In: Information science and applications (ICISA), vol 376. Springer, Singapore, pp 441–450. Print ISBN 978-981-10-0556-5

    Google Scholar 

  47. Guo Y, Tao D, Yu J, Xiong H, Li Y, Tao D (2016) Deep neural networks with relativity learning for facial expression recognition. In: 2016 IEEE international conference on multimedia & expo workshops (ICMEW). IEEE, pp 1–6

    Google Scholar 

  48. Hamester D, Barros P, Wermter S (2015) Face expression recognition with a 2-channel convolutional neural network. In: 2015 international joint conference on neural networks (IJCNN). IEEE, pp 1–8

    Google Scholar 

  49. Jung H, Lee S et al (2015) Development of deep learning-based facial expression recognition system. In: IEEE 21st Korea-Japan joint workshop on frontiers of computer vision (FCV), Mokpo, South Korea. ISBN: 978-1-4799-1720-4

    Google Scholar 

  50. Liu M, Li S et al (2015) Au-inspired deep networks for facial expression feature learning. Neurocomputing 159:126–136

    Article  Google Scholar 

  51. Ouellet S (2014) Real-time emotion recognition for gaming using deep convolutional network features. arXiv:1408.3750

  52. Liu P, Han S, Meng Z, Tong Y (2014) Facial expression recognition via a boosted deep belief network. In: The IEEE conference on computer vision and pattern recognition (CVPR), pp 1805–1812

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Punjabi University Regional Centre for IT and Management, Mohali, India

    Naveen Kumari & Rekha Bhatia

Authors
  1. Naveen Kumari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rekha Bhatia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Naveen Kumari .

Editor information

Editors and Affiliations

  1. Centre for Interdisciplinary Research, Amity University, Noida, India

    P. K. Kapur

  2. Group VC, Amity University, Noida, Uttar Pradesh, India

    Gurinder Singh

  3. Department of Economics and Management in Machine Building, Peter the Great St. Petersburg Polytechnic, Saint Petersburg, Russia

    Yury S. Klochkov

  4. Operation and Maintenance Engineering, LuleĂĄ University of Technology, LuleĂĄ, Sweden

    Uday Kumar

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Kumari, N., Bhatia, R. (2020). Comparative Study and Analysis of Various Facial Emotion Recognition Techniques. In: Kapur, P.K., Singh, G., Klochkov, Y.S., Kumar, U. (eds) Decision Analytics Applications in Industry. Asset Analytics. Springer, Singapore. https://doi.org/10.1007/978-981-15-3643-4_11

Download citation

Publish with us

Policies and ethics

Search

Navigation