Skip to main content
SpringerLink
Log in

Facial Expression Based Emotion Recognition Using Neural Networks

  • Conference paper
  • First Online:
Image Analysis and Recognition (ICIAR 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10882))

Included in the following conference series:

  • 4985 Accesses

Abstract

Facial emotion recognition has been extensively studied over the last decade due to its various applications in the fields such as human-computer interaction and data analytics. In this paper, we develop a facial emotion recognition approach to classify seven emotional states (joy, sadness, surprise, anger, fear, disgust and neutral). Seventeen action units tracked by Kinect v2 sensor have been used as features. Classification of emotions was performed by artificial neural networks (ANNs). Six subjects took part in the experiment. We have achieved average accuracy of 95.8% for the case in which we tested our approach with the same volunteers took part in our data generation process. We also evaluated the performance of the network with additional volunteers who were not part of the training data and achieved 67.03% classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mehrabian, A.: Communication without words. In: Mortensen, C.D. (ed.) Communication Theory, pp. 193–200. Transaction Publishers, New Brunswick (2008)

    Google Scholar 

  2. Ekman, P., Friesen, W.: Facial Action Coding System. Consulting Psychologists Press, Stanford University, Palo Alto (1977)

    Google Scholar 

  3. Wang, P., Barrett, F., Martin, E., Milonova, M., Gur, R.E., Gur, R.C., Kohler, C., Verma, R.: Automated video-based facial expression analysis of neuropsychiatric disorders. J. Neurosci. Methods 168(1), 224–238 (2008)

    Article  Google Scholar 

  4. Tong, Y., Chen, R., Cheng, Y.: Facial expression recognition algorithm using LGC based on horizontal and diagonal prior principle. Optik-Int. J. Light Electron. Optics 125, 4186–4189 (2014)

    Article  Google Scholar 

  5. Jabid, T., Kabir, M.H., Chae, O.: Facial expression recognition using local directional pattern (IDP). In: 17th IEEE International Conference on Image Processing (ICIP). IEEE (2010)

    Google Scholar 

  6. Guo, Y., Tian, Y., Gao, X., Zhang, X.: Micro-expression recognition based on local binary patterns from three orthogonal planes and nearest neighbor method. In: International Joint Conference on Neural Networks (IJCNN). IEEE (2014)

    Google Scholar 

  7. Gizatdinova, Y., Surakka, V., Zhao, G., Makinen, E., Raisamo, R.: Facial expression classification based on local spatiotemporal edge and texture descriptors. In: Proceedings of the 7th International Conference on Methods and Techniques in Behavioral Research. ACM (2010)

    Google Scholar 

  8. Kabir, M.H., Jabid, T., Chae, O.: Local directional pattern variance (IDPV): a robust feature descriptor for facial expression recognition. Int. Arab. J. Inf. Technol. 9(4), 382–391 (2012)

    Google Scholar 

  9. Mao, Q., Pan, X., Zhan, Y., Shen, X.: Using Kinect for real-time emotion recognition via facial expressions. Front. Inf. Technol. Electron. Eng. 16(4), 272–282 (2015)

    Article  Google Scholar 

  10. Youssef, A.E., Aly, S.F., Ibrahim, A.S., Abbott, A.L.: Auto-optimized multimodal expression recognition framework using 3D kinect data for ASD therapeutic aid. Int. J. Model. Optim. 3(2), 112–115 (2013)

    Article  Google Scholar 

  11. Zhang, Z., Cui, L., Liu, X., Zhu, T.: Emotion detection using Kinect 3D facial points. In: International Conference on Web Intelligence (2016)

    Google Scholar 

  12. Tarnowski, P., Kolodziej, M., Majkowkski, A., Rak, R.: Emotion recognition using facial expressions. In: International Conference on Computer Science, ICCS (2017)

    Article  Google Scholar 

  13. Microsoft SDK for Face Tracking. http://msdn.microsoft.com/enus/library/jj130970.aspx

Download references

Author information

Authors and Affiliations

  1. Faculty of Engineering and Natural Sciences, Sabanci University, 34956, Tuzla, Istanbul, Turkey

    Ekin Yağış & Mustafa Unel

Authors
  1. Ekin Yağış
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mustafa Unel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mustafa Unel .

Editor information

Editors and Affiliations

  1. University of Porto, Porto, Portugal

    Aurélio Campilho

  2. University of Waterloo, Waterloo, Ontario, Canada

    Fakhri Karray

  3. Biomedical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands

    Bart ter Haar Romeny

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yağış, E., Unel, M. (2018). Facial Expression Based Emotion Recognition Using Neural Networks. In: Campilho, A., Karray, F., ter Haar Romeny, B. (eds) Image Analysis and Recognition. ICIAR 2018. Lecture Notes in Computer Science(), vol 10882. Springer, Cham. https://doi.org/10.1007/978-3-319-93000-8_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-93000-8_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-92999-6

  • Online ISBN: 978-3-319-93000-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation