Skip to main content
SpringerLink
Log in

Emotion Recognition System for Human-Robot Interface: Comparison of Two Approaches

  • Conference paper
  • First Online:
Interactive Collaborative Robotics (ICR 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10459))

Included in the following conference series:

Abstract

This paper describes a system for automatic emotion recognition developed to enhance the communication capabilities of an anthropomorphic robot. Two versions of the classification algorithm are proposed and compared. The first version is based on a classic approach requiring the action unit estimation as a preliminary step to emotion recognition. The second version takes advantage of convolutional neural networks as a classifier. The designed system is capable of working in real time. The algorithms were implemented on C++ and tested on an extensive face expression database as well as in real conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bartlett, M.S., Hager, J.C., Ekman, P., Sejnowskie, T.J.: Measuring facial expressions by computer image analysis. Psychophysiology 36, 253–263 (1999). Cambridge University Press

    Article  Google Scholar 

  2. Zaboleeva-Zotova, A.: The development of the automated determination of the emotions and the possible scope. Open Educ. 2, 59–62 (2011)

    Google Scholar 

  3. den Uyl, M., van Kuilenberg, H.: The FaceReader: online facial expression recognition. In: Proceedings of Measuring Behavior 2005, 5th International Conference on Methods and Techniques in Behavioral Research, pp. 589–590. Noldus Information Technology, Wageningen (2005)

    Google Scholar 

  4. Ekman, P., Friesen, W.V., Hager, J.C.: Facial Acton Coding System: The Manual. Research Nexus Division of Network Information Research Corporation (2002)

    Google Scholar 

  5. Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition (FG 2000), Grenoble, France, pp. 46–53 (2000)

    Google Scholar 

  6. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete expression dataset for action unit and emotion-specified expression. In: Proceedings of the Third International Workshop on CVPR for Human Communicative Behavior Analysis (CVPR4HB 2010), San Francisco, USA, pp. 94–101 (2010)

    Google Scholar 

  7. Chandrani, S., Washef, A., Soma, M., Debasis, M.: Facial expressions: a cross-cultural study. In: Emotion Recognition: A Pattern Analysis Approach, pp. 69–86. Wiley (2015)

    Google Scholar 

  8. Huang, X., Wang, S., Liu, X., Zhao, G., Feng, X., Pietikäinen, M.: Spontaneous facial micro-expression recognition using discriminative spatiotemporal local binary pattern with an improved integral projection. J. Latex Class Files 14(8), 1–13 (2015)

    Google Scholar 

  9. Zhao, G., Pietikäinen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29, 915–928 (2007)

    Article  Google Scholar 

  10. King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)

    Google Scholar 

  11. Matsugu, M., Mori, K., Mitari, Y., Kaneda, Y.: Subject independent facial expression recognition with robust face detection using a convolutional neural network. Neural Netw. 16, 555–559 (2003)

    Article  Google Scholar 

  12. Owusu, E., Zhan, Y., Mao, Q.R.: A neural-AdaBoost based facial expression recognition system. Expert Syst. Appl. 41, 3383–3390 (2014)

    Article  Google Scholar 

  13. Kung, S.H., Zohdy, M.A., Bouchaffra, D.: 3D HMM-based facial expression recognition using histogram of oriented optical flow. Trans. Mach. Learn. Artif. Intell. 3(6), 42–69 (2015)

    Google Scholar 

  14. Cid, F., Prado, J.A., Bustos, P., Nunez, P.: A real time and robust facial expression recognition and imitation approach for affective human-robot interaction using Gabor filtering. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan, pp. 2188–2193 (2013)

    Google Scholar 

  15. Majumder, A., Behera, L., Subramanian, V.K.: Emotion recognition from geometric facial features using self-organizing map. Pattern Recogn. 47(3), 1282–1293 (2014)

    Article  Google Scholar 

  16. Zhang, L., Jiang, M., Farid, D., Hossain, M.A.: Intelligent facial emotion recognition and semantic-based topic detection for a humanoid robot. Expert Syst. Appl. 40(13), 5160–5168 (2013)

    Article  Google Scholar 

  17. Yuschenko, A., Vorotnikov, S., Konyshev, D., Zhonin, A.: Mimic recognition and reproduction in bilateral human-robot speech communication. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds.) ICR 2016. LNCS, vol. 9812, pp. 133–142. Springer, Cham (2016). doi:10.1007/978-3-319-43955-6_17

    Chapter  Google Scholar 

  18. Castellano, G., Kessous, L., Caridakis, G.: Emotion recognition through multiple modalities: face, body gesture, speech. In: Peter, C., Beale, R. (eds.) Affect and Emotion in Human-Computer Interaction. LNCS, vol. 4868, pp. 92–103. Springer, Heidelberg (2008). doi:10.1007/978-3-540-85099-1_8

    Chapter  Google Scholar 

  19. Jirayucharoensak, S., Pan-Ngum, S., Israsena, P.: EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation. Sci. World J. 2014, 10 p. (2014). Article ID 627892

    Google Scholar 

Download references

Acknowledgements

This work was accomplished through financial support of Russian Foundation for Basic Research (RFBR), grant â„–16-07-01080.

Author information

Authors and Affiliations

  1. Neurobotics Ltd., 4922 passage, 2/4, Zelenograd, Moscow, 124498, Russian Federation

    Anatoly Bobe

  2. BMSTU Robototechnika, Izmaylovskaya sq., 7, Moscow, 105037, Russian Federation

    Dmitry Konyshev & Sergey Vorotnikov

Authors
  1. Anatoly Bobe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dmitry Konyshev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sergey Vorotnikov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anatoly Bobe .

Editor information

Editors and Affiliations

  1. SPIIRAS , St. Petersburg, Russia

    Andrey Ronzhin

  2. Technical University of Munich, Munich, Germany

    Gerhard Rigoll

  3. Tomsk State University , Tomsk, Russia

    Roman Meshcheryakov

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Bobe, A., Konyshev, D., Vorotnikov, S. (2017). Emotion Recognition System for Human-Robot Interface: Comparison of Two Approaches. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds) Interactive Collaborative Robotics. ICR 2017. Lecture Notes in Computer Science(), vol 10459. Springer, Cham. https://doi.org/10.1007/978-3-319-66471-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-66471-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-66470-5

  • Online ISBN: 978-3-319-66471-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation