Skip to main content
SpringerLink
Log in

Spontaneous Emotion Recognition in Response to Videos

  • Conference paper
  • First Online:
Software Technology: Methods and Tools (TOOLS 2019)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 11771))

Included in the following conference series:

  • 877 Accesses

Abstract

In order to understand human emotions correctly taking into account only facial expressions, we are conducting the experiments on the spontaneous emotional facial videos of people watching musical video clips from DEAP open source dataset. We are reporting the comparative results of emotion recognition done in two ways: sequential extraction of spatial and temporal features done by CNN-RNN, simultaneous extraction of both types of features performed by 3D convolutions in our C3D networks architecture. In order to study the contribution of microexpressions to emotion recognition we are augmenting videos in two ways: reducing to 1 fps, thus losing a significant amount of temporal information, reducing to 10 fps, thus preserving most of the muscle movement information.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alu, D., Zoltan, E., Stoica, I.C.: Voice based emotion recognition with convolutional neural networks for companion robots. Sci. Technol. 20(3), 222–240 (2017)

    Google Scholar 

  2. Barrett, L.F., Robin, L., Pietromonaco, P.R., Eyssell, K.M.: Are women the “more emotional” sex? Evidence from emotional experiences in social context. Cogn. Emot. 12(4), 555–578 (1998)

    Article  Google Scholar 

  3. Braun, M., Weiser, S., Pfleging, B., Alt, F.: A comparison of emotion elicitation methods for affective driving studies, pp. 77–81 September 2018. https://doi.org/10.1145/3239092.3265945

  4. Castellano, G., Kessous, L., Caridakis, G.: Emotion recognition through multiple modalities: face, body gesture, speech. In: Peter, C., Beale, R. (eds.) Affect and Emotion in Human-Computer Interaction. LNCS, vol. 4868, pp. 92–103. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-85099-1_8

    Chapter  Google Scholar 

  5. Darwin, C.: The Expression of the Emotions in Man and Animals (1872). The original was published 1898 by Appleton, New York. Reprinted 1965 by the University of Chicago Press, Chicago and London

    Google Scholar 

  6. Das, P., Khasnobish, A., Tibarewala, D.: Emotion recognition employing ECG and GSR signals as markers of ANS. In: 2016 Conference on Advances in Signal Processing (CASP), pp. 37–42. IEEE (2016)

    Google Scholar 

  7. Dhall, A., Goecke, R., Lucey, S., Gedeon, T., et al.: Collecting large, richly annotated facial-expression databases from movies. IEEE multimedia 19(3), 34–41 (2012)

    Article  Google Scholar 

  8. Fan, Y., Lu, X., Li, D., Liu, Y.: Video-based emotion recognition using CNN-RNN and C3D hybrid networks. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction ICMI 2016, pp. 445–450. ACM, New York (2016). https://doi.org/10.1145/2993148.2997632

  9. Koelstra, S., et al.: Deap: a database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2012). https://doi.org/10.1109/T-AFFC.2011.15

    Article  Google Scholar 

  10. Paszke, A., et al.: Automatic differentiation in pytorch (2017)

    Google Scholar 

  11. Peng, M., Wang, C., Chen, T., Liu, G., Fu, X.: Dual temporal scale convolutional neural network for micro-expression recognition. Front. psychol. 8, 1745 (2017)

    Article  Google Scholar 

  12. Soleymani, M., Pantic, M., Pun, T.: Multimodal emotion recognition in response to videos. IEEE Trans. Affect. Comput. 3(2), 211–223 (2011)

    Article  Google Scholar 

  13. Sun, M.C., Hsu, S.H., Yang, M.C., Chien, J.H.: Context-aware cascade attention-based RNN for video emotion recognition. In: 2018 First Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia), pp. 1–6. IEEE (2018)

    Google Scholar 

  14. Tomkins, S.S.: Affect, Imagery, Consciousness. Springer, Heidelberg (1962)

    Google Scholar 

  15. Tomkins, S.S., McCarter, R.: What and where are the primary affects? Some evidence for a theory. Percept. Mot. Skills 18(1), 119–158 (1964). https://doi.org/10.2466/pms.1964.18.1.119

    Article  Google Scholar 

  16. Tripathi, S., Acharya, S., Sharma, R.D., Mittal, S., Bhattacharya, S.: Using deep and convolutional neural networks for accurate emotion classification on DEAP dataset. In: Twenty-Ninth IAAI Conference (2017)

    Google Scholar 

  17. Vrij, A.: Book review - ‘Telling lies: clues to deceit in the marketplace, politics, and marriage’ by Paul Ekman. Int. J. Police Sci. Manag. 5, 209–210 (2003). https://doi.org/10.1350/ijps.5.3.209.16063

    Article  Google Scholar 

  18. Wu, S., Xu, X., Shu, L., Hu, B.: Estimation of valence of emotion using two frontal EEG channels. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1127–1130. IEEE (2017)

    Google Scholar 

  19. Zeng, Z., Hu, Y., Roisman, G.I., Wen, Z., Fu, Y., Huang, T.S.: Audio-visual spontaneous emotion recognition. In: Huang, T.S., Nijholt, A., Pantic, M., Pentland, A. (eds.) Artifical Intelligence for Human Computing. LNCS (LNAI), vol. 4451, pp. 72–90. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-72348-6_4

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Software Development and Engineering, Innopolis University, Innopolis, Russia

    Alisa Gazizullina & Manuel Mazzara

Authors
  1. Alisa Gazizullina
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manuel Mazzara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alisa Gazizullina .

Editor information

Editors and Affiliations

  1. Innopolis University, Innopolis, Russia

    Manuel Mazzara

  2. IUT de Blagnac , Blagnac, France

    Jean-Michel Bruel

  3. Innopolis University, Innopolis, Russia

    Bertrand Meyer

  4. Ivannikov Institute for System Programming, Russian Academy of Sciences, Moscow, Russia

    Alexander Petrenko

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gazizullina, A., Mazzara, M. (2019). Spontaneous Emotion Recognition in Response to Videos. In: Mazzara, M., Bruel, JM., Meyer, B., Petrenko, A. (eds) Software Technology: Methods and Tools. TOOLS 2019. Lecture Notes in Computer Science(), vol 11771. Springer, Cham. https://doi.org/10.1007/978-3-030-29852-4_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-29852-4_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-29851-7

  • Online ISBN: 978-3-030-29852-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation