Advertisement

Emotion Recognition Using a Convolutional Neural Network

  • Ramon Zatarain-Cabada
  • Maria Lucia Barron-Estrada
  • Francisco González-Hernández
  • Hector Rodriguez-Rangel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10633)

Abstract

Learning-oriented emotions have not been studied by emotion recognition systems. These emotions have not been taken into account by other studies despite their importance in educational context. This work presents a recognition system which uses deep learning approach using convolutional neural network for solving that problem. A convolutional architecture was designed and tested with 3 different facial expression databases. The architecture is composed of 3 convolutional layers, 3 max-pooling layers, and 3 deep neural networks. The first database contains facial images on 6 basic emotions; the second and third databases contain images of learning-centered facial expressions. The tests show a 95% in the basic emotion database, a 97% for the first learning-centered emotion database and a 75% for the third database. We discuss about the differences in results among the three emotion databases.

Keywords

Deep learning Artificial intelligence Face expression recognition Face expression database 

References

  1. 1.
    Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)Google Scholar
  2. 2.
    Ekman, P.: An argument for basic emotions. Cogn. Emot. 6, 169–200 (1992)CrossRefGoogle Scholar
  3. 3.
    D’Mello, S.K., Graesser, A.C.: AutoTutor and affective AutoTutor: learning by talking with cognitively and emotionally intelligent computers that talk back? ACM 15, 434–442 (2011)Google Scholar
  4. 4.
    Wiggins, J.B., et al.: JavaTutor: an intelligent tutoring system that adapts to cognitive and affective states during computer programming. In: Proceedings of the 46th ACM Technical Symposium on Computer Science Education - SIGCSE 2015, p. 599. ACM (2015)Google Scholar
  5. 5.
    Pekrun, R.: Emotions and learning. Harv. Educ. Rev. 25, 95–104 (2014)Google Scholar
  6. 6.
    D’Mello, S., Graesser, A.: Dynamics of affective states during complex learning. Learn. Instr. 22, 145–157 (2012)CrossRefGoogle Scholar
  7. 7.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)CrossRefGoogle Scholar
  8. 8.
    Glorot, X.: Domain adaptation for large-scale sentiment classification: a deep learning approach. In: Proceedings of the 28th International Conference on Machine Learning, pp. 513–520 (2011)Google Scholar
  9. 9.
    Hinton, G., et al.: Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Sig. Proc. Mag. 29, 82–97 (2012)CrossRefGoogle Scholar
  10. 10.
    Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1891–1898 (2014)Google Scholar
  11. 11.
    Levi, G., Hassner, T.: Emotion recognition in the wild via convolutional neural networks and mapped binary patterns. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 503–510 (2015)Google Scholar
  12. 12.
    Kim, B., Lee, H., Roh, J., Lee, S.: Hierarchical committee of deep CNNs with exponentially-weighted decision fusion for static facial expression recognition. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction - ICMI 2015, pp. 427–434. ACM Press, New York (2015)Google Scholar
  13. 13.
    Yu, Z., Zhang, C.: Image based static facial expression recognition with multiple deep network learning. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction - ICMI 2015, pp. 435–442. ACM Press, New York (2015)Google Scholar
  14. 14.
    Kanade, T., Cohn, J., Tian, Y.: Comprehensive database for facial expression analysis. In: Automatic Face and Gesture Recognition, pp. 46–53. IEEE (2000)Google Scholar
  15. 15.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kande dataset (CK+): a complete facial expression dataset for action unit and emotion specified expression. In: CVPRW, pp. 94–101. IEEE (2010)Google Scholar
  16. 16.
    Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D.H.J., Hawk, S.T., van Knippenberg, A.: Presentation and validation of the Radboud faces database. Cogn. Emot. 24, 1377–1388 (2010)CrossRefGoogle Scholar
  17. 17.
    McKeown, G., Valstar, M., Cowie, R., Pantic, M., Schroder, M.: The SEMAINE database: annotated multimodal records of emotionally colored conversations between a person and a limited agent. IEEE Trans. Affect. Comput. 3, 5–17 (2012)CrossRefGoogle Scholar
  18. 18.
    Valstar, M.F., Pantic, M.: Induced disgust, happiness and surprise: an addition to the MMI facial expression database. In: Proceedings of International Conference on Language Resources and Evaluation, Workshop on EMOTION, pp. 65–70 (2010)Google Scholar
  19. 19.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, p. I-511–I-518. IEEE Computer Society (2001)Google Scholar
  20. 20.
    Cabada, R.Z., Estrada, M.L.B., Hernandez, F.G., Bustillos, R.O.: An affective learning environment for Java. In: 2015 IEEE 15th International Conference on Advanced Learning Technologies, pp. 350–354. IEEE (2015)Google Scholar
  21. 21.
    Cingolani, P., Alcala-Fdez, J.: jFuzzyLogic: a java library to design fuzzy logic controllers according to the standard for fuzzy control programming. Int. J. Comput. Intell. Syst. 6, 61–75 (2013)CrossRefGoogle Scholar
  22. 22.
    Zatarain-Cabada, R., Barrón-Estrada, M.L., González-Hernández, F., Oramas-Bustillos, R., Alor-Hernández, G., Reyes-García, C.A.: Building a corpus and a local binary pattern recognizer for learning-centered emotions. In: Advances in Artificial Intelligence and its Applications (2016)Google Scholar
  23. 23.
    Emotiv Systems: Emotiv EPOC. http://emotiv.wikia.com/wiki/Emotiv_EPOC
  24. 24.
    Emotiv Systems: Emotiv Insight. https://www.emotiv.com/insight/
  25. 25.
    Bosch, N., et al.: Automatic detection of learning-centered affective states in the wild. In: ACM (ed.) Proceedings of the 20th International Conference on Intelligent User Interfaces - IUI 2015, pp. 379–388 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Ramon Zatarain-Cabada
    • 1
  • Maria Lucia Barron-Estrada
    • 1
  • Francisco González-Hernández
    • 1
  • Hector Rodriguez-Rangel
    • 1
  1. 1.Posgrado en Ciencias de la ComputaciónInstituto Tecnológico de CuliacánCuliacánMexico

Personalised recommendations