Emotion Recognition Using a Convolutional Neural Network

  • Ramon Zatarain-CabadaEmail author
  • Maria Lucia Barron-Estrada
  • Francisco González-Hernández
  • Hector Rodriguez-Rangel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10633)


Learning-oriented emotions have not been studied by emotion recognition systems. These emotions have not been taken into account by other studies despite their importance in educational context. This work presents a recognition system which uses deep learning approach using convolutional neural network for solving that problem. A convolutional architecture was designed and tested with 3 different facial expression databases. The architecture is composed of 3 convolutional layers, 3 max-pooling layers, and 3 deep neural networks. The first database contains facial images on 6 basic emotions; the second and third databases contain images of learning-centered facial expressions. The tests show a 95% in the basic emotion database, a 97% for the first learning-centered emotion database and a 75% for the third database. We discuss about the differences in results among the three emotion databases.


Deep learning Artificial intelligence Face expression recognition Face expression database 


  1. 1.
    Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)Google Scholar
  2. 2.
    Ekman, P.: An argument for basic emotions. Cogn. Emot. 6, 169–200 (1992)CrossRefGoogle Scholar
  3. 3.
    D’Mello, S.K., Graesser, A.C.: AutoTutor and affective AutoTutor: learning by talking with cognitively and emotionally intelligent computers that talk back? ACM 15, 434–442 (2011)Google Scholar
  4. 4.
    Wiggins, J.B., et al.: JavaTutor: an intelligent tutoring system that adapts to cognitive and affective states during computer programming. In: Proceedings of the 46th ACM Technical Symposium on Computer Science Education - SIGCSE 2015, p. 599. ACM (2015)Google Scholar
  5. 5.
    Pekrun, R.: Emotions and learning. Harv. Educ. Rev. 25, 95–104 (2014)Google Scholar
  6. 6.
    D’Mello, S., Graesser, A.: Dynamics of affective states during complex learning. Learn. Instr. 22, 145–157 (2012)CrossRefGoogle Scholar
  7. 7.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)CrossRefGoogle Scholar
  8. 8.
    Glorot, X.: Domain adaptation for large-scale sentiment classification: a deep learning approach. In: Proceedings of the 28th International Conference on Machine Learning, pp. 513–520 (2011)Google Scholar
  9. 9.
    Hinton, G., et al.: Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Sig. Proc. Mag. 29, 82–97 (2012)CrossRefGoogle Scholar
  10. 10.
    Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1891–1898 (2014)Google Scholar
  11. 11.
    Levi, G., Hassner, T.: Emotion recognition in the wild via convolutional neural networks and mapped binary patterns. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 503–510 (2015)Google Scholar
  12. 12.
    Kim, B., Lee, H., Roh, J., Lee, S.: Hierarchical committee of deep CNNs with exponentially-weighted decision fusion for static facial expression recognition. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction - ICMI 2015, pp. 427–434. ACM Press, New York (2015)Google Scholar
  13. 13.
    Yu, Z., Zhang, C.: Image based static facial expression recognition with multiple deep network learning. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction - ICMI 2015, pp. 435–442. ACM Press, New York (2015)Google Scholar
  14. 14.
    Kanade, T., Cohn, J., Tian, Y.: Comprehensive database for facial expression analysis. In: Automatic Face and Gesture Recognition, pp. 46–53. IEEE (2000)Google Scholar
  15. 15.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kande dataset (CK+): a complete facial expression dataset for action unit and emotion specified expression. In: CVPRW, pp. 94–101. IEEE (2010)Google Scholar
  16. 16.
    Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D.H.J., Hawk, S.T., van Knippenberg, A.: Presentation and validation of the Radboud faces database. Cogn. Emot. 24, 1377–1388 (2010)CrossRefGoogle Scholar
  17. 17.
    McKeown, G., Valstar, M., Cowie, R., Pantic, M., Schroder, M.: The SEMAINE database: annotated multimodal records of emotionally colored conversations between a person and a limited agent. IEEE Trans. Affect. Comput. 3, 5–17 (2012)CrossRefGoogle Scholar
  18. 18.
    Valstar, M.F., Pantic, M.: Induced disgust, happiness and surprise: an addition to the MMI facial expression database. In: Proceedings of International Conference on Language Resources and Evaluation, Workshop on EMOTION, pp. 65–70 (2010)Google Scholar
  19. 19.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, p. I-511–I-518. IEEE Computer Society (2001)Google Scholar
  20. 20.
    Cabada, R.Z., Estrada, M.L.B., Hernandez, F.G., Bustillos, R.O.: An affective learning environment for Java. In: 2015 IEEE 15th International Conference on Advanced Learning Technologies, pp. 350–354. IEEE (2015)Google Scholar
  21. 21.
    Cingolani, P., Alcala-Fdez, J.: jFuzzyLogic: a java library to design fuzzy logic controllers according to the standard for fuzzy control programming. Int. J. Comput. Intell. Syst. 6, 61–75 (2013)CrossRefGoogle Scholar
  22. 22.
    Zatarain-Cabada, R., Barrón-Estrada, M.L., González-Hernández, F., Oramas-Bustillos, R., Alor-Hernández, G., Reyes-García, C.A.: Building a corpus and a local binary pattern recognizer for learning-centered emotions. In: Advances in Artificial Intelligence and its Applications (2016)Google Scholar
  23. 23.
    Emotiv Systems: Emotiv EPOC.
  24. 24.
    Emotiv Systems: Emotiv Insight.
  25. 25.
    Bosch, N., et al.: Automatic detection of learning-centered affective states in the wild. In: ACM (ed.) Proceedings of the 20th International Conference on Intelligent User Interfaces - IUI 2015, pp. 379–388 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Ramon Zatarain-Cabada
    • 1
    Email author
  • Maria Lucia Barron-Estrada
    • 1
  • Francisco González-Hernández
    • 1
  • Hector Rodriguez-Rangel
    • 1
  1. 1.Posgrado en Ciencias de la ComputaciónInstituto Tecnológico de CuliacánCuliacánMexico

Personalised recommendations