Building a Chinese Facial Expression Database for Automatically Detecting Academic Emotions to Support Instruction in Blended and Digital Learning Environments

  • Sunny S. J. LinEmail author
  • Wei Chen
  • Chun-Hsien Lin
  • Bing-Fei Wu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11937)


This paper specifically focuses on how to build a Chinese facial expression database collecting the facial expressions of college students and describes a strategy to develop an automatically detecting technique for academic emotions to support teachers making better decisions in blended and digital learning environments. There are some famous worldwide databases of facial emotion expressions, e.g., Amsterdam Dynamic Facial Expression Set (ADFES), Montreal set of facial displays of emotion, or Brazillian FEI database. Their major collections are full facial expression of western people with very limited Asian or Chinese faces. Because some emotion facial expressions might be culturally bounded, it arises the necessity to develop a Chinese facial expression database as a critical step to develop an automatically facial emotion expression dictating technique with high accuracy.


Facial expressions of emotion Basic emotions Academic emotions Blended and digital learning environments 


  1. 1.
    Izard, C.E.: The Psychology of Emotions. Plenum, New York (1991)CrossRefGoogle Scholar
  2. 2.
    Buck, R.: Social factors in facial display and communication: a reply to Chovil and others. J. Nonverbal Behav. 15, 155–162 (1991)CrossRefGoogle Scholar
  3. 3.
    Kobayashi, H., Hara, F.: Real time dynamic control of 6 basic facial expressions on face robot. J. Robot. Soc. Japan 14(5), 677–685 (1996)CrossRefGoogle Scholar
  4. 4.
    Ekman, P.: Emotional and conversational nonverbal signals. In: Larrazabal, J.M., Miranda, L.A.P. (eds.) Language, Knowledge, and Representation. Philosophical Studies Series, vol. 99, pp. 39–50. Springer, Dordrecht (2004). Scholar
  5. 5.
    Pekrun, R., Goetz, T., Titz, W., Perry, R.P.: Academic emotions in students’ self-regulated learning and achievement: a program of qualitative and quantitative research. Educ. Psychol. 37(2), 91–105 (2002)CrossRefGoogle Scholar
  6. 6.
    Matsumoto, D., Hwang, H.C.: Judgments of subtle facial expressions of emotion. Emotion 14(2), 349–357 (2014)CrossRefGoogle Scholar
  7. 7.
    Wu, B., Lin, C.: Adaptive feature mapping for customizing deep learning based facial expression recognition model. IEEE Access 6, 12451–12461 (2018)CrossRefGoogle Scholar
  8. 8.
    Ekman, P., Friesen, W.V.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)Google Scholar
  9. 9.
    Ekman, P.: Facial expression and emotion. Am. Psychol. 48, 384–392 (1993)CrossRefGoogle Scholar
  10. 10.
    Chen, Y.-M.: A deep learning and aggregation network based face surveillance system, tested by realistic scenarios (Unpublished master thesis advised by professor Being-Fei Wu). National Chiao Tung University, Hsinchu, Taiwan (2018)Google Scholar
  11. 11.
    Yang, J., Ren, P., Chen, D., Wen, F., Li, H., Hua, G.: Neural aggregation network for video face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CoRR (2016)Google Scholar
  12. 12.
    Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 23(10), 1499–1503 (2016)CrossRefGoogle Scholar
  13. 13.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK +): a complete facial expression dataset for action unit and emotion-specified expression. In: 3rd IEEE Workshop on CVPR for Human Communicative Behavior Analysis (2010)Google Scholar
  14. 14.
    Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D.H.J., Hawk, S.T., van Knippenberg, A.: Presentation and validation of the Radboud faces database. Cogn. Emot. 24(8), 1377–1388 (2010)CrossRefGoogle Scholar
  15. 15.
    Van der Schalk, J., Hawk, S.T., Fischer, A.H., Doosje, B.J.: Moving faces, looking places: the Amsterdam Dynamic Facial Expressions Set (ADFES). Emotion 11, 907–920 (2011)CrossRefGoogle Scholar
  16. 16.
    Facial action coding system. Accessed 2019
  17. 17.
    Zhang, X., et al.: BP4D-spontaneous: a high-resolution spontaneous 3D dynamic facial expression database. Image Vis. Comput. 32, 692–706 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Sunny S. J. Lin
    • 1
    Email author
  • Wei Chen
    • 1
  • Chun-Hsien Lin
    • 2
  • Bing-Fei Wu
    • 2
  1. 1.Institute of EducationNational Chiao Tung UniversityHsinchuTaiwan
  2. 2.Electrical and Control EngineeringNational Chiao Tung UniversityHsinchuTaiwan

Personalised recommendations