Development and Validation of Basic Virtual Human Facial Emotion Expressions

  • Miguel Á. Vicente-Querol
  • Arturo S. García
  • Patricia Fernández-Sotos
  • Roberto Rodriguez-Jimenez
  • Antonio Fernández-CaballeroEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11486)


This paper introduces the design process of facial expressions on virtual humans to play basic emotions. The design of the emotions is grounded on the Facial Action Coding System that enables describing facial expressions based on Action Units. All the tools employed to attain the final human avatar expressions are detailed. Then, an experiment with healthy volunteers is carried out to validate the designed virtual human facial emotions. As result, we obtained that the faces are correctly interpreted by healthy people with an accuracy of \(83.56\%\). Thus, as recognition works quite well with this small sample of healthy people, this paper is a first step towards validating and enhancing the avatar characters generated, experimenting with a sufficient number of healthy persons, and, then, designing therapies based on human avatars to enhance facial affect recognition in patients with deficits in facial affect recognition.


Virtual human Avatar Facial expression Facial affect recognition Virtual reality 



This work was partially supported by Spanish Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación (AEI)/European Regional Development Fund (FEDER, UE) under DPI2016-80894-R and TIN2015-72931-EXP grants, and by Biomedical Research Networking Centre in Mental Health (CIBERSAM) of the Instituto de Salud Carlos III.


  1. 1.
  2. 2.
    Adobe: Mixamo (2019).
  3. 3.
    Amini, R., Lisetti, C., Ruiz, G.: HapFACS 3.0: FACS-based facial expression generator for 3D speaking virtual characters. IEEE Trans. Affect. Comput. 6(4), 348–360 (2015)CrossRefGoogle Scholar
  4. 4.
  5. 5.
    Barkhof, E., de Sonneville, L.M., Meijer, C.J., de Haan, L.: Specificity of facial emotion recognition impairments in patients with multi-episode schizophrenia. Schizophr. Res.: Cogn. 2(1), 12–19 (2015)Google Scholar
  6. 6.
    Bordon, N., O’Rourke, S., Hutton, P.: The feasibility and clinical benefits of improving facial affect recognition impairments in schizophrenia: systematic review and meta-analysis. Schizophr. Res. 188, 3–12 (2017)CrossRefGoogle Scholar
  7. 7.
    Castillo, J.C., Fernández-Caballero, A., Castro-González, Á., Salichs, M.A., López, M.T.: A framework for recognizing and regulating emotions in the elderly. In: Ambient Assisted Living and Daily Activities, pp. 320–327 (2014)Google Scholar
  8. 8.
    Ekman, P.: Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life. Times Books, New York (2003)Google Scholar
  9. 9.
    Ekman, P., Friesen, W., Hager, J.: Facial action coding system investigator’s guide. Technical report, Research Nexus (2002)Google Scholar
  10. 10.
    Ekman, P., Friesen, W., Hager, J.: Facial action coding system: the manual on CD rom. Technical report, A Human Face (2002)Google Scholar
  11. 11.
    Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)CrossRefGoogle Scholar
  12. 12.
    Ekman, P., Friesen, W.V.: Unmasking the Face: A Guide to Recognizing Emotions from Facial Clues. ISHK (2003)Google Scholar
  13. 13.
    Fernández-Caballero, A., et al.: Human-avatar symbiosis in cognitive cybertherapies: proof of concept for auditory verbal hallucinations. In: Ubiquitous Computing and Ambient Intelligence, pp. 742–753 (2017)CrossRefGoogle Scholar
  14. 14.
    Fernández-Caballero, A., Latorre, J.M., Pastor, J.M., Fernández-Sotos, A.: Improvement of the elderly quality of life and care through smart emotion regulation. In: Ambient Assisted Living and Daily Activities, pp. 348–355 (2014)Google Scholar
  15. 15.
    Fernández-Caballero, A., et al.: Human-avatar symbiosis for the treatment of auditory verbal hallucinations in schizophrenia through virtual/augmented reality and brain-computer interfaces. Front. Neuroinformatics 11, 64 (2017)CrossRefGoogle Scholar
  16. 16.
    Fernández-Sotos, P., et al.: Digital technology for Internet access by patients with early stage schizophrenia in Spain: a multicenter research study. J. Med. Internet Res. 21, e11824 (2019)CrossRefGoogle Scholar
  17. 17.
    Friesen, E., Ekman, P.: Facial Action Coding System: a Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)Google Scholar
  18. 18.
    García, A.S., Navarro, E., Fernández-Caballero, A., González, P.: Towards the design of avatar-based therapies for enhancing facial affect recognition. In: Ambient Intelligence - Software and Applications - 9th International Symposium on Ambient Intelligence, pp. 306–313 (2019)Google Scholar
  19. 19.
    García-Sánchez, M., Teruel, M.A., Navarro, E., González, P., Fernández-Caballero, A.: A distributed tool to perform dynamic therapies for social cognitive deficit through avatars. In: Ubiquitous Computing and Ambient Intelligence, pp. 731–741 (2017)CrossRefGoogle Scholar
  20. 20.
    Guadagno, R.E., Swinth, K.R., Blascovich, J.: Social evaluations of embodied agents and avatars. Comput. Hum. Behav. 27(6), 2380–2385 (2011)CrossRefGoogle Scholar
  21. 21.
    Gutiérrez-Maldonado, J., Rus-Calafell, M., González-Conde, J.: Creation of a new set of dynamic virtual reality faces for the assessment and training of facial emotion recognition ability. Virtual Reality 18(1), 61–71 (2014)CrossRefGoogle Scholar
  22. 22.
    Lozano-Monasor, E., López, M.T., Fernández-Caballero, A., Vigo-Bustos, F.: Facial expression recognition from webcam based on active shape models and support vector machines. In: Ambient Assisted Living and Daily Activities, pp. 147–154 (2014)Google Scholar
  23. 23.
    Marwick, K., Hall, J.: Social cognition in schizophrenia: a review of face processing. Br. Med. Bull. 88(1), 43–58 (2008)CrossRefGoogle Scholar
  24. 24.
    NVIDIA: Texture Tools for Adobe Photoshop (2019).
  25. 25.
    Parke, F.I.: Computer generated animation of faces. In: Proceedings of the ACM Annual Conference. vol. 1, pp. 451–457. ACM (1972)Google Scholar
  26. 26.
    Russell, J.A.: Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychol. Bull. 115(1), 102 (1994)CrossRefGoogle Scholar
  27. 27.
    Sachs, G., et al.: Training of affect recognition (TAR) in schizophrenia-impact on functional outcome. Schizophr. Res. 138(2–3), 262–267 (2012)CrossRefGoogle Scholar
  28. 28.
    Slater, M., Steed, A.: Meeting people virtually: experiments in shared virtual environments. In: Schroeder R. (eds.) The Social Life of Avatars, pp. 146–171 (2002)CrossRefGoogle Scholar
  29. 29.
    Sokolova, M.V., Fernández-Caballero, A.: A review on the role of color and light in affective computing. Appl. Sci. 5(3), 275–293 (2015)CrossRefGoogle Scholar
  30. 30.
    Tessier, M.H., Gingras, C., Robitaille, N., Jackson, P.L.: Toward dynamic pain expressions in avatars: perceived realism and pain level of different action unit orders. Comput. Hum. Behav. 96, 95–109 (2019)CrossRefGoogle Scholar
  31. 31.
    Tinwell, A., Grimshaw, M., Williams, A.: Uncanny behaviour in survival horror games. J. Gaming Virtual Worlds 2(1), 3–25 (2010)CrossRefGoogle Scholar
  32. 32.
    Unity3d: Unity 3D (2019).
  33. 33.
    Wölwer, W., et al.: Neurophysiological correlates of impaired facial affect recognition in individuals at risk for schizophrenia. Schizophr. Bull. 38(5), 1021–1029 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Miguel Á. Vicente-Querol
    • 1
  • Arturo S. García
    • 1
    • 2
  • Patricia Fernández-Sotos
    • 3
    • 4
  • Roberto Rodriguez-Jimenez
    • 3
    • 4
    • 5
  • Antonio Fernández-Caballero
    • 1
    • 2
    • 4
    Email author
  1. 1.Instituto de Investigación en Informática de AlbaceteUniversidad de Castilla-La ManchaAlbaceteSpain
  2. 2.Departamento de Sistemas InformáticosUniversidad de Castilla-La ManchaAlbaceteSpain
  3. 3.Department of PsychiatryInstituto de Investigación Sanitaria Hospital 12 de Octubre (imas12)MadridSpain
  4. 4.CIBERSAM (Biomedical Research Networking Centre in Mental Health)MadridSpain
  5. 5.CogPsy-GroupUniversidad Complutense de MadridMadridSpain

Personalised recommendations