Responses during Facial Emotional Expression Recognition Tasks Using Virtual Reality and Static IAPS Pictures for Adults with Schizophrenia
Technology-assisted intervention has the potential to adaptively individualize and improve outcomes of traditional schizophrenia (SZ) intervention. Virtual reality (VR) technology, in particular, has the potential to simulate real world social and communication interactions and hence could be useful as a therapeutic platform for SZ. Emotional face recognition is considered among the core building blocks of social communication. Studies have shown that emotional face processing and understanding is impaired in patients with SZ. The current study develops a novel VR-based system that presents avatars that can change their facial emotion dynamically for emotion recognition tasks. Additionally, this system allows real-time measurement of physiological signals and eye gaze during the emotion recognition tasks, which can be used to gain insight about the emotion recognition process in SZ population. This study further compares VR-based facial emotion recognition with that of the more traditional emotion recognition from static faces using a small usability study. Results from the usability study suggest that VR could be a viable platform for SZ intervention and implicit signals such as physiological signals and eye gaze can be utilized to better understand the underlying pattern that is not available from user reports and performance alone.
Keywordsfacial expression emotion recognition virtual reality IAPS adaptive interaction eye tracking physiological processing schizophrenia intervention
Unable to display preview. Download preview PDF.
- 2.Bleuler, E.: Dementia praecox or the group of schizophrenias. International University Press, New York (1911)Google Scholar
- 3.Andreasen, N.C.: Scale for the assessment of negative symptoms. University of Iowa, Iowa City (1983)Google Scholar
- 15.Andreasen, N.C.: Scale for the assessment of positive symptoms. University of Iowa, Iowa City (1984)Google Scholar
- 16.Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (IAPS): Technical manual and affective ratings. Gainesville, FL: The Center for Research in Psychophysiology, University of Florida (1999)Google Scholar
- 18.Lahiri, U., Warren, Z., Sarkar, N.: Design of a Gaze-Sensitive Virtual Social Interactive System for Children With Autism. IEEE Transactions on Neural Systems and Rehabilitation Engineering (99), 1 (2012)Google Scholar
- 19.Cacioppo, J.T., Tassinary, L.G., Berntson, G.G.: Handbook of psychophysiology. Cambridge Univ. Pr. (2007)Google Scholar
- 20.Welch, K., Lahiri, U., Liu, C., Weller, R., Sarkar, N., Warren, Z.: An Affect-Sensitive Social Interaction Paradigm Utilizing Virtual Reality Environments for Autism Intervention. Paper presented at the Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction (2009)Google Scholar
- 21.Bekele, E., Zheng, Z., Swanson, A., Crittendon, J., Warren, Z., Sarkar, N.: Understanding How Adolescents with Autism Respond to Facial Expressions in Virtual Reality Environments. IEEE Transactions on Visualizations and Computer Graphics (2013) (to appear) (special issue)Google Scholar