Abstract
Technology-assisted intervention has the potential to adaptively individualize and improve outcomes of traditional schizophrenia (SZ) intervention. Virtual reality (VR) technology, in particular, has the potential to simulate real world social and communication interactions and hence could be useful as a therapeutic platform for SZ. Emotional face recognition is considered among the core building blocks of social communication. Studies have shown that emotional face processing and understanding is impaired in patients with SZ. The current study develops a novel VR-based system that presents avatars that can change their facial emotion dynamically for emotion recognition tasks. Additionally, this system allows real-time measurement of physiological signals and eye gaze during the emotion recognition tasks, which can be used to gain insight about the emotion recognition process in SZ population. This study further compares VR-based facial emotion recognition with that of the more traditional emotion recognition from static faces using a small usability study. Results from the usability study suggest that VR could be a viable platform for SZ intervention and implicit signals such as physiological signals and eye gaze can be utilized to better understand the underlying pattern that is not available from user reports and performance alone.
Chapter PDF
Similar content being viewed by others
Keywords
References
Hempel, R.J., Tulen, J.H., van Beveren, N.J., van Steenis, H.G., Mulder, P.G., Hengeveld, M.W.: Physiological responsivity to emotional pictures in schizophrenia. Journal of Psychiatric Research 39(5), 509–518 (2005)
Bleuler, E.: Dementia praecox or the group of schizophrenias. International University Press, New York (1911)
Andreasen, N.C.: Scale for the assessment of negative symptoms. University of Iowa, Iowa City (1983)
Couture, S.M., Granholm, E.L., Fish, S.C.: A path model investigation of neurocognition, theory of mind, social competence, negative symptoms and real-world functioning in schizophrenia. Schizophrenia Research 125(2), 152–160 (2011)
Herbener, E.S., Song, W., Khine, T.T., Sweeney, J.A.: What aspects of emotional functioning are impaired in schizophrenia? Schizophrenia Research 98(1), 239–246 (2008)
Kring, A.M., Moran, E.K.: Emotional response deficits in schizophrenia: Insights from affective science. Schizophrenia Bulletin 34(5), 819–834 (2008)
Freeman, D.: Studying and treating schizophrenia using virtual reality: A new paradigm. Schizophrenia Bulletin 34(4), 605–610 (2008)
Kurtz, M.M., Baker, E., Pearlson, G.D., Astur, R.S.: A virtual reality apartment as a measure of medication management skills in patients with schizophrenia: A pilot study. Schizophrenia Bulletin 33(5), 1162–1170 (2007)
Yellowlees, P., Cook, J.: Education about hallucinations using an internet virtual reality system: A qualitative survey. Academic Psychiatry 30(6), 534–539 (2006)
Kim, K., Kim, J.-J., Kim, J., Park, D.-E., Jang, H.J., Ku, J., Kim, C.-H., Kim, I.Y., Kim, S.I.: Characteristics of social perception assessed in schizophrenia using virtual reality. Cyberpsychology & Behavior 10(2), 215–219 (2006)
Park, K.-M., Ku, J., Choi, S.-H., Jang, H.-J., Park, J.-Y., Kim, S.I., Kim, J.-J.: A virtual reality application in role-plays of social skills training for schizophrenia: A randomized, controlled trial. Psychiatry Research 189(2), 166–172 (2011)
Sorkin, A., Weinshall, D., Modai, I., Peled, A.: Improving the accuracy of the diagnosis of schizophrenia by means of virtual reality. American Journal of Psychiatry 163(3), 512–520 (2006)
Liu, C., Conn, K., Sarkar, N., Stone, W.: Physiology-based affect recognition for computer-assisted intervention of children with Autism Spectrum Disorder. International Journal of Human-Computer Studies 66(9), 662–677 (2008)
Liu, C., Conn, K., Sarkar, N., Stone, W.: Online affect detection and robot behavior adaptation for intervention of children with autism. IEEE Transactions on Robotics 24(4), 883–896 (2008)
Andreasen, N.C.: Scale for the assessment of positive symptoms. University of Iowa, Iowa City (1984)
Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (IAPS): Technical manual and affective ratings. Gainesville, FL: The Center for Research in Psychophysiology, University of Florida (1999)
Bekele, E., Zheng, Z., Swanson, A., Crittendon, J., Warren, Z., Sarkar, N.: Understanding How Adolescents with Autism Respond to Facial Expressions in Virtual Reality Environments. IEEE Transactions on Visualization and Computer Graphics 19(4), 711–720 (2013)
Lahiri, U., Warren, Z., Sarkar, N.: Design of a Gaze-Sensitive Virtual Social Interactive System for Children With Autism. IEEE Transactions on Neural Systems and Rehabilitation Engineering (99), 1 (2012)
Cacioppo, J.T., Tassinary, L.G., Berntson, G.G.: Handbook of psychophysiology. Cambridge Univ. Pr. (2007)
Welch, K., Lahiri, U., Liu, C., Weller, R., Sarkar, N., Warren, Z.: An Affect-Sensitive Social Interaction Paradigm Utilizing Virtual Reality Environments for Autism Intervention. Paper presented at the Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction (2009)
Bekele, E., Zheng, Z., Swanson, A., Crittendon, J., Warren, Z., Sarkar, N.: Understanding How Adolescents with Autism Respond to Facial Expressions in Virtual Reality Environments. IEEE Transactions on Visualizations and Computer Graphics (2013) (to appear) (special issue)
Blair, J.R., Spreen, O.: Predicting premorbid IQ: A revision of the National Adult Reading Test. The Clinical Neuropsychologist 3(2), 129–136 (1989)
Andreasen, N.C., Pressler, M., Nopoulos, P., Miller, D., Ho, B.-C.: Antipsychotic dose equivalents and dose-years: A standardized method for comparing exposure to different drugs. Biological Psychiatry 67(3), 255–262 (2010)
Overall, J.E., Gorham, D.R.: The brief psychiatric rating scale. Psychological Reports 10(3), 799–812 (1962)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Bekele, E., Bian, D., Zheng, Z., Peterman, J., Park, S., Sarkar, N. (2014). Responses during Facial Emotional Expression Recognition Tasks Using Virtual Reality and Static IAPS Pictures for Adults with Schizophrenia. In: Shumaker, R., Lackey, S. (eds) Virtual, Augmented and Mixed Reality. Applications of Virtual and Augmented Reality. VAMR 2014. Lecture Notes in Computer Science, vol 8526. Springer, Cham. https://doi.org/10.1007/978-3-319-07464-1_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-07464-1_21
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07463-4
Online ISBN: 978-3-319-07464-1
eBook Packages: Computer ScienceComputer Science (R0)