Natural User Interface to Assess Social Skills in Autistic Population

  • Claudia FaitaEmail author
  • Raffaello Brondi
  • Camilla Tanca
  • Marcello Carrozzino
  • Massimo Bergamasco
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10325)


The children suffering of autism spectrum disorder (ASD) present several limitations in social communication and interactive skills. State of the art showed that Virtual Reality (VR) represents a valid alternative to learn social skills in ASD. In particular VR offers the advantage to simulate daily life interaction in a replicable and controlled environment. In this paper, we present a novel methodology to evaluate social competence in ASD based on the immersive VR system, featuring natural interaction. The VR system is based on the theoretical model which aims to evaluate the intersubjective interchange in everyday relationship between autistic population and the surrounding environment. We suggest that this technological setup can be a useful tool to increase social competence in ASD and it can become a sophisticated outcome measurement tool for studying the improvement of social competence.


Virtual reality Immersive virtual environment Natural interaction Autism spectrum disorder Social skills training 


  1. 1.
    Frith, U., Happé, F.: Autism spectrum disorder. Curr. Biol. 15(19), R786–R790 (2005)CrossRefGoogle Scholar
  2. 2.
    Wing, L.: The autistic spectrum. Lancet 350(9093), 1761 (1997)CrossRefGoogle Scholar
  3. 3.
    Chevallier, C., Parish-Morris, J., McVey, A., Rump, K.M., Sasson, N.J., Herrington, J.D., Schultz, R.T.: Measuring social attention and motivation in autism spectrum disorder using eye-tracking: stimulus type matters. Autism Res. 8(5), 620–628 (2015)CrossRefGoogle Scholar
  4. 4.
    Schilbach, L., Timmermans, B., Reddy, V., Costall, A., Bente, G., Schlicht, T., Vogeley, K.: Toward a second-person neuroscience. Behav. Brain Sci. 36(04), 393–414 (2013)CrossRefGoogle Scholar
  5. 5.
    Sebanz, N., Bekkering, H., Knoblich, G.: Joint action: bodies and minds moving together. Trends Cogn. Sci. 10(2), 70–76 (2006)CrossRefGoogle Scholar
  6. 6.
    Strickland, D.: Virtual reality for the treatment of autism. Stud. Health Technol. Inform. 44, 81–86 (1997)Google Scholar
  7. 7.
    Mineo, B.A., Ziegler, W., Gill, S., Salkin, D.: Engagement with electronic screen media among students with autism spectrum disorders. J. Autism Dev. Disord. 39(1), 172–187 (2009)CrossRefGoogle Scholar
  8. 8.
    Bölte, S., Golan, O., Goodwin, M.S., Zwaigenbaum, L.: What can innovative technologies do for autism spectrum disorders? Autism 14(3), 155–159 (2010)CrossRefGoogle Scholar
  9. 9.
    Brondi, R., Avveduto, G., Carrozzino, M., Tecchia, F., Alem, L., Bergamasco, M.: Immersive technologies and natural interaction to improve serious games engagement. In: De Gloria, A., Veltkamp, R. (eds.) GALA 2015. LNCS, vol. 9599, pp. 121–130. Springer, Cham (2016). doi: 10.1007/978-3-319-40216-1_13 Google Scholar
  10. 10.
    Brondi, R., Alem, L., Avveduto, G., Faita, C., Carrozzino, M., Tecchia, F., Bergamasco, M.: Evaluating the impact of highly immersive technologies and natural interaction on player engagement and flow experience in games. In: Chorianopoulos, K., Divitini, M., Hauge, J.B., Jaccheri, L., Malaka, R. (eds.) ICEC 2015. LNCS, vol. 9353, pp. 169–181. Springer, Cham (2015). doi: 10.1007/978-3-319-24589-8_13 CrossRefGoogle Scholar
  11. 11.
    Bellani, M., Fornasari, L., Chittaro, L., Brambilla, P.: Virtual reality in autism: state of the art. Epidemiol. Psychiatr. Sci. 20(3), 235–238 (2011)CrossRefGoogle Scholar
  12. 12.
    Strickland, D., Marcus, L.M., Mesibov, G.B., Hogan, K.: Brief report: two case studies using virtual reality as a learning tool for autistic children. J. Autism Dev. Disord. 26(6), 651–659 (1996)CrossRefGoogle Scholar
  13. 13.
    Moore, D., Cheng, Y., McGrath, P., Powell, N.J.: Collaborative virtual environment technology for people with autism. Focus Autism Dev. Disabil. 20(4), 231–243 (2005)CrossRefGoogle Scholar
  14. 14.
    Cheng, Y., Ye, J.: Exploring the social competence of students with autism spectrum conditions in a collaborative virtual learning environment-the pilot study. Comput. Educ. 54(4), 1068–1077 (2010)CrossRefGoogle Scholar
  15. 15.
    Adolphs, R.: Recognizing emotion from facial expressions: psychological and neurological mechanisms. Behav. Cogn. Neurosci. Reviews 1(1), 21–62 (2002)CrossRefGoogle Scholar
  16. 16.
    Tartaro, A., Cassell, J.: Playing with virtual peers: bootstrapping contingent discourse in children with autism. In: Proceedings of the 8th International Conference on International Conference for the Learning Sciences, International Society of the Learning Sciences, vol. 2, pp. 382–389 (2008)Google Scholar
  17. 17.
    Wallace, S., Parsons, S., Westbury, A., White, K., White, K., Bailey, A.: Sense of presence and atypical social judgments in immersive virtual environments responses of adolescents with autism spectrum disorders. Autism 14(3), 199–213 (2010)CrossRefGoogle Scholar
  18. 18.
    Fornasari, L., Chittaro, L., Ieronutti, L., Cottini, L., Dassi, S., Cremaschi, S., Molteni, M., Fabbro, F., Brambilla, P.: Navigation and exploration of an urban virtual environment by children with autism spectrum disorder compared to children with typical development. Res. Autism Spect. Disord. 7(8), 956–965 (2013)CrossRefGoogle Scholar
  19. 19.
    Lorenzo, G., Pomares, J., Lledó, A.: Inclusion of immersive virtual learning environments and visual control systems to support the learning of students with asperger syndrome. Comput. Educ. 62, 88–101 (2013)CrossRefGoogle Scholar
  20. 20.
    Cheng, Y., Huang, C.-L., Yang, C.-S.: Using a 3D immersive virtual environment system to enhance social understanding and social skills for children with autism spectrum disorders. Focus Autism Dev. Disabil. 30(4), 222–236 (2015)CrossRefGoogle Scholar
  21. 21.
    Brondi, R., Avveduto, G., Alem, L., Faita, C., Carrozzino, M., Tecchia, F., Pisan, Y., Bergamasco, M.: Evaluating the effects of competition vs collaboration on user engagement in an immersive game using natural interaction. In: Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology, p. 191. ACM (2015)Google Scholar
  22. 22.
    Trepagnier, C., Sebrechts, M., Finkelmeyer, A., Coleman, M., Stewart, W., Werner-Adler, M.: Virtual environments to address autistic social deficits. Annu. Rev. CyberTherapy Telemedicine Decade VR 3, 101–108 (2005)Google Scholar
  23. 23.
    Cheng, Y., Chiang, H.-C., Ye, J., Cheng, L.-H.: Enhancing empathy instruction using a collaborative virtual learning environment for children with autistic spectrum conditions. Comput. Educ. 55(4), 1449–1458 (2010)CrossRefGoogle Scholar
  24. 24.
    Miller, H.L., Bugnariu, N.L.: Level of immersion in virtual environments impacts the ability to assess and teach social skills in autism spectrum disorder. Cyberpsychology Behav. Soc. Networking 19(4), 246–256 (2016)CrossRefGoogle Scholar
  25. 25.
    Baron-Cohen, S., Leslie, A.M., Frith, U.: Does the autistic child have a “theory of mind”? Cognition 21(1), 37–46 (1985)CrossRefGoogle Scholar
  26. 26.
    Goldman, A.I., et al.: Theory of mind. In: Margolis, E., Samuels, R., Stich, S.P. (eds.) Oxford Handbook of Philosophy. Oxford University Press, UK (2012)Google Scholar
  27. 27.
    Gallagher, S.: Understanding interpersonal problems in autism: Interaction theory as an alternative to theory of mind. Philos. Psychiatry Psychol. 11(3), 199–217 (2004)CrossRefGoogle Scholar
  28. 28.
    Fuchs, T.: Pathologies of intersubjectivity in autism and schizophrenia. J. Conscious. Stud. 22(1–2), 191–214 (2015)Google Scholar
  29. 29.
    De la Torre, F., Hodgins, J., Bargteil, A., Martin, X., Macey, J., Collado, A., Beltran, P.: Guide to the Carnegie Mellon University multimodal activity (CMU-MMAC) database. Robotics Institute, p. 135 (2008)Google Scholar
  30. 30.
    Bouaziz, S., Wang, Y., Pauly, M.: Online modeling for realtime facial animation. ACM Trans. Graph. (TOG) 32(4), 40 (2013)CrossRefzbMATHGoogle Scholar
  31. 31.
    Association, A.P., et al.: Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). American Psychiatric Pub, Virginia (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Claudia Faita
    • 1
    Email author
  • Raffaello Brondi
    • 1
  • Camilla Tanca
    • 1
  • Marcello Carrozzino
    • 1
  • Massimo Bergamasco
    • 1
  1. 1.Institute of Communication, Information and Perception TechnologiesScuola Superiore Sant’Anna, Perceptual Robotics LaboratoryPisaItaly

Personalised recommendations