Connecting Through Kinect: Designing and Evaluating a Collaborative Game with and for Autistic Individuals

  • Kristen GillespieEmail author
  • Gabriel Goldstein
  • David Shane Smith
  • Ariana Riccio
  • Michael Kholodovsky
  • Cali Merendino
  • Stanislav Leskov
  • Rayan Arab
  • Hassan Elsherbini
  • Pavel Asanov
  • Deborah Sturm
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10289)


We are developing a game to help autistic people collaborate with their siblings and peers while improving their social-communicative skills. Autistic college students have been involved in game design and evaluation since the project’s inception. They have provided invaluable feedback that we are incorporating with their help. By involving autistic students in game design and evaluation, we are helping them develop employment-readiness skills while ensuring that the game is well designed to teach autistic people.


Autism Participatory research Game design Collaboration Emotion recognition Autistic college students 



This project has been supported by a CUNY Interdisciplinary Research Grant (#2316) awarded to Kristen Gillespie-Lynch and Deborah Sturm, a PSC-CUNY Award (RF-CUNY award # 67108-00 45), jointly funded by the Professional Staff Congress and the City University of New York, awarded to Deborah Sturm and a grant awarded by the FAR Fund to Kristen Gillespie-Lynch to support Project REACH, the mentorship program we are recruiting autistic collaborators from. We would like to thank Patricia J. Brooks, Bertram Ploog, Christina Shane-Simpson, Rita Obeid, Danielle DeNigris, Jonathan Pickens, and mentors and mentees from Building Bridges Project REACH for initial feedback on study design.


  1. 1.
    American Psychiatric Association APA: Diagnostic and statistical manual of mental disorders, 5 (2013)Google Scholar
  2. 2.
    Mottron, L., Dawson, M., Soulieres, I., Hubert, B., Burack, J.: Enhanced perceptual functioning in autism: an update, and eight principles of autistic perception. J. Autism Dev. Disord. 36(1), 27–43 (2006)CrossRefGoogle Scholar
  3. 3.
    Sodian, B., Frith, U.: Deception and sabotage in autistic, retarded and normal children. J. Child Psychol. Psychiatry 33(3), 591–605 (1992)CrossRefGoogle Scholar
  4. 4.
    Baron-Cohen, S., Ashwin, E., Tavassoli, T., Chakrabarti, B.: Talent in autism: hyper-systemizing, hyper-attention to detail and sensory hypersensitivity. Philos. Trans. R. Soc. London B Biol. Sci. 364(1522), 1377–1383 (2009)CrossRefGoogle Scholar
  5. 5.
    Gillespie-Lynch, K., Kapp, S.K., Shane-Simpson, C., Smith, D.S., Hutman, T.: Intersections between the autism spectrum and the internet: perceived benefits and preferred functions of computer-mediated communication. Intell. Dev. Disabil. 52(6), 456–469 (2014)CrossRefGoogle Scholar
  6. 6.
    Murray, D., Lesser, M.: Autism and computing. In: Autism 99 Online Conference Organised by the NAS with the Shirley Foundation (1999)Google Scholar
  7. 7.
    Wei, X., Jennifer, W.Y., Shattuck, P., McCracken, M., Blackorby, J.: Science, technology, engineering, and mathematics (STEM) participation among college students with an autism spectrum disorder. J. Autism Dev. Disord. 43(7), 1539–1546 (2013)CrossRefGoogle Scholar
  8. 8.
    Burgess, S., Cimera, R.E.: Employment outcomes of transition-aged adults with autism spectrum disorders: a state of the states report. Am. J. Intell. Dev. Disabil. 119(1), 64–83 (2014)CrossRefGoogle Scholar
  9. 9.
    Baldwin, S., Costley, D., Warren, A.: Employment activities and experiences of adults with high-functioning autism and Asperger’s disorder. J. Autism Dev. Disord. 44(10), 2440–2449 (2014)CrossRefGoogle Scholar
  10. 10.
    Bublitz, D., Fitzgerald, K., Alarcon, M., D’Onofrio, J., Gillespie-Lynch, K.: Verbal behaviors during employment interviews of college students with and without ASD. J. Vocat. Rehabil. (In press)Google Scholar
  11. 11.
    Müller, E., Schuler, A., Yates, G.B.: Social challenges and supports from the perspective of individuals with Asperger syndrome and other autism spectrum disabilities. Autism 12(2), 173–190 (2008)CrossRefGoogle Scholar
  12. 12.
    Moore, M., Calvert, S.: Brief report: vocabulary acquisition for children with autism: teacher or computer instruction. J. Autism Dev. Disord. 30(4), 359–362 (2000)CrossRefGoogle Scholar
  13. 13.
    Fletcher-Watson, S.: A targeted review of computer-assisted learning for people with autism spectrum disorder: towards a consistent methodology. Rev. J. Autism Dev. Disord. 1(2), 87–100 (2014)CrossRefGoogle Scholar
  14. 14.
    Whyte, E.M., Smyth, J.M., Scherf, K.S.: Designing serious game interventions for individuals with Autism. J. Autism Dev. Disord. 45(12), 3820–3831 (2015)CrossRefGoogle Scholar
  15. 15.
    Harris, S.L.: Teaching language to nonverbal children-with emphasis on problems of generalization. Psychol. Bull. 82(4), 565 (1975)CrossRefGoogle Scholar
  16. 16.
    Gillespie-Lynch, K., Brooks, P., Shane-Simpson, C., Gaggi, N.L., Sturm, D., Ploog, B.O.: Selecting computer-mediated interventions to support the social and emotional development of individuals with autism spectrum disorder. In: Silton, N. (ed.) Recent Advances in Assistive Technologies to Support Children with Developmental Disorders (2016)Google Scholar
  17. 17.
    Golan, O., Baron-Cohen, S.: Systemizing empathy: teaching adults with Asperger syndrome or high-functioning autism to recognize complex emotions using interactive multimedia. Dev. Psychopathol. 18(02), 591–617 (2006)CrossRefGoogle Scholar
  18. 18.
    Hopkins, I.M., Gower, M.W., Perez, T.A., Smith, D.S., Amthor, F.R., Wimsatt, F.C., Biasini, F.J.: Avatar assistant: improving social skills in students with an ASD through a computer-based intervention. J. Autism Dev. Disord. 41(11), 1543–1555 (2011)CrossRefGoogle Scholar
  19. 19.
    Benton, L., Johnson, H., Ashwin, E., Brosnan, M., Grawemeyer, B.: Developing IDEAS: supporting children with autism within a participatory design team. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2599–2608. ACM, May 2012Google Scholar
  20. 20.
    Parsons, S., Cobb, S.: Reflections on the role of the ‘users’: challenges in a multi-disciplinary context of learner-centred design for children on the autism spectrum. Int. J. Res. Method Educ. 37(4), 421–441 (2014)CrossRefGoogle Scholar
  21. 21.
    Jivraj, J., Sacrey, L.A., Newton, A., Nicholas, D., Zwaigenbaum, L.: Assessing the influence of researcher–partner involvement on the process and outcomes of participatory research in autism spectrum disorder and neurodevelopmental disorders: a scoping review. Autism 18(7), 782–793 (2014)CrossRefGoogle Scholar
  22. 22.
    Nicolaidis, C., Raymaker, D., McDonald, K., Dern, S., Ashkenazy, E., Boisclair, C., Baggs, A.: Collaboration strategies in nontraditional community-based participatory research partnerships: lessons from an academic–community partnership with autistic self-advocates. Prog. Community Health Partnerships 5(2), 143 (2011)CrossRefGoogle Scholar
  23. 23.
    Brosnan, M., Parsons, S., Good, J., Yuill, N.: How can participatory design inform the design and development of innovative technologies for autistic communities? J. Assistive Technol. 10(2), 115–120 (2016)CrossRefGoogle Scholar
  24. 24.
    Benton, L., Johnson, H., Brosnan, M., Ashwin, E., Grawemeyer, B.: IDEAS: an interface design experience for the autistic spectrum. In: Extended Abstracts CHI 2011. ACM (2011)Google Scholar
  25. 25.
    Benton, L., Vasalou, A., Khaled, R., Johnson, H., Gooch, D.: Diversity for design: a framework for involving neurodiverse children in the technology design process. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, pp. 3747–3756. ACM, April 2014Google Scholar
  26. 26.
    Bossavit, B., Parsons, S.: Designing an educational game for and with teenagers with high functioning autism. In: Proceedings of the 14th Participatory Design Conference: Full papers - Volume 1 (PDC 2016), vol. 1. ACM, New York, NY, USA, pp. 11–20 (2016). doi:
  27. 27.
    Cobb, S., Beardon, L., Eastgate, R., Glover, T., Kerr, S., Neale, H., Reynard, G.: Applied virtual environments to support learning of social interaction skills in users with Asperger’s Syndrome. Digital Creativity 13(1), 11–22 (2002)CrossRefGoogle Scholar
  28. 28.
    Frauenberger, C., Good, J., Alcorn, A., Pain, H.: Conversing through and about technologies: Design critique as an opportunity to engage children with autism and broaden research (er) perspectives. Int. J. Child-Comput. Interact. 1(2), 38–49 (2013)CrossRefGoogle Scholar
  29. 29.
    Keay-Bright, W.: The reactive colours project: demonstrating participatory and collaborative design methods for the creation of software for autistic children (2007)Google Scholar
  30. 30.
    Millen, L., Cobb, S., Patel, H.: A method for involving children with autism in design. In: Proceedings of the 10th International Conference on Interaction Design and Children, pp. 185–188. ACM, June 2011Google Scholar
  31. 31.
    Neale, H.: Using virtual reality to teach social skills to people with Asperger’s syndrome: explaining virtual reality and user-centred methodology. In: Proceedings of the 12th Annual Durham International Conference on Autism, April 2001Google Scholar
  32. 32.
    Piper, A.M., O’Brien, E., Morris, M.R., Winograd, T.: SIDES: a cooperative tabletop computer game for social skills development. In: Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work, pp. 1–10. ACM, November 2006Google Scholar
  33. 33.
    Mesibov, G.B., Shea, V.: The TEACCH program in the era of evidence-based practice. J. Autism Dev. Disord. 40(5), 570–579 (2010)CrossRefGoogle Scholar
  34. 34.
    Capps, L., Yirmiya, N., Sigman, M.: Understanding of simple and complex emotions in non-retarded children with Autism. J. Child Psychol. Psychiatry 33(7), 1169–1182 (1992)CrossRefGoogle Scholar
  35. 35.
    Levinson, L.J., Reid, G.: The effects of exercise intensity on the stereotypic behaviours of individuals with autism. McGill University (1992)Google Scholar
  36. 36.
    Grelotti, D.J., Klin, A.J., Gauthier, I., Skudlarski, P., Cohen, D.J., Gore, J.C., Schultz, R.T.: fMRI activation of the fusiform gyrus and amygdala to cartoon characters but not to faces in a boy with autism. Neuropsychologia 43(3), 373–385 (2005)CrossRefGoogle Scholar
  37. 37.
    Gee, J.P.: Learning by design: games as learning machines. Interact. Educ. Multimedia 8, 15–23 (2010)Google Scholar
  38. 38.
    Holt, S., Yuill, N.: Facilitating other-awareness in low-functioning children with autism and typically-developing preschoolers using dual-control technology. J. Autism Dev. Disord. 44(1), 236–248 (2014)CrossRefGoogle Scholar
  39. 39.
    Jasmin, E., Couture, M., McKinley, P., Reid, G., Fombonne, E., Gisel, E.: Sensori-motor and daily living skills of preschool children with autism spectrum disorders. J. Autism Dev. Disord. 39(2), 231–241 (2009)CrossRefGoogle Scholar
  40. 40.
    Must, A., Phillips, S.M., Curtin, C., Anderson, S.E., Maslin, M., Lividini, K., Bandini, L.G.: Comparison of sedentary behaviors between children with autism spectrum disorders and typically developing children. Autism 18(4), 376–384 (2013). doi: 10.1177/1362361313479039 CrossRefGoogle Scholar
  41. 41.
    Holt, S., Yuill, N.: Tablets for two: how dual tablets can facilitate other-awareness and communication in learning disabled children with autism. Int. J. Child-Comput. Interact. (2016)Google Scholar
  42. 42.
    Fine, M., Torre, M.E., Burns, A., Payne, Y.A.: Youth research/participatory methods for reform. In: Thiessen, D., Cook-Sather, A. (eds.) International Handbook of Student Experience in Elementary and Secondary School, pp. 805–828. Springer, Dordrecht (2007)CrossRefGoogle Scholar
  43. 43.
    Schilbach, L., Timmermans, B., Reddy, V., Costall, A., Bente, G., Schlicht, T., Vogeley, K.: Toward a second-person neuroscience. Behav. Brain Sci. 36(04), 393–414 (2013)CrossRefGoogle Scholar
  44. 44.
    Tsang, T., Gillespie-Lynch, K., Hutman, T.: Theory of mind indexes the broader autism phenotype in siblings of children with autism at school age. Autism Res. Treat. (2016)Google Scholar
  45. 45.
    Jones, R.S., Huws, J.C., Beck, G.: ‘I’m not the only person out there’: insider and outsider understandings of autism. Int. J. Dev. Disabil. 59(2), 134–144 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Kristen Gillespie
    • 1
    Email author
  • Gabriel Goldstein
    • 2
  • David Shane Smith
    • 1
  • Ariana Riccio
    • 1
  • Michael Kholodovsky
    • 2
  • Cali Merendino
    • 2
  • Stanislav Leskov
    • 2
  • Rayan Arab
    • 1
  • Hassan Elsherbini
    • 2
  • Pavel Asanov
    • 1
    • 2
  • Deborah Sturm
    • 2
  1. 1.Department of PsychologyCSI/CUNYNew YorkUSA
  2. 2.Department of Computer ScienceCSI/CUNYStaten IslandUSA

Personalised recommendations