Journal of Autism and Developmental Disorders

, Volume 49, Issue 10, pp 4009–4018 | Cite as

Atypical Visual Processing but Comparable Levels of Emotion Recognition in Adults with Autism During the Processing of Social Scenes

  • Julia S. Y. TangEmail author
  • Nigel T. M. Chen
  • Marita Falkmer
  • Sven Bӧlte
  • Sonya Girdler
Original Paper


Understanding the underlying visual scanning patterns of individuals with autism spectrum disorder (ASD) during the processing of complex emotional scenes remains limited. This study compared the complex emotion recognition performance of adults with ASD (n = 23) and matched neurotypical participants (n = 25) using the Reading the Mind in Films Task. Behaviourally, both groups exhibited similar emotion recognition accuracy. Visual fixation time towards key social regions of each stimuli was examined via eye tracking. Individuals with ASD demonstrated significantly longer fixation time towards the non-social areas. No group differences were evident for the facial and body regions of all characters in the social scenes. The findings provide evidence of the heterogeneity associated with complex emotion processing in individuals with ASD.


Autism Dynamic stimuli Eye tracking Naturalistic Social cognition 



The researchers would like to extend our gratitude to the participants for the time and effort taken to participate in this study as well as Melissa Black, who assisted in data collection.

Author Contributions

JT conceptualised and designed the study, involved in data collection, analysed and interpreted the data, and contributed to drafting the article. NC conceptualised and designed the study, analysed and interpreted the data and critically revised the draft. MF conceptualised and designed the study, interpreted the data, and revised the draft. SB revised and critically contributed to the intellectual content of the article. SG conceptualised and designed the study, interpreted the data and critically revised the article. All authors contributed to and have approved the final manuscript.


This research is supported by the Cooperative Research Centre for Living with Autism (Autism CRC) [Project Number 3.032RS], established and supported under the Australian Government’s Cooperative Research Centres Program. JT is supported through an Australian Government Research Training Program Scholarship.

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest. SB reports no direct conflict of interest related to this article. SB discloses that he has in the last 5 years acted as an author, consultant or lecturer for Shire, Medice, Roche, Eli Lilly, Prima Psychiatry, GLGroup, System Analytic, Kompetento, Expo Medica, and Prophase. He receives royalties for text books and diagnostic tools from Huber/Hogrefe, Kohlhammer and UTB.


  1. Adolphs, R. (2001). The neurobiology of social cognition. Current Opinion in Neurobiology, 11(2), 231–239. Scholar
  2. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders. Washington, DC: American Psychiatric Association.Google Scholar
  3. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: American Psychiatric Association.CrossRefGoogle Scholar
  4. Ashwin, C., Chapman, E., Colle, L., & Baron-Cohen, S. (2006). Impaired recognition of negative basic emotions in autism: A test of the amygdala theory. Social Neuroscience, 1(3–4), 349–363. Scholar
  5. Baron-Cohen, S. (2004). The cognitive neuroscience of autism. Journal of Neurology, Neurosurgery and Psychiatry, 75(7), 945–948. Scholar
  6. Berggren, S., Engstrom, A. C., & Bölte, S. (2016). Facial affect recognition in autism, ADHD and typical development. Cognitive Neuropsychiatry, 21(3), 213–227. Scholar
  7. Bird, G., Press, C., & Richardson, D. C. (2011). The role of alexithymia in reduced eye-fixation in autism spectrum conditions. Journal of Autism and Developmental Disorders, 41(11), 1556–1564. Scholar
  8. Black, M. H., Chen, N. T. M., Iyer, K. K., Lipp, O. V., Bölte, S., Falkmer, M., et al. (2017). Mechanisms of facial emotion recognition in autism spectrum disorders: Insights from eye tracking and electroencephalography. Neuroscience and Biobehavioral Reviews, 80, 488–515. Scholar
  9. Bölte, S., Poustka, F., & Constantino, J. N. (2008). Assessing autistic traits: Cross-cultural validation of the Social Responsiveness Scale (SRS). Autism Research, 1(6), 354–363. Scholar
  10. Borod, J. C., Pick, L. H., Hall, S., Sliwinski, M., Madigan, N., Obler, L. K., et al. (2000). Relationships among facial, prosodic, and lexical channels of emotional perceptual processing. Cognition and Emotion, 14(2), 193–211. Scholar
  11. Bruni, T. P. (2014). Test review: Social responsiveness scale-second edition (SRS-2). Journal of Psychoeducational Assessment, 32(4), 365–369. Scholar
  12. Bӧlte, S., & Poustka, F. (2003). The recognition of facial affect in autistic and schizophrenic subjects and their first-degree relatives. Psychological Medicine, 33(5), 907–915. Scholar
  13. Cassidy, S., Mitchell, P., Chapman, P., & Ropar, D. (2015). Processing of spontaneous emotional responses in adolescents and adults with autism spectrum disorders: Effect of stimulus type. Autism Research, 8(5), 534–544. Scholar
  14. Castelli, F. (2005). Understanding emotions from standardized facial expressions in autism and normal development. Autism, 9(4), 428–449. Scholar
  15. Chevallier, C., Parish-Morris, J., McVey, A., Rump, K. M., Sasson, N. J., Herrington, J. D., et al. (2015). Measuring social attention and motivation in autism spectrum disorder using eye-tracking: Stimulus type matters. Autism Research, 8(5), 620–628. Scholar
  16. Chita-Tegmark, M. (2016). Social attention in ASD: A review and meta-analysis of eye-tracking studies. Research in Developmental Disabilities, 48, 79–93. Scholar
  17. Clark, T. F., Winkielman, P., & McIntosh, D. N. (2008). Autism and the extraction of emotion from briefly presented facial expressions: Stumbling at the first step of empathy. Emotion, 8(6), 803–809. Scholar
  18. Constantino, J. N., & Gruber, C. P. (2011). Social Responsiveness Scale, second edition (SRS-2). Torrance: Western Psychological Services.Google Scholar
  19. Durand, K., Gallay, M., Seigneuric, A., Robichon, F., & Baudouin, J.-Y. (2007). The development of facial emotion recognition: The role of configural information. Journal of Experimental Child Psychology, 97(1), 14–27. Scholar
  20. Eack, S. M., Mazefsky, C. A., & Minshew, N. J. (2015). Misinterpretation of facial expressions of emotion in verbal adults with autism spectrum disorder. Autism, 19(3), 308–315. Scholar
  21. Falkmer, M., Bjällmark, A., Larsson, M., & Falkmer, T. (2011). Recognition of facially expressed emotions and visual search strategies in adults with Asperger syndrome. Research in Autism Spectrum Disorders, 5(1), 210–217. Scholar
  22. Falkmer, T., Dahlman, J., Dukic, T., Bjällmark, A., & Larsson, M. (2008). Fixation identification in centroid versus start-point modes using eye-tracking data. Perceptual and Motor Skills, 106(3), 710–724. Scholar
  23. Fridenson-Hayo, S., Berggren, S., Lassalle, A., Tal, S., Pigat, D., Bölte, S., et al. (2016). Basic and complex emotion recognition in children with autism: Cross-cultural findings. Molecular Autism, 7(1), 52. Scholar
  24. Frith, U., & Blakemore, S. (2006). Social cognition. In L. Tarassenko & M. Kenward (Eds.), Cognitive systems—Information processing meets brain science (pp. 138–162). London: Academic Press.CrossRefGoogle Scholar
  25. Gepner, B., Deruelle, C., & Grynfeltt, S. (2001). Motion and emotion: A novel approach to the study of face processing by young autistic children. Journal of Autism and Developmental Disorders, 31(1), 37–45. Scholar
  26. Golan, O., Baron-Cohen, S., & Golan, Y. (2008). The ‘Reading the Mind in Films’ task [child version]: Complex emotion and mental state recognition in children with and without autism spectrum conditions. Journal of Autism and Developmental Disorders, 38(8), 1534–1541. Scholar
  27. Golan, O., Baron-Cohen, S., & Hill, J. (2006a). The Cambridge Mindreading (CAM) Face-Voice Battery: Testing complex emotion recognition in adults with and without Asperger syndrome. Journal of Autism and Developmental Disorders, 36(2), 169–183. Scholar
  28. Golan, O., Baron-Cohen, S., Hill, J. J., & Golan, Y. (2006b). The “Reading the Mind in Films” task: Complex emotion recognition in adults with and without autism spectrum conditions. Social Neuroscience, 1(2), 111–123. Scholar
  29. Griffiths, S., Jarrold, C., Penton-Voak, I. S., Woods, A. T., Skinner, A. L., & Munafò, (2017). Impaired recognition of basic emotions from facial expressions in young people with autism spectrum disorder: Assessing the importance of expression intensity. Journal of Autism and Developmental Disorders. Scholar
  30. Guillon, Q., Hadjikhani, N., Baduel, S., & Rogé, B. (2014). Visual social attention in autism spectrum disorder: Insights from eye tracking studies. Neuroscience and Biobehavioral Reviews, 42, 279–297. Scholar
  31. Harms, M. B., Martin, A., & Wallace, G. L. (2010). Facial emotion recognition in autism spectrum disorders: A review of behavioral and neuroimaging studies. Neuropsychology Review, 20(3), 290–322. Scholar
  32. Hillier, A., & Allinson, L. (2002). Understanding embarrassment among those with autism: Breaking down the complex emotion of embarrassment among those with autism. Journal of Autism and Developmental Disorders, 32(6), 583–592.CrossRefPubMedGoogle Scholar
  33. Joosten, A., Girdler, S., Albrecht, M. A., Horlin, C., Falkmer, M., Leung, D., et al. (2016). Gaze and visual search strategies of children with Asperger syndrome/high functioning autism viewing a magic trick. Developmental Neurorehabilitation, 19(2), 95–102. Scholar
  34. Kliemann, D., Rosenblau, G., Bӧlte, S., Heekeren, H., & Dziobek, I. (2013). Face puzzle—Two new video-based tasks for measuring explicit and implicit aspects of facial emotion recognition. Frontiers in Psychology, 4, 376.CrossRefPubMedPubMedCentralGoogle Scholar
  35. Klin, A. (2000). Attributing social meaning to ambiguous visual stimuli in higher-functioning autism and Asperger syndrome: The social attribution task. Journal of Child Psychology and Psychiatry, 41(7), 831–846.CrossRefPubMedGoogle Scholar
  36. Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002). Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry, 59(9), 809–816.CrossRefPubMedGoogle Scholar
  37. Kuhn, G., Kourkoulou, A., & Leekam, S. R. (2010). How magic changes our expectations about autism. Psychological Science, 21(10), 1487–1493. Scholar
  38. Leung, D., Ordqvist, A., Falkmer, T., Parsons, R., & Falkmer, M. (2013). Facial emotion recognition and visual search strategies of children with high functioning autism and Asperger syndrome. Research in Autism Spectrum Disorders, 7(7), 833–844. Scholar
  39. Lozier, L. M., Vanmeter, J. W., & Marsh, A. A. (2014). Impairments in facial affect recognition associated with autism spectrum disorders: A meta-analysis. Development and Psychopathology, 26(4), 933–945. Scholar
  40. Magnée, M. J. C. M., de Gelder, B., van Engeland, H., & Kemner, C. (2011). Multisensory integration and attention in autism spectrum disorder: Evidence from event-related potentials. PLoS ONE, 6(8), e24196. Scholar
  41. Mathersul, D., McDonald, S., & Rushby, J. A. (2013). Understanding advanced theory of mind and empathy in high-functioning adults with autism spectrum disorder. Journal of Clinical and Experimental Neuropsychology, 35(6), 655–668. Scholar
  42. McCabe, K. L., Melville, J. L., Rich, D., Strutt, P. A., Cooper, G., Loughland, C. M., et al. (2013). Divergent patterns of social cognition performance in autism and 22q11.2 deletion syndrome (22q11DS). Journal of Autism and Developmental Disorders, 43(8), 1926–1934.CrossRefPubMedGoogle Scholar
  43. McCrimmon, A. W., & Smith, A. D. (2012). Review of the Wechsler Abbreviated Scale of Intelligence, second edition (WASI-II). Journal of Psychoeducational Assessment, 31(3), 337–341.CrossRefGoogle Scholar
  44. Mondloch, C. J., Geldart, S., Maurer, D., & Grand, R. L. (2003). Developmental changes in face processing skills. Journal of Experimental Child Psychology, 86(1), 67–84. Scholar
  45. Müller, N., Baumeister, S., Dziobek, I., Banaschewski, T., & Poustka, L. (2016). Validation of the Movie for the Assessment of Social Cognition in adolescents with ASD: Fixation duration and pupil dilation as predictors of performance. Journal of Autism and Developmental Disorders, 46(9), 2831–2844. Scholar
  46. Murray, K., Johnston, K., Cunnane, H., Kerr, C., Spain, D., Gillan, N., et al. (2017). A new test of advanced theory of mind: The “Strange Stories Film Task” captures social processing differences in adults with autism spectrum disorders. Autism Research, 10(6), 1120–1132. Scholar
  47. Nakano, T., Tanaka, K., Endo, Y., Yamane, Y., Yamamoto, T., Nakano, Y., et al. (2010). Atypical gaze patterns in children and adults with autism spectrum disorders dissociated from developmental changes in gaze behaviour. Proceedings Biological Sciences, 277(1696), 2935–2943. Scholar
  48. Nuske, H. J., Vivanti, G., & Dissanayake, C. (2013). Are emotion impairments unique to, universal, or specific in autism spectrum disorder? A comprehensive review. Cognition and Emotion, 27(6), 1042–1061. Scholar
  49. Pelphrey, K. A., Morris, J. P., McCarthy, G., & LaBar, K. S. (2007). Perception of dynamic changes in facial affect and identity in autism. Social Cognitive and Affective Neuroscience, 2(2), 140–149.CrossRefPubMedPubMedCentralGoogle Scholar
  50. Peterson, C. C., Slaughter, V., & Brownell, C. (2015). Children with autism spectrum disorder are skilled at reading emotion body language. Journal of Experimental Child Psychology, 139, 35–50. Scholar
  51. Rice, K., Moriuchi, J. M., Jones, W., & Klin, A. (2012). Parsing heterogeneity in autism spectrum disorders: Visual scanning of dynamic social scenes in school-aged children. Journal of the American Academy of Child and Adolescent Psychiatry, 51(3), 238–248. Scholar
  52. Rosenblau, G., Kliemann, D., Heekeren, H. R., & Dziobek, I. (2015). Approximating implicit and explicit mentalizing with two naturalistic video-based tasks in typical development and autism spectrum disorder. Journal of Autism and Developmental Disorders, 45(4), 953–965. Scholar
  53. Sato, W., Sawada, R., Uono, S., Yoshimura, S., Kochiyama, T., Kubota, Y., et al. (2017). Impaired detection of happy facial expressions in autism. Scientific Reports, 7(13340), 1–12. Scholar
  54. Senju, A. (2013). Atypical development of spontaneous social cognition in autism spectrum disorders. Brain and Development, 35(2), 96–101. Scholar
  55. Spezio, M. L., Adolphs, R., Hurley, R. S. E., & Piven, J. (2007). Abnormal use of facial information in high-functioning autism. Journal of Autism and Developmental Disorders, 37(5), 929–939. Scholar
  56. Tardif, C., Lainé, F., Rodriguez, M., & Gepner, B. (2007). Slowing down presentation of facial movements and vocal sounds enhances facial expression recognition and induces facial–vocal imitation in children with autism. Journal of Autism and Developmental Disorders, 37(8), 1469–1484. Scholar
  57. Tracy, J. L., Robins, R. W., Schriber, R. A., & Solomon, M. (2011). Is emotion recognition impaired in individuals with autism spectrum disorders? Journal of Autism and Developmental Disorders, 41(1), 102–109. Scholar
  58. Uljarevic, M., & Hamilton, A. (2013). Recognition of emotions in autism: A formal meta-analysis. Journal of Autism and Developmental Disorders, 43(7), 1517–1526. Scholar
  59. Walker-Andrews, A. S. (1998). Emotions and social development: Infants’ recognition of emotions in others. Pediatrics 102(5), 1268–1271.PubMedGoogle Scholar
  60. Wechsler, D. (2011). Wechsler Abbreviated Scale of Intelligence-second edition (WASI-II). San Antonio: NCS Pearson.Google Scholar
  61. Whitehouse, A. J. O., Evans, K., Eapen, V., Prior, M., & Wray, J. (2018). A national guideline for the assessment and diagnosis of autism spectrum disorders in Australia. Cooperative Research Centre for Living with Autism (Autism CRC). Retrieved from
  62. Williams, B. T., & Gray, K. M. (2013). The relationship between emotion recognition ability and social skills in young children with autism. Autism, 17(6), 762–768. Scholar
  63. Young-Browne, G., Rosenfeld, H. M., & Horowitz, F. D. (1977). Infant discrimination of facial expressions. Child Development, 48(2), 555–562.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Occupational Therapy, Social Work and Speech PathologyCurtin UniversityBentley, PerthAustralia
  2. 2.Cooperative Research Centre for Living with Autism (Autism CRC)IndooroopillyAustralia
  3. 3.School of Psychological ScienceUniversity of Western AustraliaPerthAustralia
  4. 4.School of Education and CommunicationCHILD, Swedish Institute of Disability Research, Jӧnkӧping UniversityJӧnkӧping CountySweden
  5. 5.Division of Neuropsychiatry Unit, Department of Women’s and Children’s HealthCenter of Neurodevelopment Disorders at Karolinska Institutet (KIND)StockholmSweden
  6. 6.Child and Adolescent Psychiatry, Center for Psychiatry Research, Stockholm County CouncilStockholm CountySweden

Personalised recommendations