A Review on Facial Expression Based Behavioral Analysis Using Computational Technique for Autistic Disorder Patients

  • Camellia RayEmail author
  • Hrudaya Kumar Tripathy
  • Sushruta Mishra
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1046)


Within recent decades the chances of a child being diagnosed with autism spectrum disorder have increased dramatically. Individuals with autism disorder have markedly different social and emotional actions and reactions than non-autistic individuals. It is a chronic disorder whose symptoms include failure to develop normal social relations with other people, impaired development of communicative ability, lack of imaginative ability, and repetitive, stereotyped movements. There exist numerous techniques associated to detect autism disorders in children. Facial expression-based method is an effective technique frequently used by medical experts to detect the emotional patterns of autistic children. Our paper reviews this technique to determine the behavioral analysis of autistic children. Comparative analysis of existing techniques is undertaken to select the most optimal technique of autism detection.


Autistic disorder Facial expression Emotion recognition Shanon’s entropy Kinect sensor $P recognizer 3D facial model 



The paper is one of the review paper of behavioral analysis of the autistic child before starting my implementation to diagnose the autistic child. I want to express my most profound thankfulness to each one of the individuals who has given me the likelihood to complete the paper. An exceptional appreciation I provide for my guide and my co-guide in invigorating recommendations and support, helped me to organize my point particularly in composing this paper. I would like to express my gratitude to the dean of my college for being a greater part to complete my paper.


  1. 1.
    Li, Y., Elmaghraby, A.S.: A Framework for using games for behavioral analysis of autistic children. In: Computer Games: AI, Animation, Mobile, Multimedia, Educational and Serious Games (CGAMES), pp. 1–4. IEEE, Louisville (2014)Google Scholar
  2. 2.
    Abirached, B., et al.: Improving communication skills of children with ASDs through interaction with visual characters. In: 1st International Conference on Serious Games and Applications for Health (SeGAH), pp. 1–4. IEEE, Braga (2011)Google Scholar
  3. 3.
    Heni, N., Hamam, H.: Design of emotional educational system mobile games for autistic children. In: 2nd International Conference on Advanced Technologies for Signal and Image Processing, pp. 631–637. IEEE, Monastir (2016)Google Scholar
  4. 4.
    Lacroix, A., Guidette, M., Roge, B., Reilly, J.: Facial emotion recognition in 4-to 8-year-olds with autism spectrum disorder: A developmental trajectory approach. Res. Autism Spectr. Disord. 8(9), 1146–1154 (2014)CrossRefGoogle Scholar
  5. 5.
    Chen, C.H., Lee, I.J., Lin, L.Y.: Augmented reality-based self-facial modeling to promote the emotional expression and social skills of adolescents with autism spectrum disorders. Res. Dev. Disabil. 36(8), 396–403 (2014)Google Scholar
  6. 6.
    Cheng, Y., Luo, S.Y., Lin, H.C.: Investigating the performance on comprehending 3D social emotion through a mobile learning system for individuals with autistic spectrum disorder. In: 5th IIAI International Congress on Advanced Applied Informatics, pp. 414–417. IEEE, Kumamoto (2016)Google Scholar
  7. 7.
    Del Coco, M., et al.: A computer vision based approach for understanding emotional involvements in children with autism spectrum disorders. In: International Conference on Computer Vision Workshops, pp. 1401–1407. IEEE, Venice (2017)Google Scholar
  8. 8.
    Jazouli, M., Majda, A., Zarghili, A.: A $P recognizer for automatic facial emotion recognition using Kinect sensor. In: Intelligent Systems and Computer Vision (ISCV), pp. 402–406. IEEE, Fez (2017)Google Scholar
  9. 9.
    Silva, V., Soares, F., Esteves, J.S.: Mirroring and recognizing emotions through facial expressions for a RoboKind platform. In: 5th Portuguese Meeting on Bioengineering (ENBENG), pp. 1–4. IEEE, Coimbra (2017)Google Scholar
  10. 10.
    Griffiths, S., Jarrold, C., Penton-Voak, I.S., Woods, A.T., Skinner, A.L., Munafò, M.R.: Impaired recognition of basic emotions from facial expressions in young people with autism spectrum disorder. J. Autism Dev. Disord. 47, 1–11 (2017)CrossRefGoogle Scholar
  11. 11.
    Trigeorgis, G., et al.: Adieu features?: End-to-end speech emotion recognition using a deep convolutional recurrent network. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5200–5204. IEEE, Sanghai (2016)Google Scholar
  12. 12.
    Gong, Y., Poellabauer, C.: Continuous assessment of children’s emotional states using acoustic analysis. In: International Conference on Healthcare Informatics (ICHI), pp. 5200–5204. IEEE, Park City Park City (2017)Google Scholar
  13. 13.
    Tzirakis, P., Trigeorgis, G., Nicolaou, M.A., Schuller, B.W., Zafeiriou, S.: End-to-end multimodal emotion recognition using deep neural networks. IEEE J. Sel. Top. Sig. Process. 11(6), 1301–1309 (2017)CrossRefGoogle Scholar
  14. 14.
    Mollahosseini, A., Chan, D., Mahoor, M.H.: Going deeper in facial expression recognition using deep neural networks. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–10. IEEE, Lake Placid (2016)Google Scholar
  15. 15.
    Yeung, M.K., Han, Y.M.Y., Sze, S.L., Chan, A.S.: Altered right frontal cortical connectivity during facial emotion recognition in children with autism spectrum disorders. Res. Autism Spect. Disord. 8(11), 1567–1577 (2014)CrossRefGoogle Scholar
  16. 16.
    Rodriguez-Bermudez, G., Garcia-Laencina, P.: Analysis of EEG signals using nonlinear dynamics and chaos: a review. Appl. Math. Inf. Sci. Int. J. Nat. Sci. 9(5), 2309–2321 (2015)MathSciNetGoogle Scholar
  17. 17.
    Othmana, M., Wahaba, A., Karima, I., Dzulkifli, M.A., Alshaiklia, I.F.T.: EEG emotion recognition based on the dimensional models of emotions. Proc. Soc. Behav. Sci. 97, 30–37 (2013)CrossRefGoogle Scholar
  18. 18.
    Black, M.H., et al.: Mechanisms of facial emotion recognition in autism spectrum disorders: insights from eye tracking and electroencephalography. Neurosci. Biobehav. Rev. 80, 488–515 (2017)CrossRefGoogle Scholar
  19. 19.
    Jamil, N., Khir, N.H.M., Ismail, M., Razak, F.H.A.: Gait-based emotion detection of children with autism spectrum disorders: a preliminary investigation. Proc. Comput. Sci. 76, 342–348 (2015)CrossRefGoogle Scholar
  20. 20.
    Zhou, Y., Yu, F., Duong, T.: Multiparametric MRI characterization and prediction in autism spectrum disorder using graph theory and machine learning. PlosOne 9(6), 1–14 (2014)Google Scholar
  21. 21.
    Leo, M., Del Coco, M., Carcagn, P., Distante, C.: Automatic emotion recognition in robot-children interaction for ASD Treatment. In: IEEE International Conference on Computer Vision Workshop (ICCVW), pp. 537–545. IEEE, Santiago (2015)Google Scholar
  22. 22.
    Swangnetr, M., Kaber, D.B.: Emotional state classification in patient–robot interaction using wavelet analysis and statistics-based feature selection. IEEE Trans. Hum.-Mach. Syst. 43(1), 63–75 (2013)CrossRefGoogle Scholar
  23. 23.
    Soares, F., et al.: Robotica-Autismo project: technology for autistic children. In: 3rd Portuguese Meeting in Bioengineering IEEE (ENBENG), pp. 1–4. IEEE, Braga (2013)Google Scholar
  24. 24.
    Fan, J., Bekele, E., Warren, Z., Sarkar, N.: EEG analysis of facial affect recognition process of individuals with ASD performance prediction leveraging social context. In: Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), pp. 38–43. IEEE, San Antonio (2017)Google Scholar
  25. 25.
    Li, B.Y., Mian, A.S., Liu, W., Krishna, A.: Using Kinect for face recognition under varying poses, expressions, illumination and disguise. In: IEEE Workshop on Applications of Computer Vision (WACV), pp. 186–192. IEEE, Tampa (2013)Google Scholar
  26. 26.
    Manfredonia, J., et al.: Automatic recognition of posed facial expression of emotion in individuals with autism spectrum disorder. J. Autism Dev. Disord. 49(1), 279–293 (2019)CrossRefGoogle Scholar
  27. 27.
    Golan, O., Gordon, I., Fichman, K., Keinan, G.: Specific patterns of emotion recognition from faces in children with ASD: results of a cross-modal matching paradigm. J. Autism Dev. Disord. 48(3), 844–852 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Camellia Ray
    • 1
    Email author
  • Hrudaya Kumar Tripathy
    • 1
  • Sushruta Mishra
    • 1
  1. 1.School of Computer EngineeringKalinga Institute of Industrial Technology, Deemed to be UniversityBhubaneswarIndia

Personalised recommendations