Advertisement

Comparative study on emotions analysis from facial expressions in children with and without learning disabilities in virtual learning environment

  • Nihal OuherrouEmail author
  • Oussama Elhammoumi
  • Fatimaezzahra Benmarrakchi
  • Jamal El Kafi
Article
  • 30 Downloads

Abstract

Children with Learning Disabilities (LDs) show some emotional difficulties and behavioral problems in classroom compared with their peers without LDs. Emotions constitute an important part of the learning process. Recent evidence suggests that the use of Information and Communication Technology (ICT) in special education permits to remove barriers in learning for the target children. Besides, it offers a learning environment for a diversity of emotional experiences. In this present study, we explored the benefits of ICT use to identify the ways in which emotions are involved during the learning process in Virtual Learning Environments (VLE). We conducted a user study with 42 children divided into two groups; experimental group (n = 14) and age matched control group (n = 28) to compare their emotional experiences in VLE. We used advances in Artificial Intelligence (AI) to detect children’s emotions through their facial expressions by analyzing seven basic facial emotion expressions (angry, disgust, fear, happy, sad, surprise and neutral) while playing an educational game. The initial results indicate that emotions are present in VLE and they appear to suggest that children with LDs experience the same emotions as their peers without LDs in VLE. Besides, they show that children with LDs experience less negative emotions compared to literature evidence about the presence of a higher level of negative emotions in classroom.

Keywords

Learning disabilities (LDs) Information and communication technology (ICT) Assistive technology (AT) Artificial intelligence (AI) Virtual learning environment (VLE) Emotion recognition 

Notes

Acknowledgments

This work was financially supported by an Excellence Grant accorded to Nihal Ouherrou (3UCD2018) and Oussama Elhammoumi (11UAE2017) by the National Center of Scientific and Technical Research (CNRST)-Minister of National Education, Higher Education, Staff Training and Scientific Research, Morocco.

The authors would like to acknowledge the president and staff at Speech-Language Pathology Service-Health center, El Jadida Morocco and also children who have participated in this study. The authors would like also to thank the speech therapist Ilham Elhousni for her valuable suggestions and recommendations and the stuff at the primary school l’Ange Bleu El Jadida for their cooperation.

References

  1. Abrams, J. C. (1986). On learning disabilities: Affective considerations. Journal of Reading, Writing, and Learning Disabilities International, 2(3), 189–196.  https://doi.org/10.1080/0748763860020303.Google Scholar
  2. Adam, T., & Tatnall, A. (2010). Use of ICT to assist students with learning difficulties: An Actor-Network Analysis. In Key competencies in the knowledge society (pp. 1–11). Springer.Google Scholar
  3. Adam, T., & Tatnall, A. (2017). The value of using ICT in the education of school students with learning difficulties. Education and Information Technologies, 22(6), 2711–2726.Google Scholar
  4. Adebisi, R. O., Liman, N. A., & Longpoe, P. K. (2015). Using assistive Technology in Teaching Children with learning disabilities in the 21st century. Journal of Education and Practice, 6(24), 14–20.Google Scholar
  5. Ahmad, W. F. W., Akhir, E. A. P., & Azmee, S. (2010). Game-based learning courseware for children with learning disabilities (pp. 1–4). IEEE.  https://doi.org/10.1109/ITSIM.2010.5561303.
  6. AL Zyoudi, M. (2010). Differences in self-concept among student with and without learning disabilities in Al Karak District in Jordan. International Journal of Special Education, 25(2), 72–77.Google Scholar
  7. Alexander, S., Sarrafzadeh, A., & Hill, S. (2006). Easy with eve: A functional affective tutoring system. In Workshop on Motivational and Affective Issues in ITS. 8th International Conference on ITS (pp. 5–12). Citeseer.Google Scholar
  8. Bahreini, K., Nadolski, R., & Westera, W. (2016). Towards multimodal emotion recognition in e-learning environments. Interactive Learning Environments, 24(3), 590–605.Google Scholar
  9. Ben Ammar, M., Neji, M., Alimi, A. M., & Gouardères, G. (2010). The affective tutoring system. Expert Systems with Applications, 37(4), 3013–3023.  https://doi.org/10.1016/j.eswa.2009.09.031.Google Scholar
  10. Benmarrakchi, F., El Kafi, J., & Hore, A. E. (2015). A different learning way for pupils with specific learning disabilities. International Journal of Computers & Technology, 14, 6157–6162.Google Scholar
  11. Benmarrakchi, F. E., El Kafi, J., & Elhore, A. (2016). Supporting dyslexic’s learning style preferences in adaptive virtual learning environment (pp. 1–6). IEEE.  https://doi.org/10.1109/ICEMIS.2016.7745294.
  12. Benmarrakchi, F. E., El Kafi, J., & Elhore, A. (2017a). Communication Technology for Users with specific learning disabilities. Procedia Computer Science, 110, 258–265.  https://doi.org/10.1016/j.procs.2017.06.093.Google Scholar
  13. Benmarrakchi, F. E., El Kafi, J., & Elhore, A. (2017b). User modeling approach for dyslexic students in virtual learning environments. International Journal of Cloud Applications and Computing (IJCAC), 7(2), 1–9.Google Scholar
  14. Benmarrakchi, F., El Kafi, J., Elhore, A., & Haie, S. (2017c). Exploring the use of the ICT in supporting dyslexic students’ preferred learning styles: A preliminary evaluation. Education and Information Technologies, 22(6), 2939–2957.  https://doi.org/10.1007/s10639-016-9551-4.Google Scholar
  15. Bower, G. H. (1981). Mood and memory. American Psychologist, 36(2), 129–148.  https://doi.org/10.1037/0003-066X.36.2.129.Google Scholar
  16. Bowlby, J. (1969). Attachment, Vol. 1 of attachment and loss. New York: Basic Books.Google Scholar
  17. Bryan, T., Burstein, K., & Ergul, C. (2004). The social-emotional side of learning disabilities: A science-based presentation of the state of the art. Learning Disability Quarterly, 27(1), 45–51.  https://doi.org/10.2307/1593631.Google Scholar
  18. Burgstahler, S. (2003). The role of Technology in Preparing Youth with disabilities for postsecondary education and employment. Journal of Special Education Technology, 18(4), 7–19.  https://doi.org/10.1177/016264340301800401.Google Scholar
  19. Bursuck, W. (1989). A comparison of students with learning disabilities to low achieving and higher achieving students on three dimensions of social competence. Journal of Learning Disabilities, 22(3), 188–194.  https://doi.org/10.1177/002221948902200310.Google Scholar
  20. Calleja, G. (2011). In-game: From immersion to incorporation. MIT Press.Google Scholar
  21. Carver, C. S., & Scheier, M. F. (2012). Perspectives on personality. Pearson education.Google Scholar
  22. Cleveland-Innes, M., & Campbell, P. (2012). Emotional presence, learning, and the online learning environment. The International Review of Research in Open and Distributed Learning, 13(4), 269–292.  https://doi.org/10.19173/irrodl.v13i4.1234.Google Scholar
  23. Cortiella, C., & Horowitz, S. H. (2014). The state of learning disabilities: Facts, trends and emerging issues (pp. 2–45). New York: National Center for Learning Disabilities.Google Scholar
  24. Cosden, M., Elliott, K., Noble, S., & Kelemen, E. (1999). Self-understanding and self-esteem in children with learning disabilities. Learning Disability Quarterly, 22(4), 279–290.  https://doi.org/10.2307/1511262.Google Scholar
  25. Crutzen, R., van ‘t Riet, J., & Short, C. E. (2015). Enjoyment: A conceptual exploration and overview of experimental evidence in the context of games for health. Games for Health Journal, 5(1), 15–20.  https://doi.org/10.1089/g4h.2015.0059.Google Scholar
  26. D’Mello, S., Picard, R. W., & Graesser, A. (2007). Toward an affect-sensitive AutoTutor. IEEE Intelligent Systems, 22(4), 53–61.  https://doi.org/10.1109/MIS.2007.79.Google Scholar
  27. Dillon, P. (2004). Trajectories and tensions in the theory of information and communication Technology in Education. British Journal of Educational Studies, 52(2), 138–150.  https://doi.org/10.1111/j.1467-8527.2004.00259.x.Google Scholar
  28. Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2), 124–129.  https://doi.org/10.1037/h0030377.Google Scholar
  29. Ekman, I., Chanel, G., Järvelä, S., Kivikangas, J. M., Salminen, M., & Ravaja, N. (2012). Social interaction in games: Measuring physiological linkage and social presence. Simulation & Gaming, 43(3), 321–338.  https://doi.org/10.1177/1046878111422121.Google Scholar
  30. Forgas, J. P. (2012). Feeling and speaking: Affective influences on communication strategies and language use. Social Cognition and Communication, 63–81.Google Scholar
  31. Forgas, J. P. (2017). Mood effects on cognition: Affective influences on the content and process of information processing and behavior. In Emotions and affect in human factors and human-computer interaction (pp. 89–122). Elsevier.  https://doi.org/10.1016/B978-0-12-801851-4.00003-3.
  32. Gajadhar, B. J., de Kort, Y. A. W., & IJsselsteijn, W. A. (2008). Shared Fun Is Doubled Fun: Player Enjoyment as a Function of Social Setting. In P. Markopoulos, B. de Ruyter, W. IJsselsteijn, & D. Rowland (Eds.), Fun and Games (pp. 106–117). Berlin Heidelberg: Springer.Google Scholar
  33. Gallegos, J., Langley, A., & Villegas, D. (2012). Anxiety, depression, and coping skills among Mexican school children: A comparison of students with and without learning disabilities. Learning Disability Quarterly, 35(1), 54–61.  https://doi.org/10.1177/0731948711428772.Google Scholar
  34. Graetz, K. A. (2006). The psychology of learning environments. Learning Spaces, 6.Google Scholar
  35. Gross, J. J. (2010). The Future’s so bright, I Gotta Wear shades. Emotion Review, 2(3), 212–216.  https://doi.org/10.1177/1754073910361982.Google Scholar
  36. Hammill, D. D. (1990). On defining learning disabilities: An emerging consensus. Journal of Learning Disabilities, 23(2), 74–84.  https://doi.org/10.1177/002221949002300201.Google Scholar
  37. Hammoumi, O. E., Benmarrakchi, F.E., Ouherrou, N., El Kafi, J., & ElHore, A. (2018). Emotion Recognition in E-learning Systems. In 2018 6th International Conference on Multimedia Computing and Systems (ICMCS) (pp. 1–6).  https://doi.org/10.1109/ICMCS.2018.8525872.
  38. Hassan, A. E. H. (2015). Emotional and behavioral problems of children with learning disabilities. Journal of Educational Policy and Entrepreneurial Research (JEPER), 2(10), 66–74.Google Scholar
  39. Hill, K. T., & Wigfield, A. (1984). Test anxiety: A major educational problem and what can be done about it. The Elementary School Journal, 85(1), 105–126.Google Scholar
  40. Horvat, M., Kukolja, D., & Ivanec, D. (2015). Comparing affective responses to standardized pictures and videos: A study report. ArXiv:1505.07398 [Cs]. Retrieved from https://arxiv.org/ftp/arxiv/papers/1505/1505.07398.pdf.
  41. Ismaili, J. (2017). Mobile learning as alternative to assistive technology devices for special needs students. Education and Information Technologies, 22(3), 883–899.Google Scholar
  42. Jacko, J. A. (2012). Human computer interaction handbook: Fundamentals, evolving technologies, and emerging applications, third edition. CRC Press.Google Scholar
  43. Kah, A. E., & Lakhouaja, A. (2018). Developing effective educative games for Arabic children primarily dyslexics. Education and Information Technologies, 23, 1–20.  https://doi.org/10.1007/s10639-018-9750-2.Google Scholar
  44. Kanade, T., Cohn, J. F., & Tian, Y. (2000). Comprehensive database for facial expression analysis. In Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) (pp. 46–53).  https://doi.org/10.1109/AFGR.2000.840611.
  45. Kistner, J., Haskett, M., White, K., & Robbins, F. (1987). Perceived competence and self-worth of LD and normally achieving students. Learning Disability Quarterly, 10(1), 37–44.  https://doi.org/10.2307/1510753.Google Scholar
  46. Kort, B., Reilly, R., & Picard, R. W. (2001). An affective model of interplay between emotions and learning: reengineering educational pedagogy-building a learning companion. In Proceedings IEEE International Conference on Advanced Learning Technologies (pp. 43–46).  https://doi.org/10.1109/ICALT.2001.943850.
  47. Linnenbrink-Garcia, L., & Pekrun, R. (2011). Students’ emotions and academic engagement: Introduction to the special issue. Contemporary Educational Psychology, 36(1), 1–3.  https://doi.org/10.1016/j.cedpsych.2010.11.004.Google Scholar
  48. Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010). The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops (pp. 94–101).  https://doi.org/10.1109/CVPRW.2010.5543262.
  49. Lundqvist, D., Flykt, A., & Öhman, A. (1998). The Karolinska directed emotional faces (KDEF). CD ROM from Department of Clinical Neuroscience, Psychology Section, Karolinska Institutet, (1998).Google Scholar
  50. Maag, J. W., & Reid, R. (2006). Depression among students with learning disabilities: Assessing the risk. Journal of Learning Disabilities, 39(1), 3–10.  https://doi.org/10.1177/00222194060390010201.Google Scholar
  51. Margalit, M. (1991). Understanding loneliness among students with learning disabilities. Behaviour Change, 8(04), 167–173.  https://doi.org/10.1017/S0813483900006641.Google Scholar
  52. Margalit, M., & Ben-Dov, I. (1995). Learning disabilities and social environments: Kibbutz versus City comparisons of loneliness and social competence. International Journal of Behavioral Development, 18(3), 519–536.  https://doi.org/10.1177/016502549501800308.Google Scholar
  53. Nelson, J. M., & Harwood, H. (2011). Learning disabilities and anxiety: A meta-analysis. Journal of Learning Disabilities, 44(1), 3–17.  https://doi.org/10.1177/0022219409359939.Google Scholar
  54. Pekrun, R. (1992). The impact of emotions on learning and achievement: Towards a theory of cognitive/motivational mediators. Applied Psychology, 41(4), 359–376.  https://doi.org/10.1111/j.1464-0597.1992.tb00712.x.Google Scholar
  55. Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37(2), 91–105.Google Scholar
  56. Petrovica, S., Anohina-Naumeca, A., & Ekenel, H. K. (2017). Emotion recognition in affective tutoring systems: Collection of ground-truth data. Procedia Computer Science, 104, 437–444.  https://doi.org/10.1016/j.procs.2017.01.157.Google Scholar
  57. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178.  https://doi.org/10.1037/h0077714.Google Scholar
  58. Russell, J. A., & Barrett, L. F. (1999). Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant. Journal of Personality and Social Psychology, 76(5), 805–819.Google Scholar
  59. Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67.  https://doi.org/10.1006/ceps.1999.1020.Google Scholar
  60. Sandanayake, T. C., Madurapperuma, A. P., & Dias, D. (2011). Affective E learning model for Recognising learner emotions. International Journal of Information and Education Technology, 1(4), 315.Google Scholar
  61. Sarrafzadeh, A., Hosseini, H. G., Fan, C., & Overmyer, S. P. (2003). Facial expression analysis for estimating learner’s emotional state in intelligent tutoring systems. In Proceedings 3rd IEEE International Conference on Advanced Technologies (pp. 336–337).  https://doi.org/10.1109/ICALT.2003.1215111.
  62. Sarrafzadeh, A., Alexander, S., Dadgostar, F., Fan, C., & Bigdeli, A. (2008). “How do you know that I don’t understand?” a look at the future of intelligent tutoring systems. Computers in Human Behavior, 24(4), 1342–1363.  https://doi.org/10.1016/j.chb.2007.07.008.Google Scholar
  63. Sathik, M., & Jonathan, S. G. (2013). Effect of facial expressions on student’s comprehension recognition in virtual educational environments. SpringerPlus, 2(1), 455.  https://doi.org/10.1186/2193-1801-2-455.Google Scholar
  64. Shen, L., Leon, E., Callaghan, V., & Shen, R. (2007). Exploratory research on an affective elearning model. In International Workshop on Blended Learning (pp. 15–17).Google Scholar
  65. Shen, L., Wang, M., & Shen, R. (2009). Affective e-learning: Using" emotional" data to improve learning in pervasive learning environment. Journal of Educational Technology & Society, 12(2).Google Scholar
  66. Sorour, A. S., Mohamed, N. A., & El-Maksoud, M. M. A. (2014). Emotional and behavioral problems of primary school children with and without learning disabilities: A comparative study. Journal of Education and Practice, 5(8), 1–11.Google Scholar
  67. Stemmler, G., & Wacker, J. (2010). Personality, emotion, and individual differences in physiological responses. Biological Psychology, 84(3), 541–551.  https://doi.org/10.1016/j.biopsycho.2009.09.012.Google Scholar
  68. Tam, H. E., & Hawkins, R. (2012). Self-concept and depression levels of students with dyslexia in Singapore.Google Scholar
  69. Toro, P. A., Weissberg, R. P., Guare, J., & Liebenstein, N. L. (1990). A comparison of children with and without learning disabilities on social problem-solving skill, school behavior, and family background. Journal of Learning Disabilities, 23(2), 115–120.  https://doi.org/10.1177/002221949002300207.Google Scholar
  70. Vaughn, S., Haager, D., Hogan, A., & Kouzekanani, K. (1992). Self-concept and peer acceptance in students with learning disabilities: A four- to five-year prospective study. Journal of Educational Psychology, 84(1), 43–50.  https://doi.org/10.1037/0022-0663.84.1.43.Google Scholar
  71. Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 (Vol. 1, pp. I-511-I–518 vol.1).  https://doi.org/10.1109/CVPR.2001.990517.
  72. Vodopivec, J. L., & Bagon, S. (2016). Motivation for using ICT and pupils with learning difficulties. International Journal of Emerging Technologies in Learning (IJET), 11(10), 70–75.Google Scholar
  73. Wiener, J., & Schneider, B. H. (2002). A multisource exploration of the friendship patterns of children with and without learning disabilities. Journal of Abnormal Child Psychology, 30(2), 127–141.Google Scholar
  74. Yannakakis, G. N., & Paiva, A. (2014). Emotion in games. Handbook on Affective Computing, 459–471.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of computer science, Faculty of ScienceChouaib Doukkali UniversityEl JadidaMorocco

Personalised recommendations