Skip to main content
SpringerLink
Log in

Behavioral Engagement Detection of Students in the Wild

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10331))

Included in the following conference series:

Abstract

This paper aims to investigate students’ behavioral engagement (On-Task vs. Off-Task) in authentic classrooms. We propose a two-phased approach for automatic engagement detection: In Phase 1, contextual logs are utilized to assess active usage of the content platform. If there is active use, the appearance information is utilized in Phase 2 to infer behavioral engagement. Through authentic classroom pilots, we collected around 170 hours of in-the-wild data from 28 students in two different classrooms using two different content platforms (one for Math and one for English as a Second Language (ESL)). Our data collection application captured appearance data from a 3D camera and context data from uniform resource locator (URL) logs. We experimented with two test cases: (1) Cross-classroom, where trained models were tested on a different classroom’s data; (2) Cross-platform, where the data collected in different subject areas (Math or ESL) were utilized in training and testing, respectively. For the first case, the behavioral engagement was detected with an F1-score of 77%, using only appearance. Incorporating the contextual information improved the overall performance to 82%. For the second case, even though the subject areas and content platforms changed, the proposed appearance classifier still achieved 72% accuracy (compared to 77%). Our experiments proved that the accuracy of the proposed model is not adversely impacted considering different set of students or different subject areas.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Training set sizes differ for each student, as leave-one-subject-out approach is utilized in model training.

References

  1. Madden, M., Lenhart, A., Duggan, M., Cortesi, S., Gasser, U.: Teens and Technology. Pew Internet & American Life Project, Washington, DC (2013)

    Google Scholar 

  2. Woolf, B., Burleson, W., Arroyo, I., Dragon, T., Cooper, D., Picard, R.: Affect-aware tutors: recognising and responding to student affect. Int. J. Learn. Technol. 4(3–4), 129–164 (2009)

    Article  Google Scholar 

  3. D’Mello, S., Olney, A., Williams, C., Hays, P.: Gaze tutor: a gaze-reactive intelligent tutoring system. Int. J. Hum. Comput. Stud. 70(5), 377–398 (2012)

    Article  Google Scholar 

  4. Fredricks, J.A., Blumenfeld, P.C., Paris, A.H.: School engagement: potential of the concept, state of the evidence. Rev. Educ. Res. 74(1), 59–109 (2004)

    Article  Google Scholar 

  5. Bosch, N., D’Mello, S., Baker, R., Ocumpaugh, J., Shute, V., Ventura, M., Wang, L., Zhao, W.: Automatic detection of learning-centered affective states in the wild. In: International Conference on Intelligent User Interfaces (2015)

    Google Scholar 

  6. Chen, J., Luo, N., Liu, Y., Liu, L., Zhang, K., Kolodziej, J.: A hybrid intelligence-aided approach to affect-sensitive e-learning. Computing 98(1–2), 215–233 (2016)

    Article  MathSciNet  Google Scholar 

  7. Alyuz, N., Okur, E., Oktay, E., Genc, U., Aslan, S., Mete, S.E., Stanhill, D., Arnrich, B., Esme, A.A.: Towards an emotional engagement model: can affective states of a learner be automatically detected in a 1:1 learning scenario. In: International Conference on User Modeling, Adaptation, and Personalization - Workshop on Personalization Approaches in Learning Environments (2016)

    Google Scholar 

  8. Alyuz, N., Okur, E., Oktay, E., Genc, U., Aslan, S., Mete, S.E., Arnrich, B., Esme, A.A.: Semi-supervised model personalization for improved detection of learner’s emotional engagement. In: International Conference on Multimodal Interaction (2016)

    Google Scholar 

  9. Reinhard Pekrun, L.L.-G.: Academic emotions and student engagement. In: Handbook of Research on Student Engagement, pp. 259–282. Springer, New York (2012)

    Google Scholar 

  10. Rodrigo, M.M.T., Baker, R.S.J.D., Rossi, L.: Student off-task behavior in computer-based learning in the Philippines: comparison to prior research in the USA. Teachers Coll. Rec. 115(10), 1–27 (2013)

    Google Scholar 

  11. Grafsgaard, J.F., Wiggins, J.B., Boyer, K.E., Wiebe, E.N., Lester, J.C.: Embodied affect in tutorial dialogue: student gesture and posture. In: International Conference on Artificial Intelligence in Education (2013)

    Google Scholar 

  12. Paquette, L., Rowe, J., Baker, R., Mott, B., Lester, J., DeFalco, J., Brawner, K., Sottilare, R., Georgoulas, V.: Sensor-free or sensor-full: a comparison of data modalities in multi-channel affect detection. In: International Conference on Educational Data Mining (2015)

    Google Scholar 

  13. Miller, W.L., Baker, R.S., Labrum, M.J., Petsche, K., Liu, Y.H., Wagner, A.Z.: Automated detection of proactive remediation by teachers in Reasoning Mind classrooms. In: International Conference on Learning Analytics and Knowledge (2015)

    Google Scholar 

  14. Fancsali, S.E.: Data-Driven causal modeling of “gaming the system” and off-task behavior in cognitive tutor algebra. In: NIPS Workshop on Data Driven Education (2013)

    Google Scholar 

  15. Gobert, J.D., Baker, R.S., Wixon, M.B.: Operationalizing and detecting disengagement within online science microworlds. Educ. Psychol. 50(1), 43–57 (2015)

    Article  Google Scholar 

  16. Mohammadi-Aragh, M.J., Williams, C.B.: Student attention in unstructured-use, computer-infused classrooms. In: American Society of Engineering Education Annual Conference (2013)

    Google Scholar 

  17. Su, Y.N., Hsu, C.C., Chen, H.C., Huang, K.K., Huang, Y.M.: Developing a sensor-based learning concentration detection system. Eng. Comput. 31(2), 216–230 (2014)

    Article  Google Scholar 

  18. Aslan, S., Cataltepe, Z., Diner, I., Dundar, O., Esme, A.A., Ferens, R., Kamhi, G., Oktay, E., Soysal, C., Yener, M.: Learner engagement measurement and classification in 1:1 learning. In: International Conference on Machine Learning and Applications (2014)

    Google Scholar 

  19. Intel Corporation: Intel RealSense SDK: Design Guidelines (2014). https://software.intel.com/sites/default/files/managed/27/50/Intel%20RealSense%20SDK%20Design%20Guidelines%20F200%20v2.pdf. Accessed 2016

  20. Chen, C., Liaw, A., Breiman, L.: Using Random Forest to Learn Imbalanced Data. University of California, Berkeley (2004)

    Google Scholar 

  21. Aslan, S., Mete, S.E., Okur, E., Oktay, E., Alyuz, N., Genc, U., Stanhill, D., Esme, A.A.: Human expert labeling process (HELP): towards a reliable higher-order user state labeling by human experts. In: International Conference on Intelligent Tutoring Systems - Workshops (2016)

    Google Scholar 

  22. Krippendorff, K.: Reliability in content analysis. Hum. Commun. Res. 30(3), 411–433 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Intel Labs, Intel Corporation, Hillsboro, USA

    Eda Okur, Nese Alyuz, Sinem Aslan, Utku Genc, Cagri Tanriover & Asli Arslan Esme

Authors
  1. Eda Okur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nese Alyuz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sinem Aslan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Utku Genc
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cagri Tanriover
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Asli Arslan Esme
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eda Okur .

Editor information

Editors and Affiliations

  1. Human-Centered Multimedia, University of Augsburg, Augsburg, Germany

    Elisabeth André

  2. University of Pennsylvania, Philadelphia, Pennsylvania, USA

    Ryan Baker

  3. Department of Psychology, University of Memphis, Memphis, Tennessee, USA

    Xiangen Hu

  4. Ateneo de Manila University, Quezon City, Philippines

    Ma. Mercedes T. Rodrigo

  5. University of Sussex, Brighton, United Kingdom

    Benedict du Boulay

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Okur, E., Alyuz, N., Aslan, S., Genc, U., Tanriover, C., Arslan Esme, A. (2017). Behavioral Engagement Detection of Students in the Wild. In: André, E., Baker, R., Hu, X., Rodrigo, M., du Boulay, B. (eds) Artificial Intelligence in Education. AIED 2017. Lecture Notes in Computer Science(), vol 10331. Springer, Cham. https://doi.org/10.1007/978-3-319-61425-0_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-61425-0_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-61424-3

  • Online ISBN: 978-3-319-61425-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation