Skip to main content
SpringerLink
Log in

Multimodal Analytics for Real-Time Feedback in Co-located Collaboration

  • Conference paper
  • First Online:
Lifelong Technology-Enhanced Learning (EC-TEL 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11082))

Included in the following conference series:

Abstract

Collaboration is an important 21st century skill; it can take place in a remote or co-located setting. Co-located collaboration (CC) is a very complex process which involves subtle human interactions that can be described with multimodal indicators (MI) like gaze, speech and social skills. In this paper, we first give an overview of related work that has identified indicators during CC. Then, we look into the state-of-the-art studies on feedback during CC which also make use of MI. Finally, we describe a Wizard of Oz (WOz) study where we design a privacy-preserving research prototype with the aim to facilitate real-time collaboration in-the-wild during three co-located group PhD meetings (of 3–7 members). Here, human observers stationed in another room act as a substitute for sensors to track different speech-based cues (like speaking time and turn taking); this drives a real-time visualization dashboard on a public shared display. With this research prototype, we want to pave way for design-based research to track other multimodal indicators of CC by extending this prototype design using both humans and sensors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    An integrated sensor tracking simultaneously infrared, depth, audio and video.

  2. 2.

    An electronic sensing device worn around the neck that can collect and analyze social dynamics.

References

  1. Anastasiou, D., Ras, E.: A questionnaire-based case study on feedback by a tangible interface. In: Proceedings of the 2017 ACM WS on Intelligent Interfaces for Ubiquitous and Smart Learning, pp. 39–42. ACM (2017)

    Google Scholar 

  2. Bachour, K., Kaplan, F., Dillenbourg, P.: An interactive table for supporting participation balance in face-to-face collaborative learning. IEEE Trans. Learn. Technol. 3(3), 203–213 (2010)

    Article  Google Scholar 

  3. Balaam, M., Fitzpatrick, G., Good, J., Harris, E.: Enhancing interactional synchrony with an ambient display. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 867–876. ACM (2011)

    Google Scholar 

  4. Bassiou, N., Tsiartas, A., Smith, J., Bratt, H., Richey, C., Shriberg, E., D’Angelo, C., Alozie, N.: Privacy-preserving speech analytics for automatic assessment of student collaboration. In: INTERSPEECH, pp. 888–892 (2016)

    Google Scholar 

  5. Bergstrom, T., Karahalios, K.: Conversation clock: visualizing audio patterns in co-located groups. In: 40th Annual Hawaii International Conference on System Sciences, p. 78. IEEE (2007)

    Google Scholar 

  6. Chikersal, P., Tomprou, M., Kim, Y.J., Woolley, A.W., Dabbish, L.: Deep structures of collaboration: physiological correlates of collective intelligence and group satisfaction. In: CSCW, pp. 873–888 (2017)

    Google Scholar 

  7. Cukurova, M., Luckin, R., Mavrikis, M., Millán, E.: Machine and human observable differences in groups’ collaborative problem-solving behaviours. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds.) EC-TEL 2017. LNCS, vol. 10474, pp. 17–29. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66610-5_2

    Chapter  Google Scholar 

  8. Cukurova, M., Luckin, R., Millán, E., Mavrikis, M.: The nispi framework: analysing collaborative problem-solving from students’ physical interactions. Comput. Educ. 116, 93–109 (2018)

    Article  Google Scholar 

  9. Davidsen, J., Ryberg, T.: This is the size of one meter: childrens bodily-material collaboration. Int. J. CSCL 12(1), 65–90 (2017)

    Google Scholar 

  10. Dede, C.: Comparing frameworks for 21st century skills. In: 21st Century Skills: Rethinking How Students Learn, vol. 20, pp. 51–76 (2010)

    Google Scholar 

  11. Di Mitri, D., Schneider, J., Drachsler, H., Specht, M.: From signals to knowledge. A conceptual model for multimodal learning analytics. J. Comput. Assist. Learn. 34(4), 338–349 (2018)

    Article  Google Scholar 

  12. Dillenbourg, P.: What do you mean by collaborative learning? (1999)

    Google Scholar 

  13. DiMicco, J.M., Pandolfo, A., Bender, W.: Influencing group participation with a shared display. In: Proceedings of the 2004 ACM Conference on CSCW, pp. 614–623. ACM (2004)

    Google Scholar 

  14. Dyckhoff, A.L.: Action research and learning analytics in higher education. eleed 10(1) (2014)

    Google Scholar 

  15. Grover, S., Bienkowski, M., Tamrakar, A., Siddiquie, B., Salter, D., Divakaran, A.: Multimodal analytics to study collaborative problem solving in pair programming. In: Proceedings of the 6th International Conference on LAK, pp. 516–517. ACM (2016)

    Google Scholar 

  16. Jermann, P., Mullins, D., Nüssli, M.A., Dillenbourg, P.: Collaborative gaze footprints: correlates of interaction quality. In: CSCL 2011 Conference Proceedings, vol. 1, pp. 184–191. International Society of the Learning Sciences (2011)

    Google Scholar 

  17. Kim, T., Chang, A., Holland, L., Pentland, A.S.: Meeting mediator: enhancing group collaboration using sociometric feedback. In: Proceedings of the 2008 ACM Conference on CSCW, pp. 457–466. ACM (2008)

    Google Scholar 

  18. Kulyk, O., Wang, J., Terken, J.: Real-time feedback on nonverbal behaviour to enhance social dynamics in small group meetings. In: Renals, S., Bengio, S. (eds.) MLMI 2005. LNCS, vol. 3869, pp. 150–161. Springer, Heidelberg (2006). https://doi.org/10.1007/11677482_13

    Chapter  Google Scholar 

  19. Lubold, N., Pon-Barry, H.: Acoustic-prosodic entrainment and rapport in collaborative learning dialogues. In: Proceedings of the 2014 ACM WS on Multimodal Learning Analytics Workshop and Grand Challenge, pp. 5–12. ACM (2014)

    Google Scholar 

  20. Madan, A., Caneel, R., Pentland, A.S.: Groupmedia: distributed multi-modal interfaces. In: Proceedings of the 6th International Conference on Multimodal Interfaces, pp. 309–316. ACM (2004)

    Google Scholar 

  21. Martinez-Maldonado, R., Clayphan, A., Yacef, K., Kay, J.: Mtfeedback: providing notifications to enhance teacher awareness of small group work in the classroom. IEEE Trans. Learn. Technol. 8(2), 187–200 (2015)

    Article  Google Scholar 

  22. Meier, A., Spada, H., Rummel, N.: A rating scheme for assessing the quality of computer-supported collaboration processes. Int. J. CSCL 2(1), 63–86 (2007)

    Google Scholar 

  23. O’Donnell, A.M.: The role of peers and group learning (2006)

    Google Scholar 

  24. Pijeira-Daz, H.J., Drachsler, H., Kirschner, P.A., Järvelä, S.: Profiling sympathetic arousal in a physics course: how active are students? J. Comput. Assist. Learn. 34(4), 397–408 (2018)

    Article  Google Scholar 

  25. Richardson, D.C., Dale, R.: Looking to understand: the coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cogn. Sci. 29(6), 1045–1060 (2005)

    Article  Google Scholar 

  26. Scherr, R.E., Hammer, D.: Student behavior and epistemological framing: examples from collaborative active-learning activities in physics. Cogn. Instr. 27(2), 147–174 (2009)

    Article  Google Scholar 

  27. Schneider, B., Blikstein, P.: Unraveling students interaction around a tangible interface using multimodal learning analytics. J. Educ. Data Min. 7(3), 89–116 (2015)

    Google Scholar 

  28. Schneider, B., Pea, R.: Real-time mutual gaze perception enhances collaborative learning and collaboration quality. Int. J. CSCL 8(4), 375–397 (2013)

    Google Scholar 

  29. Schneider, B., Pea, R.: Toward collaboration sensing. Int. J. CSCL 9(4), 371–395 (2014)

    Google Scholar 

  30. Schneider, B., Sharma, K., Cuendet, S., Zufferey, G., Dillenbourg, P., Pea, R.D.: 3d tangibles facilitate joint visual attention in dyads. International Society of the Learning Sciences, Inc. [ISLS] (2015)

    Google Scholar 

  31. Schneider, J., Di Mitri, D., Limbu, B., Drachsler, H.: Multimodal learning hub: a tool for capturing customizable multimodal learning experiences. In: Drachsler, H., et al. (eds.) EC-TEL 2018. LNCS, vol. 11082, pp. 45–58. Springer, AG (2018)

    Chapter  Google Scholar 

  32. Shih, P.C., Nguyen, D.H., Hirano, S.H., Redmiles, D.F., Hayes, G.R.: Groupmind: supporting idea generation through a collaborative mind-mapping tool. In: Proceedings of the ACM 2009 International Conference on Supporting Group Work, pp. 139–148. ACM (2009)

    Google Scholar 

  33. Spikol, D., Ruffaldi, E., Dabisias, G., Cukurova, M.: Supervised machine learning in multimodal learning analytics for estimating success in project-based learning. J. Comput. Assist. Learn. 34(4), 366–377 (2018)

    Article  Google Scholar 

  34. Spikol, D., Ruffaldi, E., Landolfi, L., Cukurova, M.: Estimation of success in collaborative learning based on multimodal learning analytics features. In: Proceedings of the 17th ICALT, pp. 269–273. IEEE (2017)

    Google Scholar 

  35. Stahl, G., Law, N., Hesse, F.: Reigniting CSCL flash themes. Int. J. CSCL 8(4), 369–374 (2013)

    Google Scholar 

  36. Stiefelhagen, R., Zhu, J.: Head orientation and gaze direction in meetings. In: CHI 2002 Extended Abstracts on Human Factors in Computing Systems, pp. 858–859. ACM (2002)

    Google Scholar 

  37. Tausch, S., Hausen, D., Kosan, I., Raltchev, A., Hussmann, H.: Groupgarden: supporting brainstorming through a metaphorical group mirror on table or wall. In: Proceedings of the 8th Nordic Conference on Human-Computer Interaction, pp. 541–550. ACM (2014)

    Google Scholar 

  38. Terken, J., Sturm, J.: Multimodal support for social dynamics in co-located meetings. Pers. Ubiquitous Comput. 14(8), 703–714 (2010)

    Article  Google Scholar 

  39. Triglianos, V., Praharaj, S., Pautasso, C., Bozzon, A., Hauff, C.: Measuring student behaviour dynamics in a large interactive classroom setting. In: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization, pp. 212–220. ACM (2017)

    Google Scholar 

  40. Worsley, M., Blikstein, P.: Leveraging multimodal learning analytics to differentiate student learning strategies. In: Proceedings of the 5th International Conference on LAK, pp. 360–367. ACM (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Open Universiteit, Valkenburgerweg 177, 6419AT, Heerlen, Netherlands

    Sambit Praharaj, Maren Scheffel, Hendrik Drachsler & Marcus Specht

  2. DIPF, Schloßstr. 29, 60486, Frankfurt am Main, Germany

    Hendrik Drachsler

  3. Goethe Universität, Robert-Mayer-Str. 11-15, 60629, Frankfurt am Main, Germany

    Hendrik Drachsler

Authors
  1. Sambit Praharaj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maren Scheffel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hendrik Drachsler
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marcus Specht
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sambit Praharaj .

Editor information

Editors and Affiliations

  1. Graz University of Technology, Graz, Austria

    Viktoria Pammer-Schindler

  2. Pontificia Universidad Católica de Chile, Providencia, Santiago de Chile, Chile

    Mar Pérez-Sanagustín

  3. DIPF | Leibniz Institute for Research and Information in Education, Frankfurt, Germany

    Hendrik Drachsler

  4. RayCom BV, Utrecht, Utrecht, The Netherlands

    Raymond Elferink

  5. Open University Netherlands, Heerlen, The Netherlands

    Maren Scheffel

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Praharaj, S., Scheffel, M., Drachsler, H., Specht, M. (2018). Multimodal Analytics for Real-Time Feedback in Co-located Collaboration. In: Pammer-Schindler, V., Pérez-Sanagustín, M., Drachsler, H., Elferink, R., Scheffel, M. (eds) Lifelong Technology-Enhanced Learning. EC-TEL 2018. Lecture Notes in Computer Science(), vol 11082. Springer, Cham. https://doi.org/10.1007/978-3-319-98572-5_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-98572-5_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-98571-8

  • Online ISBN: 978-3-319-98572-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation