Abstract
Teacher orchestration of technology-enhanced learning (TEL) processes plays a major role in students’ outcomes, especially in face-to-face classrooms. However, few studies look into the fine-grained details of how such orchestration unfolds, the challenges and cognitive overload that using technologies at a classroom level pose for teachers. This paper proposes a mixed-method approach to the study of orchestration cognitive load, combining physio-behavioural (eye-tracking) and subjective measurements (questionnaires, stimulated recall interviews). We illustrate the approach by applying it to study the orchestration of two technology-enhanced geometry lessons, by a secondary school teacher. The results of our mixed-method analyses highlight the difficulty of classroom-level (as opposed to individual- or group-level) interactions, especially in modelling students’ progress and understanding. Such insights can be useful in the design of new classroom technologies, and to focus researchers’ attention on critical orchestration episodes during their evaluation.
This is a preview of subscription content, log in via an institution to check access.
Notes
- 1.
We restrict ourselves to the extreme load episodes as a way to keep the needed manpower under control (in a hour-long session the number of extreme load episodes to code tends to be around 100, while the total number of episodes can be close to 1000), and to focus the attention on the most critical orchestration episodes, contrasting them with the ‘least critical’ ones [23].
- 2.
- 3.
- 4.
- 5.
After normalization by the first 10 s episode of each session, to account for variability in the data due to the different time of the day or wakefulness of the teacher in the concerned days.
- 6.
Teacher could see either a mosaic with the miniature version of all screens where little could be made out, or a single student’s screen – which often provides an incomplete view unless the teacher actually walked there to talk with the student.
References
Alavi, H.S., Dillenbourg, P.: An ambient awareness tool for supporting supervised collaborative problem solving. IEEE Trans. Learn. Technol. 5(3), 264–274 (2012)
Anderson, J.R.: Spanning seven orders of magnitude: a challenge for cognitive modeling. Cogn. Sci. 26(1), 85–112 (2002)
Bailey, B.: Usability testing: an early history. http://webusability.com/usability-testing-a-early-history/. Accessed on 17 March 2015
Beers, P.J., Boshuizen, H.P., Kirschner, P.A., Gijselaers, W., Westendorp, J.: Cognitive load measurements and stimulated recall interviews for studying the effects of information and communications technology. Edu. Technol. Res. Develop. 56(3), 309–328 (2008)
Brunken, R., Plass, J.L., Leutner, D.: Direct measurement of cognitive load in multimedia learning. Edu. Psychol. 38(1), 53–61 (2003)
Buettner, R.: Cognitive workload of humans using artificial intelligence systems: towards objective measurement applying eye-tracking technology. In: Timm, I.J., Thimm, M. (eds.) KI 2013. LNCS, vol. 8077, pp. 37–48. Springer, Heidelberg (2013)
Creswell, J.W.: Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage publications, Thousand Oaks (2013)
Cuendet, S., Bonnard, Q., Do-Lenh, S., Dillenbourg, P.: Designing augmented reality for the classroom. Comput. Edu. 68, 557–569 (2013)
DeLeeuw, K.E., Mayer, R.E.: A comparison of three measures of cognitive load: evidence for separable measures of intrinsic, extraneous, and germane load. J. Edu. Psychol. 100(1), 223 (2008)
Dillenbourg, P.: Design for classroom orchestration. Comput. Edu. 69, 485–492 (2013)
Dillenbourg, P., Järvelä, S., Fischer, F.: The evolution of research on computer-supported collaborative learning. In: Balacheff, N., Ludvigsen, S., de Jong, T., Lazonder, A., Barnes, S. (eds.) TEL, pp. 3–19. Springer, The Netherlands (2009)
Dillenbourg, P., Zufferey, G., Alavi, H., Jermann, P., Do-Lenh, S., Bonnard, Q., Cuendet, S., Kaplan, F.: Classroom orchestration: the third circle of usability. In: Proceedings of CSCL 2011, vol. 1, pp. 510–517 (2011)
Doyle, W.: Ecological approaches to classroom management. In: Evertson, C.M., Weinstein, C.S. (eds.) Handbook of Classroom Management: Research, Practice, and Contemporary Issues , pp. 97–125 (2006)
Gómez, F., Nussbaum, M., Weitz, J.F., Lopez, X., Mena, J., Torres, A.: Co-located single display collaborative learning for early childhood education. Int. J. Comput. Support. Collab. Learn. 8(2), 225–244 (2013)
Gould, J.D., Lewis, C.: Designing for usability: key principles and what designers think. Commun. ACM 28(3), 300–311 (1985)
Hart, S.G., Staveland, L.E.: Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv. Psychol. 52, 139–183 (1988)
Kharrufa, A., Martinez-Maldonado, R., Kay, J., Olivier, P.: Extending tabletop application design to the classroom. In: Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces, pp. 115–124. ACM (2013)
Newell, A.: Unified Theories of Cognition. Harvard University Press, Cambridge (1994)
Onrubia, J., Engel, A.: The role of teacher assistance on the effects of a macro-script in collaborative writing tasks. Int. J. Comput. Support. Collab. Learn. 7(1), 161–186 (2012)
Paas, F., Renkl, A., Sweller, J.: Cognitive load theory: instructional implications of the interaction between information structures and cognitive architecture. Instr. Sci. 32(1), 1–8 (2004)
Paas, F., Tuovinen, J.E., Tabbers, H., Van Gerven, P.W.: Cognitive load measurement as a means to advance cognitive load theory. Edu. Psychol. 38(1), 63–71 (2003)
Prieto, L.P., Dlab, M.H., Gutiérrez, I., Abdulwahed, M., Balid, W.: Orchestrating technology enhanced learning: a literature review and a conceptual framework. Int. J. Technol. Enhanced Learn. 3(6), 583–598 (2011)
Prieto, L.P., Sharma, K., Wen, Y., Dillenbourg, P.: The burden of facilitating collaboration: towards estimation of teacher orchestration load using eye-tracking measures. In: Proceedings of the 11th International Conference on Computer-Supported Collaborative Learning, Vol. I, pp. 212–219 (2015)
Prieto, L.P., Wen, Y., Caballero, D., Sharma, K., Dillenbourg, P.: Studying teacher cognitive load in multi-tabletop classrooms using mobile eye-tracking. In: Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, pp. 339–344. ACM (2014)
Roschelle, J., Dimitriadis, Y., Hoppe, U.: Classroom orchestration: synthesis. Comput. Edu. 69, 523–526 (2013)
Acknowledgment
This research was supported by a Marie Curie Fellowship within the 7th European Community Framework Programme (MIOCTI, FP7–PEOPLE–2012–IEF project no. 327384). Special thanks to Wendy Farden and the International School of Lausanne for their collaboration.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Prieto, L.P., Sharma, K., Dillenbourg, P. (2015). Studying Teacher Orchestration Load in Technology-Enhanced Classrooms. In: Conole, G., Klobučar, T., Rensing, C., Konert, J., Lavoué, E. (eds) Design for Teaching and Learning in a Networked World. EC-TEL 2015. Lecture Notes in Computer Science(), vol 9307. Springer, Cham. https://doi.org/10.1007/978-3-319-24258-3_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-24258-3_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-24257-6
Online ISBN: 978-3-319-24258-3
eBook Packages: Computer ScienceComputer Science (R0)