An Initial Evaluation of Metacognitive Scaffolding for Experiential Training Simulators

  • Marcel Berthold
  • Adam Moore
  • Christina M. Steiner
  • Conor Gaffney
  • Declan Dagger
  • Dietrich Albert
  • Fionn Kelly
  • Gary Donohoe
  • Gordon Power
  • Owen Conlan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7563)


This paper elaborates on the evaluation of a Metacognitive Scaffolding Service (MSS), which has been integrated into an already existing and mature medical training simulator. The MSS is envisioned to facilitate self-regulated learning (SRL) through thinking prompts and appropriate learning hints enhancing the use of metacognitive strategies. The MSS is developed in the European ImREAL (Immersive Reflective Experience-based Adaptive Learning) project that aims to augment simulated learning environments throughout services that are decoupled from the simulation itself. Results comparing a baseline evaluation of the ‘pure’ simulator (N=131) and a first user trial including the MSS (N=143) are presented. The findings indicate a positive effect on learning motivation and perceived performance with consistently good usability. The MSS and simulator are perceived as an entity by medical students involved in the study. Further steps of development are discussed and outlined.


self-regulated learning metacognitive scaffolding training simulator augmentation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Veenman, M.V.J.: Learning to Self-Monitor and Self-Regulate. Regulation (2008)Google Scholar
  2. 2.
    Molenaar, I., Van Boxtel, C.A.M., Sleegers, P.J.C.: The effects of scaffolding metacognitive activities in small groups. Computers in Human Behavior 26, 1727–1738 (2010)CrossRefGoogle Scholar
  3. 3.
    Puustinen, M., Pulkkinen, L.: Models of Self-regulated Learning: A review. Scandinavian Journal of Educational Research 45, 269–286 (2001)CrossRefGoogle Scholar
  4. 4.
    Zimmerman, B.: Becoming a Self-Regulated Learner: An Overview. Theory Into Practice 41, 64–70 (2002)CrossRefGoogle Scholar
  5. 5.
    Pintrich, P.R.: The role of motivation in promoting and sustaining self-regulated learning. International Journal of Educational Research 31, 459–470 (1999)CrossRefGoogle Scholar
  6. 6.
    Veenman, M.V.J.: Learning to Self-Monitor and Self-Regulate. In: Mayer, R., Alexander, P. (eds.) Handbook of Research on Learning and Instruction, pp. 197–218. Routledge, New York (2011)Google Scholar
  7. 7.
    Manuela, B.: Effects of Reflection Prompts when Learning with Hypermedia. Journal of Educational Computing Research 35, 359–375 (2006)CrossRefGoogle Scholar
  8. 8.
    Krapp, A.: Die Psychologie der Lernmotivation. Perspektiven der Forschung und Probleme ihrer pädagogischen Rezeption [The psychology of motivation to learn. Perspectives of research and their pedagogical reception]. Zeitschrift für Pädagogik 39, 187–206 (1993)Google Scholar
  9. 9.
    Vygotskiǐ, L.S., Cole, M.: Mind in society: the development of higher psychological processes. Harvard University Press, Cambridge (1978)Google Scholar
  10. 10.
    Wood, D., Bruner, J.S., Ross, G.: The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry 17, 89–100 (1976)CrossRefGoogle Scholar
  11. 11.
    Bruner, J.S.: Toward a theory of instruction. Harvard University Press, Cambridge (1966)Google Scholar
  12. 12.
    Lajoie, S.P.: Extending the Scaffolding Metaphor. Instructional Science 33, 541–557 (2005)CrossRefGoogle Scholar
  13. 13.
    Clark, A.: Towards a science of the bio-technological mind. International Journal of Cognition and Technology 1, 21–33 (2002)CrossRefGoogle Scholar
  14. 14.
    Pol, J., Volman, M., Beishuizen, J.: Scaffolding in Teacher–Student Interaction: A Decade of Research. Educational Psychology Review 22, 271–296 (2010)CrossRefGoogle Scholar
  15. 15.
    Dinsmore, D.L., Alexander, P.A., Loughlin, S.M.: Focusing the Conceptual Lens on Metacognition, Self-regulation, and Self-regulated Learning. Educational Psychology Review 20, 391–409 (2008)CrossRefGoogle Scholar
  16. 16.
    Greene, J., Azevedo, R.: The Measurement of Learners’ Self-Regulated Cognitive and Metacognitive Processes While Using Computer-Based Learning Environments. Educational Psychologist 45, 203–209 (2010)CrossRefGoogle Scholar
  17. 17.
    Azevedo, R.: Does adaptive scaffolding facilitate students’ ability to regulate their learning with hypermedia? Contemporary Educational Psychology 29, 344–370 (2004)CrossRefGoogle Scholar
  18. 18.
    Azevedo, R., Hadwin, A.F.: Scaffolding Self-regulated Learning and Metacognition – Implications for the Design of Computer-based Scaffolds. Instructional Science 33, 367–379 (2005)CrossRefGoogle Scholar
  19. 19.
    Luckin, R., Hammerton, L.: Getting to Know Me: Helping Learners Understand Their Own Learning Needs through Metacognitive Scaffolding. In: Cerri, S.A., Gouardéres, G., Paraguaçu, F. (eds.) ITS 2002. LNCS, vol. 2363, pp. 759–771. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  20. 20.
    Roll, I., Aleven, V., McLaren, B.M., Koedinger, K.R.: Designing for metacognition—applying cognitive tutor principles to the tutoring of help seeking. Metacognition and Learning 2, 125–140 (2007)CrossRefGoogle Scholar
  21. 21.
    Roll, I., Aleven, V., McLaren, B.M., Koedinger, K.R.: Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system. Learning and Instruction 21, 267–280 (2011)CrossRefGoogle Scholar
  22. 22.
    Porayska-Pomsta, K., Pain, H.: Providing Cognitive and Affective Scaffolding Through Teaching Strategies: Applying Linguistic Politeness to the Educational Context. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 77–86. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  23. 23.
    Aist, G., et al.: Experimentally augmenting an intelligent tutoring system with human-supplied capabilities: adding human-provided emotional scaffolding to an automated reading tutor that listens. In: Proceedings Fourth IEEE International Conference on Multimodal Interfaces, pp. 483–490 (2002)Google Scholar
  24. 24.
    Sharma, P., Hannafin, M.J.: Scaffolding in technology-enhanced learning environments. Interactive Learning Environments 15, 27–46 (2007)CrossRefGoogle Scholar
  25. 25.
    Boyer, K.E., Phillips, R., Wallis, M., Vouk, M.A., Lester, J.C.: Balancing Cognitive and Motivational Scaffolding in Tutorial Dialogue. In: Woolf, B.P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 239–249. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  26. 26.
    Fielding, R.T.: Architectural Styles and the Design of Network-based Software Architectures, PhD-thesis, Citeseer (2000)Google Scholar
  27. 27.
    Macarthur, V., Moore, A., Mulwa, C., Conlan, O.: Towards a Cognitive Model to Support Self-Reflection: Emulating Traits and Tasks in Higher Order Schemata. In: EC-TEL 2011 Workshop on Augmenting the Learning Experience with Collaborative Reflection, Palermo, Sicily, Italy (2011)Google Scholar
  28. 28.
    Schraw, G., Sperling Dennison, R.: Assessing metacognitive awareness. Contemporary Educational Psychology 19, 460–475 (1994)CrossRefGoogle Scholar
  29. 29.
    Fill Giordano, R., Litzenberger, M., Berthold, M.: On the Assessment of strategies in self-regulated learning (SRL)–differences in adolescents of different age group and school type. Poster. 9. Tagung der Österreichischen Gesellschaft für Psychologie, Salzburg (2010)Google Scholar
  30. 30.
    Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In: Hancock, N.M.P.A. (ed.) Human Mental Workload, pp. 239–250. North Holland Press, Amsterdam (1988)Google Scholar
  31. 31.
    Hart, S.G.: NASA-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50, 904–908 (2006)CrossRefGoogle Scholar
  32. 32.
    Brooke, J.: SUS: A “quick and dirty” usability scale. In: Jordan, P.W., Thomas, B., Weerdmeester, B., McClelland, A.L. (eds.) Usability Evaluation in Industry, pp. 189–194. Taylor & Francis, London (1996)Google Scholar
  33. 33.
    SPSS Inc., PASW Statistics 18 Core System User ’s Guide. SPSS Inc. (2007)Google Scholar
  34. 34.
    Weinstein, C.E., Mayer, R.E.: The teaching of learning strategies. In: Wittrock, M. (ed.) Handbook of Research on Teaching, pp. 315–327. Macmillan, New York (1986)Google Scholar
  35. 35.
    Pressley, M.: More about the development of self-regulation: Complex, long-term, and thoroughly social. Educational Psychologist 30, 207–212 (1995)CrossRefGoogle Scholar
  36. 36.
    Hetzner, S., Steiner, C., Dimitrova, V., Brna, P., Conlan, O.: Adult Self-regulated Learning through Linking Experience in Simulated and Real World: A Holistic Approach. In: Kloos, C.D., Gillet, D., Garcia, R.M.C., Wild, F., Wolpers, M. (eds.) EC-TEL 2011. LNCS, vol. 6964, pp. 166–180. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  37. 37.
    Vollmeyer, R., Rheinberg, F.: Motivational effects on self-regulated learning with different tasks. Educational Psychology Review 18, 239–253 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Marcel Berthold
    • 1
  • Adam Moore
    • 2
  • Christina M. Steiner
    • 1
  • Conor Gaffney
    • 3
  • Declan Dagger
    • 3
  • Dietrich Albert
    • 1
  • Fionn Kelly
    • 4
  • Gary Donohoe
    • 4
  • Gordon Power
    • 3
  • Owen Conlan
    • 2
  1. 1.Knowledge Management InstituteGraz University of TechnologyGrazAustria
  2. 2.Knowledge, Data & Engineering Group, School of Computer Science and StatisticsTrinity CollegeDublinIreland
  3. 3.EmpowerTheUser, Trinity Technology & Enterprise CampusDublinIreland
  4. 4.Department of Psychiatry, School of MedicineTrinity CollegeDublinIreland

Personalised recommendations