Evaluating Serious Game Trainings

  • P. Saskia BayerlEmail author
  • Steffi Davey
  • Philipp Lohrmann
  • Jonathan Saunders
Part of the Security Informatics and Law Enforcement book series (SILE)


Developing and conducting serious game-based trainings can consume considerable resources and time. This chapter offers recommendations on how to plan and conduct evaluations of virtual reality (VR)-based serious games for training purposes. A concrete example of such an evaluation from the AUGGMED project showcases the practical steps of running meaningful VR-based training evaluations as well as potential challenges that need to be taken into account.


Training evaluations Evaluation designs Serious games Virtual reality Learning outcomes 


  1. Alfalah, S., Falah, J., Alfalah, T., Elfalah, M., Muhaidat, N., & Falah, O. (2019). A comparative study between a virtual reality heart anatomy system and traditional medical teaching modalities. Virtual Reality, 23(3), 229–234.
  2. Arthur Jr., W., Bennett Jr., W., Stanush, P., & McNelly, T. (1998). Factors that influence skill decay and retention: A quantitative review and analysis. Human Performance, 11(1), 57–101.CrossRefGoogle Scholar
  3. Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41, 63–105.CrossRefGoogle Scholar
  4. Bevan, N. (2001). International standards for HCI and usability. International Journal of Human-Computer Studies, 55(4), 533–552.CrossRefGoogle Scholar
  5. Blume, B., Ford, J. K., Baldwin, T. T., & Huang, J. L. (2010). Transfer of training: A meta-analytic review. Journal of Management, 36(4), 1065–1105.CrossRefGoogle Scholar
  6. Burke, L. A. & Hutchins, H. M. (2007). Training transfer: An integrative literature review. Human Resource Development Review, 6(3), 263–296.Google Scholar
  7. CAIPE (1999). Evaluating interprofessional education: A United Kingdom review for health and social care. The British Educational Research Association and The United Kingdom Centre for the Advancement of Interprofessional Education, London.Google Scholar
  8. Carpenter, J. (2011). Evaluating social work education: A review of outcomes, measures, research designs and practicalities. Social Work Education, 30 (2), 122–140.Google Scholar
  9. Fagan, K. (2018). Here’s what happens to your body when you’ve been in virtual reality for too long. [Online]. Available at: (Accessed 4 July 2019).
  10. Feng, Z., Gonzales, V., Amor, R., Lovreglio, R., & Cabrera-Guerrero, G. (2018). Immersive virtual reality serious games for evacuation training and research: A systematic literature review. Computers & Education, 127, 252–266.CrossRefGoogle Scholar
  11. Haferkamp, N., Kraemer, N., Linehan, C., & Schembri, M. (2011). Training disaster communication by means of serious games in virtual environments. Entertainment Computing, 2, 81–88.CrossRefGoogle Scholar
  12. Hanley, P., Chambers, B., & Haslam, J. (2016). Reassessing RCTs as the ‘gold standard’: Synergy not separatism in evaluation designs. International Journal of Research & Method in Education, 39(3), 287–298.CrossRefGoogle Scholar
  13. Howard, M., & Jacobs, R. (2016). The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): Two novel evaluation methods for developing optimal training programs. Journal of Organizational Behavior, 37, 1246–1270.CrossRefGoogle Scholar
  14. Jin, G., Tu, M., Kim, T.-H., Heffron, J., & White, J. (2018). Game based cybersecurity training for high school students. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE ‘18) (pp. 68–73). New York: ACM.Google Scholar
  15. Joël, M., Pu, Z., Wiegert, O., Oitzl, M., & Krugers, H. (2006). Learning under stress: How does it work? Trends in Cognitive Sciences, 10(4), 152–158.CrossRefGoogle Scholar
  16. Kim, J., Ritter, F., & Koubek, R. (2013). An integrated theory for improved skill acquisition and retention in the three stages of learning. Theoretical Issues in Ergonomics Science, 14(1), 22–37.CrossRefGoogle Scholar
  17. Kirkpatrick, D. L. (1967). Evaluation of training. In R. L. Craig & L. R. Bittel (Eds.), Training and development handbook (pp. 87–112). New York: McGraw-Hill.Google Scholar
  18. LaViola Jr., J. J. (2000). A discussion of cybersickness in virtual environments. ACM SIGCHI Bulletin, 32(1), 47–56.CrossRefGoogle Scholar
  19. Mancuso, D. S., Chlup, D. T., & McWhorter, R. R. (2010). A study of adult learning in a virtual world. Advances in Developing Human Resources, 12(6), 681–699.CrossRefGoogle Scholar
  20. Morina, N., Ijntemaa, H., Meyerbröker, K., & Emmelkamp, P. (2015). Can virtual reality exposure therapy gains be generalized to real-life? A meta-analysis of studies applying behavioral assessments. Behavior Research and Therapy, 17, 18–24.CrossRefGoogle Scholar
  21. Overall, J. E., Tonidandel, S., & Schmitz, J. M. (2009). Testing the significance of difference in average rates of change in controlled longitudinal studies with high dropout rates. Methodology, 5(2), 46–54.CrossRefGoogle Scholar
  22. Ricci, K. E., Salas, E., & Cannon-Bowers, J. A. (1996). Do computer-based games facilitate knowledge acquisition and retention? Military Psychology, 8, 295–307.CrossRefGoogle Scholar
  23. Sackett, P., & Mullen, E. (1993). Beyond formal experimental design: Towards an expanded view of the training evaluation process. Personnel Psychology, 46, 613–627.CrossRefGoogle Scholar
  24. Saks, A. M. (2002). So what is a good transfer of training estimate? A reply to Fitzpatrick. The Industrial-Organizational Psychologist, 39, 29–30.Google Scholar
  25. Sanchez-Vives, M., & Slater, M. V. (2005). From presence to consciousness through virtual reality. Nature Reviews Neuroscience, 6(4), 332–339.CrossRefGoogle Scholar
  26. Saunders, J., Davey, S., Bayerl, P. S., & Lohrmann, P. (2019). Validating virtual reality as an effective training medium in the security domain. In Proceedings of the 2019 IEEEVR Conference.Google Scholar
  27. Schendel, J. D., & Hagman, J. D. (1991). Long-term retention of motor skills. In J. E. Morrison (Ed.), Training for performance: Principles of applied human learning (pp. 53–92). New York: Wiley and Sons.Google Scholar
  28. Schmidt, S. W. (2007). The relationship between satisfaction with workplace training and overall job satisfaction. Human Resource Development Quarterly, 18(4), 481–498.CrossRefGoogle Scholar
  29. Slater, M., Khanna, P., Mortensen, J., & Yu, I. (2009). Visual realism enhances realistic response in an immersive virtual environment. IEEE Computer Graphics and Applications, 29(3), 76–84.CrossRefGoogle Scholar
  30. Weech, S., Kenny, S., & Barnett-Cowan, M. (2019). Presence and cybersickness in virtual reality are negatively related: A review. Frontiers in Psychology, 10, 158.CrossRefGoogle Scholar
  31. Wei, L., Zhou, H., & Nahavandi, S. (2019). Haptically enabled simulation system for firearm shooting training. Virtual Reality, 23(3), 217–228. Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • P. Saskia Bayerl
    • 1
    Email author
  • Steffi Davey
    • 1
  • Philipp Lohrmann
    • 2
  • Jonathan Saunders
    • 1
  1. 1.CENTRIC, Sheffield Hallam UniversitySheffieldUK
  2. 2.ROQAS Solutions LimitedHornchurchUK

Personalised recommendations