Advertisement

Examining Designed Experiences: A Walkthrough for Understanding Video Games as Performance Assessments

  • Michael P. McCreeryEmail author
  • P. G. Schrader
  • S. Kathleen Krach
  • Jeffrey R. Laferriere
  • Catherine A. Bacos
  • Joseph P. Fiorentini
Chapter
Part of the Advances in Game-Based Learning book series (AGBL)

Abstract

Although there is an extensive body of video game research, very few resources exist that outline the potential for games to provide process-oriented data related to human behavior and learning. The current chapter offers guidance for researchers to extract dynamic, emergent, and complex data from video game contexts, and thus unlock the potential for games to function as performance assessments. This chapter describes how to leverage behavioral observation as a means to examine player interactions that are associated with knowledge acquisition, formative (within-game) activities, and summative (end-game) outcomes. Specifically, through employing a moral-choice video game as an example, we walk through deconstructing the game, defining game elements, and the construction of a behavioral observation protocol. Although some games aren’t well-suited for this approach, we provide a framework for how in-game player interactions can be captured, coded, and analyzed. Each step is used to illustrate how games can function as performance assessments.

Keywords

Video games Performance assessment Observable behavior 

References

  1. Alevizos, P., DeRisi, W., Liberman, R., Eckman, T., & Callahan, E. (1978). The behavior observation instrument: A method of direct observation for program evaluation. Journal of Applied Behavior Analysis, 11(2), 243–257.  https://doi.org/10.1901/jaba.1978.11-243CrossRefGoogle Scholar
  2. Anetta, L. A., Minogue, J., Holmes, S. Y., & Cheng, M. (2009). Investigating the impact of video games on high school students’ engagement and learning about genetics. Computers & Education, 53, 74–85.CrossRefGoogle Scholar
  3. de Freitas, S. (2018). Are games effective learning tools? A review of educational games. Journal of Educational Technology & Society, 21(2), 74–84. Retrieved from http://www.jstor.org/stable/26388380Google Scholar
  4. Dewey, J. (1899). The school and society. Chicago, IL: University of Chicago Press.Google Scholar
  5. Fortes Tondello, G., Valtchanov, D., Reetz, A., Wehbe, R. R., Orji, R., & Nacke, L. E. (2018). Towards a trait model of video game preferences. International Journal of Human–Computer Interaction, 34(8), 732–748.CrossRefGoogle Scholar
  6. Govaerts, M. J. B., Van der Vleuten, C. P. M., Schuwirth, L. W. T., & Muijtjens, A. M. M. (2007). Broadening perspectives on clinical performance assessment: Rethinking the nature of in-training assessment. Advances in Health Sciences Education, 12(2), 239–260.  https://doi.org/10.1007/s10459-006-9043-1CrossRefGoogle Scholar
  7. Grab the Games. (2015). The deed [Computer software]. Retrieved from http://store.steampowered.com/app/420740/The_Deed/
  8. Harvianinen, J. T., Lainema, T., & Saarinen, E. (2014). Player-reported impediments to game- based learning. Transactions of the Digital Games Research Association, 1(2), 1–11. Retrieved June 14, 2018, from http://todigra.org/index.php/todigra/article/view/14/21Google Scholar
  9. Jabbari, N., & Eslami, Z. R. (2019). Second language learning in the context of massively multiplayer online games: A scoping review. ReCALL, 31(1), 92–113.CrossRefGoogle Scholar
  10. Kline, T. (2005). Psychological testing: A practical approach to design and evaluation. Thousand Oaks, CA: Sage.Google Scholar
  11. Little, T. D., Bovaird, J. A., & Slegers, D. W. (2006). Methods for the analysis of change. In D. K. Mroczek & T. D. Little (Eds.), Handbook of personality development (pp. 181–211). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.Google Scholar
  12. Mancini, T., Imperato, C., & Sibilla, F. (2019). Does avatar’s character and emotional bond expose to gaming addiction? Two studies on virtual self-discrepancy, avatar identification and gaming addiction in massively multiplayer online role-playing game players. Computers in Human Behavior, 92, 297–305.CrossRefGoogle Scholar
  13. McCreery, M. P., Krach, S. K., Schrader, P. G., & Boone, R. (2012). Defining the virtual self: Personality, behavior and the psychology of embodiment. Computers in Human Behavior, 28(3), 976–983.  https://doi.org/10.1016/j.chb.2011.12.019CrossRefGoogle Scholar
  14. McCreery, M. P., Laferriere, J., Bacos, C. A., & Krach, S. K. (2018). Video games as a model for assessing learning behaviors. In American Educational Research Association Annual Conference, New York, NY.Google Scholar
  15. McCreery, M. P., Schrader, P. G., & Krach, S. K. (2011). Navigating massively multiplayer online games: Evaluating 21st century skills for learning within virtual environments. Journal of Educational Computing Research, 44(4), 473–493.  https://doi.org/10.2190/EC.44.4.fCrossRefGoogle Scholar
  16. McCreery, M. P., Vallett, D., & Clark, C. (2015). Social interaction in a virtual environment: Examining sociospatial interactivity and social presence using behavioral analytics. Computers in Human Behavior, 51, 203–206.  https://doi.org/10.1016/j.chb.2015.04.044CrossRefGoogle Scholar
  17. McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282. Retrieved from https://hrcak.srce.hr/89395CrossRefGoogle Scholar
  18. Milne, D. (Ed.). (2015). Evaluating mental health practice (psychology revivals): Methods and applications. London, UK: Routledge.Google Scholar
  19. Moore, M. G. (1993). Theory of transactional distance. In Theoretical principles of distance education (Vol. 1, pp. 22–38). London, UK: Routledge.Google Scholar
  20. Schrader, P. G., Deniz, H., & Keilty, J. (2016). Breaking SPORE: Building instructional value in science education using a commercial, off-the shelf game. Journal of Learning and Teaching in Digital Age, 1(1), 32–46. Retrieved from http://joltida.org/index.php/joltida/article/view/6/54Google Scholar
  21. Schrader, P. G., & McCreery, M. (2008). The acquisition of skill and expertise in massively multiplayer online games. Educational Technology Research & Development, 56(5–6), 557–574.CrossRefGoogle Scholar
  22. Schrader, P. G., & McCreery, M. (2012). Are all games the same? Examining three frameworks for assessing learning from, with, and in games. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning: Foundations, innovations, and perspectives (pp. 11–28). New York, NY: Springer.CrossRefGoogle Scholar
  23. Schrader, P. G., McCreery, M. P., & Vallett, D. (2017). Performance in situ. In Y. K. Baek (Ed.), Game-based learning: Theory, strategies and performance outcomes (pp. 421–438). Hauppauge, NY: Nova Science Publishers.Google Scholar
  24. Shaffer, D. W., Squire, K. R., Halverson, R., & Gee, J. P. (2005). Video games and the future of learning. Phi Delta Kappan, 87(2), 104–111.  https://doi.org/10.1177/003172170508700205CrossRefGoogle Scholar
  25. Shavelson, R., Baxter, G., & Gao, X. (1993). Sampling variability of performance assessments. Journal of Educational Measurement, 30(3), 215–232. Retrieved from http://www.jstor.org/stable/1435044CrossRefGoogle Scholar
  26. Shute, V. J., Ke, F., & Wang, L. (2017). Assessment and adaptation in games. In P. Wouters & H. van Oostendorp (Eds.), Instructional techniques to facilitate learning and motivation of serious games. Cham, Switzerland: Springer International Publishing.  https://doi.org/10.1007/978-3-319-39298-1_4CrossRefGoogle Scholar
  27. Shute, V. J., Leighton, J. P., Jang, E. E., & Chu, M. W. (2016). Advances in the science of assessment. Educational Assessment, 21(1), 34–59.CrossRefGoogle Scholar
  28. Squire, K. D. (2011). Video games and learning: Teaching and participatory culture in the digital age. New York, NY: Teachers College Press.Google Scholar
  29. Stefanidis, K., Psaltis, A., Apostolakis, K. C., Dimitropoulos, K., & Daras, P. (2019). Learning prosocial skills through multiadaptive games: A case study. Journal of Computers in Education, 6, 167–190.CrossRefGoogle Scholar
  30. Tapp, J., Wehby, J., & Ellis, D. (1995). A multiple option observation system for experimental studies: MOOSES. Behavior Research Methods, Instruments, & Computers, 27(1), 25–31.CrossRefGoogle Scholar
  31. Vallett, D. (2016). Soft failure in educational and learning games. In R. Lamb & D. McMahon (Eds.), Educational and learning games: New research. New York, NY: Nova Science.Google Scholar
  32. Wang, L., Shute, V., & Moore, G. R. (2015). Lessons learned and best practices of stealth assessment. International Journal of Gaming and Computer-Mediated Simulations, 7(4), 66–87.CrossRefGoogle Scholar
  33. Whiteside, A. L., & Garrett Dikkers, A. (2012). Using the Social Presence Model to maximize interactions in online environments. In K. S. Amant & S. Kelsey (Eds.), Computer-mediated communication across cultures: International interactions in online environments. Hershey, PA: IGI Global.Google Scholar
  34. Wouters, P., Van Nimwegen, C., Van Oostendorp, H., & Van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249–265.  https://doi.org/10.1037/a0031311CrossRefGoogle Scholar
  35. Yen, W. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30(3), 187–213.  https://doi.org/10.1111/j.1745-3984.1993.tb00423.xCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Michael P. McCreery
    • 1
    Email author
  • P. G. Schrader
    • 1
  • S. Kathleen Krach
    • 2
  • Jeffrey R. Laferriere
    • 3
  • Catherine A. Bacos
    • 1
  • Joseph P. Fiorentini
    • 1
  1. 1.University of Nevada, Las VegasLas VegasUSA
  2. 2.Florida State UniversityTallahasseeUSA
  3. 3.Lebanon Valley CollegeAnnvilleUSA

Personalised recommendations