Assessing Problem-Solving Skills in Game-Based Immersive Environments

  • Valerie J. ShuteEmail author
  • Benjamin Emihovich
Reference work entry
Part of the Springer International Handbooks of Education book series (SIHE)


According to Pearson’s Law: “That which is measured improves.” But some important constructs, like problem-solving skill, are generally viewed as very difficult to measure for a variety of reasons (e.g., lack of a clear and agreed-upon definition, psychological and/or statistical multidimensionality of the construct, subjectivity of scoring, and so on). Adding to the challenge of validly measuring problem-solving skill is that formal education settings tend to focus only on well-structured problems – those that have correct and incorrect answers. However, these problems tend to have little transfer to the real world. In short, there is a gap in the kinds of problems being assessed and taught in schools and those desired in workplace environments. In this chapter, we focus on how game-based immersive environments, such as well-designed video games, can be used to both measure and promote the development of problem-solving skills in formal education settings. In this chapter, we discuss the theoretical foundations of problem-solving and provide a worked example of assessing it from gameplay using a well-designed video game. We conclude the chapter with a discussion on future implications for using game-based environments to measure and promote problem-solving skills in education.


Assessment Game-based learning Problem-solving skill Stealth assessment Video games 


  1. Ackerman, P. L., Kanfer, R., & Calderwood, C. (2010). Use it or lose it? Wii brain exercise practice and reading for domain knowledge. Psychology and Aging, 25(4), 753–766.CrossRefGoogle Scholar
  2. Almond, R. G. (2010). Using evidence centered design to think about assessments. In V. J. Shute & B. J. Becker (Eds.), Innovative assessment for the 21st century: Supporting educational needs (pp. 75–100). New York: Springer.CrossRefGoogle Scholar
  3. Almond, R. G., Shute, V. J., Underwood, J. S., & Zapata-Rivera, D. (2009). Bayesian networks: A teacher’s view. International Journal of Approximate Reasoning, 50, 450–460.CrossRefGoogle Scholar
  4. Anderson, J. R. (1980). Cognitive psychology and its implications. New York: Freeman.Google Scholar
  5. Baniqued, P. L., Kranz, M. B., Voss, M. W., Lee, H., Cosman, J. D., Severson, J., & Kramer, A. F. (2013). Cognitive training with casual video games: Points to consider. Frontiers in Psychology, 4, 1010.Google Scholar
  6. Barab, S., & Dede, C. (2007). Games and immersive participatory simulations for science education: An emerging type of curricula. Journal of Science Education and Technology, 16(1), 1–3.CrossRefGoogle Scholar
  7. Barzilai, S., & Blau, I. (2014). Scaffolding game-based learning: Impact on learning achievements, perceived learning, and game experiences. Computers & Education, 70, 65–79.CrossRefGoogle Scholar
  8. Boot, W. R., Kramer, A. F., Simons, D. J., Fabiani, M., & Gratton, G. (2008). The effects of video game playing on attention, memory, and executive control. Acta Psychologica, 129(3), 387–398.CrossRefGoogle Scholar
  9. Bransford, J. D., & Stein, B. S. (1984). The IDEAL problem solver. A guide for improving thinking, learning, and creativity. New York: Freeman.Google Scholar
  10. Chuang, T.-Y., & Chen, W.-F. (2009). Effect of computer-based video games on children: An experimental study. Educational Technology & Society, 12(2), 1–10.Google Scholar
  11. Clark, D. B., Tanner-Smith, E., & Killingsworth, S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86(1), 79–122. Scholar
  12. Cukier, K. N., & Mayer-Schoenberger, V. (2013). The rise of Big Data: How it’s changing the way we think about the world. Foreign Affairs, 92(3). Retrieved 19 Aug 2017 from
  13. Dede, C. (2005). Planning for neomillennial learning styles. EDUCAUSE Quarterly, 28(1), 7–12.Google Scholar
  14. Dede, C. (2009). Immersive interfaces for engagement and learning. Science, 323(66), 66–69.CrossRefGoogle Scholar
  15. DiCerbo, K. E., & Behrens, J. T. (2012). Implications of the digital ocean on current and future assessment. In R. Lissitz & H. Jiao (Eds.), Computers and their impact on state assessment: Recent history and predictions for the future (pp. 273–306). Charlotte: Information Age Publishing.Google Scholar
  16. Duncker, K. (1945). The structure and dynamics of problem-solving processes. Psychological Monographs, 58(5), 1–112.CrossRefGoogle Scholar
  17. Entertainment Software Association. (2016). 2016 Essential facts about the computer and video game industry. The Entertainment Software Association. Retrieved from
  18. Gagné, R. M. (1980). Learnable aspects of problem solving. Educational Psychologist, 15(2), 84–92.CrossRefGoogle Scholar
  19. Gagné, R. M., Wager, W. W., Golas, K. C., & Keller, J. M. (2005). Principles of instructional design (5th ed.). Belmont: Wadsworth/Thomson Learning.Google Scholar
  20. Gee, J. P. (2007). Games and learning: Issues, perils and potentials. In J. P. Gee (Ed.), Good video games and good learning: Collected essays on video games, learning and literacy (pp. 129–174). New York: Palgrave Macmillan.CrossRefGoogle Scholar
  21. Gee, J. P. (2008). Learning and games. In K. Salen (Ed.), The ecology of games: Connecting youth, games, and learning (pp. 21–40). Cambridge, MA: MIT Press.Google Scholar
  22. Gee, J. P., & Hayes, E. (2011). Nurturing affinity spaces and game-based learning. In C. Steinkuehler, K. Squire, & S. Barab (Eds.), Games, learning, and society: Learning and meaning in the digital age (pp. 129–153). New York: Cambridge University Press.Google Scholar
  23. Gick, M. L. (1986). Problem-solving strategies. Educational Psychologist, 21(1–2), 99–120.CrossRefGoogle Scholar
  24. Gobert, J., Sao Pedro, M., Raziuddin, J., & Baker, R. (2013). From log files to assessment metrics for science inquiry using educational data mining. Journal of the Learning Sciences, 22(4), 521–563.CrossRefGoogle Scholar
  25. Green, C. S., & Bavelier, D. (2012). Learning, attentional control, and action video games. Current Biology, 22(6), 197–206.CrossRefGoogle Scholar
  26. Hart Research Associates. (2015). Falling short? College learning and career success. Washington, DC: Association of American Colleges and Universities.Google Scholar
  27. Ifenthaler, D., Eseryel, D., & Ge, X. (2012). Assessment for game-based learning. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning. Foundations, innovations, and perspectives (pp. 3–10). New York: Springer.CrossRefGoogle Scholar
  28. Jackson, L. A., Witt, E. A., Games, A. I., Fitzgerald, H. E., von Eye, A., & Zhao, Y. (2012). Information technology use and creativity: Findings from the Children and Technology Project. Computers in Human Behavior, 28, 370–376. Scholar
  29. Jonassen, D. H. (1997). Instructional design models for well-structured and III-structured problem-solving learning outcomes. Educational Technology Research and Development, 45(1), 65–94.CrossRefGoogle Scholar
  30. Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research and Development, 48(4), 63–85.CrossRefGoogle Scholar
  31. Jonassen, D. H. (2002). Engaging and supporting problem solving in online learning. Quarterly Review of Distance Education, 3(1), 1–13.Google Scholar
  32. Jonassen, D. (2003). Using cognitive tools to represent problems. Journal of Research on Technology in Education, 35(3), 362–381.CrossRefGoogle Scholar
  33. Krulak, C. (1997). Military thinking and decision-making exercises (No. 1500.55). Washington, DC. Retrieved 19 Aug 2017 from
  34. Lanyon, R. I., & Goodstein, L. D. (1997). Personality assessment (3rd ed.). New York: Wiley.Google Scholar
  35. Lessiter, J., Freeman, J., Keogh, E., & Davidoff, J. (2001). A cross-media presence questionnaire: The ITC-sense of presence inventory. Presence, 10(3), 282–297.CrossRefGoogle Scholar
  36. Mayer, R. E. (1992). Thinking, problem solving, cognition (2nd ed.). New York: Freeman.Google Scholar
  37. Mayer, R., & Wittrock, M. (2006). Problem solving. In P. Alexander & P. Winne (Eds.), Handbook of educational psychology (2nd ed., pp. 287–303). Mahwah: Erlbaum.Google Scholar
  38. Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessment. Measurement: Interdisciplinary Research and Perspective, 1(1), 3–62.Google Scholar
  39. Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41, 49–100.CrossRefGoogle Scholar
  40. Newell, A., Shaw, J. C., & Simon, H. A. (1958). Elements of a theory of human problem solving. Psychological Review, 65(3), 151–166.CrossRefGoogle Scholar
  41. Pagliery, J. (2014). Why I put World of Warcraft on my resume. Retrieved 26 Nov 2016 from
  42. Partnership for 21st Century Learning. (2016). Retrieved 21 Nov 2016 from
  43. Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes: Measures of social psychological attitudes (Vol. 1, pp. 17–59). San Diego: Academic.CrossRefGoogle Scholar
  44. Polya, G. (1945). How to solve it: A new aspect of mathematical method. Princeton: Princeton University Press.Google Scholar
  45. Quellmalz, E. S., Timms, M. J., Silberglitt, M. D., & Buckley, B. C. (2012). Science assessments for all: Integrating science simulations into balanced state science assessment systems. Journal of Research in Science Teaching, 49(3), 363–393.CrossRefGoogle Scholar
  46. Rowe, J. P., Shores, L. R., Mott, B. W., & Lester, J. C. (2011). Integrating learning, problem solving, and engagement in narrative-centered learning environments. International Journal of Artificial Intelligence in Education, 21(1), 115–133.Google Scholar
  47. Schrader, P., Zheng, D., & Young, M. (2006). Teachers’ perceptions of video games: MMOGs and the future of preservice teacher education. Innovate, 2(3). Retrieved April 17, 2018 from
  48. Schweizer, F., Wüstenberg, S., & Greiff, S. (2013). Validity of the MicroDYN approach: Complex problem solving predicts school grades beyond working memory capacity. Learning and Individual Differences, 24, 42–52.CrossRefGoogle Scholar
  49. Selfe, C. L., & Hawisher, G. E. (2004). Literate lives in the information age: Narratives on literacy from the United States. Mahwah: Erlbaum.Google Scholar
  50. Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 503–524). Charlotte: Information Age Publishers.Google Scholar
  51. Shute, V. J., & Ventura, M. (2013). Measuring and supporting learning in games: Stealth assessment. Cambridge, MA: MIT Press.Google Scholar
  52. Shute, V. J., & Wang, L. (2016). Assessing and supporting hard-to-measure constructs. In A. A. Rupp & J. P. Leighton (Eds.), The handbook of cognition and assessment: Frameworks, methodologies, and application (pp. 535–562). Hoboken: Wiley.CrossRefGoogle Scholar
  53. Shute, V. J., Ventura, M., & Ke, F. (2015a). The power of play: The effects of Portal 2 and Lumosity on cognitive and noncognitive skills. Computers & Education, 80, 58–67.CrossRefGoogle Scholar
  54. Shute, V. J., D’Mello, S. K., Baker, R., Bosch, N., Ocumpaugh, J., Ventura, M., & Almeda, V. (2015b). Modeling how incoming knowledge, persistence, affective states, and in-game progress influence student learning from an educational game. Computers & Education, 86, 224–235.CrossRefGoogle Scholar
  55. Shute, V. J., Wang, L., Greiff, S., Zhao, W., & Moore, G. R. (2016). Measuring problem solving skills via stealth assessment in an engaging video game. Computers in Human Behavior, 63, 106–117.CrossRefGoogle Scholar
  56. Shute, V. J., Ke, F., & Wang, L. (2017). Assessment and adaptation in games. In P. Wouters & H. van Oostendorp (Eds.), Instructional techniques to facilitate learning and motivation of serious games (pp. 59–78). New York: Springer.CrossRefGoogle Scholar
  57. Simon, H. A., & Newell, A. (1971). Human problem solving: The state of the theory in 1970. American Psychologist, 26(2), 145–159.CrossRefGoogle Scholar
  58. Sitzmann, T. (2011). A meta-analysis of self-regulated learning in work-related training and educational attainment: What we know and where we need to go. Psychological Bulletin, 137, 421–442.CrossRefGoogle Scholar
  59. Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Psychological Review, Monograph Supplement, 2(4), 1–8.Google Scholar
  60. Van Eck, R. (2006). Digital game-based learning: It’s not just the digital natives who are restless. EDUCAUSE review, 41(2), 1–16.Google Scholar
  61. Van Eck, R., Hung, W., Bowman, R., & Love, S. (2009). 21st century game design: A model and prototype for promoting scientific problem solving. In Proceedings of the twelfth IASTED international conference on computers and advanced technology in education: Globalization of education through advanced technology. Calgary: ACTA Press.Google Scholar
  62. Van Eck, R. N., Shute, V. J., & Rieber, L. P. (2017). Leveling up: Game design research and practice for instructional designers. In R. Reiser & J. Dempsey (Eds.), Trends and issues in instructional design and technology (4rd ed., pp. 227–285). Upper Saddle River: Pearson Education.Google Scholar
  63. Ventura, M., Shute, V. J., Wright, T., & Zhao, W. (2013). An investigation of the validity of the virtual spatial navigation assessment. Frontiers in Psychology, 4, 1–7.CrossRefGoogle Scholar
  64. Wouters, P. J. M., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105, 249–265.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Florida State UniversityTallahasseeUSA

Section editors and affiliations

  • Mary Webb
    • 1
  • Dirk Ifenthaler
    • 2
  1. 1.King's College LondonLondonUK
  2. 2.University of MannheimMannheimGermany

Personalised recommendations