Advertisement

Alternative Assessment Strategies for Complex Problem Solving in Game-Based Learning Environments

  • Deniz EseryelEmail author
  • Dirk Ifenthaler
  • Xun Ge
Chapter

Abstract

The central thesis of this chapter is that emerging technologies such as digital games compel educators, educational researchers, and instructional designers to conceptualize learning, instruction, and assessment in fundamentally different ways. New technologies, including massively multi-player digital games offer new opportunities for learning and instruction; however, there is as yet insufficient evidence to support sustained impact on learning and instruction, apart from the case of military training based on large simulated war games. Technologically sophisticated design and assessment frameworks are likely to facilitate progress in this area, and that is our focus in this chapter. Specifically, we provide an integrated framework for assessing complex problem solving in digital game-based learning in the context of a longitudinal design-based research study.

Keywords

Game-based learning Assessment Complex problem-solving 

References

  1. Almond, R. G., Steinberg, L. S., & Mislevy, R. J. (2002). Enhancing the design and delivery of assessment systems: A four process architecture. Journal of Technology, Learning, and Assessment, 1(5), 3–63.Google Scholar
  2. Baker, E. L., Chung, G. K. W. K., & Delacruz, G. C. (2008). Design and validation of technology-based performance assessments. In J. M. Spector, M. D. Merrill, J. van Merrienboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (pp. 595–604). New York: Taylor & Francis Group.Google Scholar
  3. Barab, S. A., & Duffy, T. (2000). From practice fields to communities of practice. In D. H. Jonassen & S. M. Land (Eds.), Theoretical foundations of learning environments (pp. 25–56). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  4. Barab, S. A., Tomas, M., Dodge, T., Carteaux, R., & Tuzun, H. (2005). Making learning fun: Quest Atlantis, a game without guns. Educational Technology Research and Development, 53(1), 86–107.CrossRefGoogle Scholar
  5. Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42.Google Scholar
  6. Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6(3), 271–315.CrossRefGoogle Scholar
  7. Chi, M. T. H., & Glaser, R. (1985). Problem solving ability. In R. J. Sternberg (Ed.), Human abilities: An information processing approach (pp. 227–257). San Francisco: W. H. Freeman & Co.Google Scholar
  8. Chin, J., Dukes, R., & Gamson, W. (2009). Assessment in simulation and gaming: A review of the last 40 years. Simulation & Gaming, 40, 553–568.CrossRefGoogle Scholar
  9. Chung, G. K. W. K., & Baker, E. L. (2003). An exploratory study to examine the feasibility of measuring problem-solving processes using a click-through interface. Journal of Technology, Learning and Assessment, 2(2). Retrieved April 22, 2010, from http://www.jtla.org.
  10. Dickey, M. D. (2005). Engaging by design: How engagement strategies in popular computer and video games can inform instructional design. Educational Technology Research and Development, 53(2), 67–83.CrossRefGoogle Scholar
  11. Dummer, P., & Ifenthaler, D. (2005). Planning and assessing navigation in model-centered learning environments. Why learners often do not follow the path laid out for them. In G. Chiazzese, M. Allegra, A. Chifari, & S. Ottaviano (Eds.), Methods and technologies for learning (pp. 327–334). Sothhampton: WIT Press.Google Scholar
  12. Dunkle, M. E., Schraw, G., & Bendixen, L. D. (1995). Cognitive processes in well-structured and ill-defined problem solving. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA.Google Scholar
  13. Eigler, G. (2005). Textproduzieren als Wissensnutzungs- und Wissenserwerbsstrategie. In H. Mandl & H. F. Friedrich (Eds.), Handbuch Lernstrategien (pp. 187–205). Göttingen: Hogrefe.Google Scholar
  14. Eliaa, I., Gagatsisa, A., & Demetriou, A. (2007). The effects of different modes of representation on the solution of one-step additive problems. Learning and Instruction, 17(6), 658–672.CrossRefGoogle Scholar
  15. Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215–251.CrossRefGoogle Scholar
  16. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.Google Scholar
  17. Eseryel, D. (2006). Expert conceptualizations of the domain of instructional design: An investigative study on the DEEP assessment methodology for complex problem-solving outcomes. Unpublished doctoral dissertation, Syracuse University.Google Scholar
  18. Eseryel, D., Ge, X., Law, V., Hayes, T., Guo, Y., & Ifenthaler, D. (2009). The effects of an educational massively multiplayer online game on students’ complex problem solving skill acquisition. Paper presented at the Annual Convention of Association for Educational Communications and Technology, Louisville, KY.Google Scholar
  19. Funke, J. (1985). Steuerung dynamischer Prozesse durch Aufbau und Anwendung subjektiver Kausalmodelle. Zeitschrift für Psychologie, 193(4), 443–465.Google Scholar
  20. Galarneau, L., & Zibit, M. (2007). Online games for 21st century skills. In D. Gibson, C. Aldrich, & M. Prensky (Eds.), Games and simulations in online learning: Research and development frameworks (pp. 59–88). Hershey, PA: Information Science Publishing.Google Scholar
  21. Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York: Palgrave-Macmillan.Google Scholar
  22. Gick, M. L. (1986). Problem-solving strategies. Educational Psychologist, 21(1&2), 99–120.CrossRefGoogle Scholar
  23. Hardy, I., & Stadelhofer, B. (2006). Concept Maps wirkungsvoll als Strukturierungshilfen einsetzen. Welche Rolle spielt die Selbstkonstruktion? Zeitschrift für Pädagogische Psychologie, 20(3), 175–187.CrossRefGoogle Scholar
  24. Herl, H. E., Baker, E. L., & Niemi, D. (1996). Construct validation of an approach to modeling cognitive structure of U.S. history knowledge. Journal of Educational Research, 89(4), 206–218.CrossRefGoogle Scholar
  25. Hong, N. S., Jonassen, D. H., & McGee, S. (2003). Predictors of well-structured and ill-structured problem solving in an astronomy simulation. Journal of Research in Science Teaching, 40(1), 6–33.CrossRefGoogle Scholar
  26. Ifenthaler, D. (2008). Practical solutions for the diagnosis of progressing mental models. In D. Ifenthaler, P. Pirnay-Dummer, & J. M. Spector (Eds.), Understanding models for learning and instruction. Essays in honor of Norbert M. Seel (pp. 43–61). New York: Springer.CrossRefGoogle Scholar
  27. Ifenthaler, D. (2009a). Bridging the gap between expert-novice differences: The model-based feedback approach. In Kinshuk, D. G. Sampson, J. M. Spector, P. Isaias, & D. Ifenthaler (Eds.), Proceedings of the IADIS international conference on cognition and exploratory learning in the digital age (pp. 49–60), Rome: IADIS Press.Google Scholar
  28. Ifenthaler, D. (2009b). Model-based feedback for improving expertise and expert performance. Technology, Instruction, Cognition and Learning, 7(2), 83–101.Google Scholar
  29. Ifenthaler, D. (2009c). Using a causal model for the design and development of a simulation game for teacher education. Technology, Instruction, Cognition and Learning, 6(3), 193–212.Google Scholar
  30. Ifenthaler, D. (2010a). Learning and instruction in the digital age. In J. M. Spector, D. Ifenthaler, P. Isaías, Kinshuk, & D. G. Sampson (Eds.), Learning and instruction in the digital age: Making a difference through cognitive approaches, technology-facilitated collaboration and assessment, and personalized communications (pp. 3–10). New York: Springer.Google Scholar
  31. Ifenthaler, D. (2010b). Relational, structural, and semantic analysis of graphical representations and concept maps. Educational Technology Research and Development, 58(1), 81–97.CrossRefGoogle Scholar
  32. Ifenthaler, D. (2010c). Scope of graphical indices in educational diagnostics. In D. Ifenthaler, P. Pirnay-Dummer, & N. M. Seel (Eds.), Computer-based diagnostics and systematic analysis of knowledge (pp. 213–234). New York: Springer.CrossRefGoogle Scholar
  33. Ifenthaler, D., Masduki, I., & Seel, N. M. (2009). The mystery of cognitive structure and how we can detect it. Tracking the development of cognitive structures over time. Instructional Science. Retrieved October 18, 2010, from http://www.springerlink.com/content/y62024472v394468/
  34. Ifenthaler, D., & Pirnay-Dummer, P. (2009). Assessment of knowledge: Do graphical notes and texts represent different things? In M. R. Simonson (Ed.), Annual proceedings of selected research and development papers presented at the national convention of the Association for Educational Communications and Technology (32nd, Louisville, KY, 2009), (Vol. 2, pp. 86–93). Bloomington, IN: AECT.Google Scholar
  35. Ifenthaler, D., Pirnay-Dummer, P., & Seel, N. M. (Eds.). (2010). Computer-based diagnostics and systematic analysis of knowledge. New York: Springer.Google Scholar
  36. Ifenthaler, D., & Seel, N. M. (2005). The measurement of change: Learning-dependent progression of mental models. Technology, Instruction, Cognition and Learning, 2(4), 317–336.Google Scholar
  37. Johnson-Laird, P. N. (1989). Mental models. In M. I. Posner (Ed.), Foundations of cognitive science (pp. 469–499). Cambridge, MA: MIT Press.Google Scholar
  38. Johnson, T. E., Ifenthaler, D., Pirnay-Dummer, P., & Spector, J. M. (2009). Using concept maps to assess individuals and team in collaborative learning environments. In P. L. Torres & R. C. V. Marriott (Eds.), Handbook of research on collaborative learning using concept mapping (pp. 358–381). Hershey, PA: Information Science Publishing.Google Scholar
  39. Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes. Educational Technology Research and Development, 45(1), 65–94.CrossRefGoogle Scholar
  40. Klabbers, J. H. (Ed.). (2008). The magic circle: Principles of gaming & simulation. Rotterdam: Sense.Google Scholar
  41. Koschmann, T. (Ed.). (1996). CSCL, theory and practice of an emerging paradigm. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  42. Macy, M., Squires, J. K., & Barton, E. E. (2009). Providing optimal opportunities: Structuring practicum experiences in early intervention and early childhood special education preservice programs. Topics in Early Childhoos Special Education, 28(4), 209–218.CrossRefGoogle Scholar
  43. Mandl, H., & Fischer, F. (2000). Mapping-Techniken und Begriffsnetze in Lern- und Kooperationsprozessen. In H. Mandl & F. Fischer (Eds.), Wissen sichtbar machen – Wissensmanagement mit Mapping-Techniken (pp. 3–12). Göttingen: Hogrefe.Google Scholar
  44. McFarlane, A. (2003). Editorial. assessment for the digital age, Assessment in Education: Principles, Policy & Practice, 10, 261–266.CrossRefGoogle Scholar
  45. Michael, D., & Chen, S. (2006). Serious games: Games that educate, train, and inform. Boston: Thomson Course Technology PTR.Google Scholar
  46. Miller, R. B., Eseryel, D., & Ge, X. (2009). Surviving in space: The effects of a massively multiplayer online game (MMOG) on students’ motivation. Paper presented at the Annual Meeting of the American Educational Research Association, San Diego, CA, USA.Google Scholar
  47. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
  48. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231–259.CrossRefGoogle Scholar
  49. Papert, S., & Caperton, G. (1999). Vision for education: The Caperton-Papert platform. Paper presented at the 91st Annual National Governor’ Association Meeting. Retrieved April 22, 2010, from http://www.papert.org/articles/Vision_for_education.html
  50. Pellegrino, J. W., Chudowsky, N., & Glaser, R. (Eds.). (2003). Knowing what students know. The secience and design of educational assessment. Washington, DC: National Academy Press.Google Scholar
  51. Pirnay-Dummer, P., & Ifenthaler, D. (2010). Automated knowledge visualization and assessment. In D. Ifenthaler, P. Pirnay-Dummer, & N. M. Seel (Eds.), Computer-based diagnostics and systematic analysis of knowledge (pp. 77–115). New York: Springer.CrossRefGoogle Scholar
  52. Pirnay-Dummer, P., Ifenthaler, D., & Rohde, J. (2009). Text-guided automated self-assessment. In Kinshuk, D. G. Sampson, J. M. Spector, P. Isaias, & D. Ifenthaler (Eds.), Proceedings of the IADIS international conference on cognition and exploratory learning in the digital age (pp. 311–316). Rome: IADIS Press.Google Scholar
  53. Pirnay-Dummer, P., Ifenthaler, D., & Spector, J. M. (2010). Highly integrated model assessment technology and tools. Educational Technology Research and Development, 58(1), 3–18.CrossRefGoogle Scholar
  54. Prensky, M. (2001). Digital game-based learning. New York: McGraw-Hill.Google Scholar
  55. Prensky, M. (2006). Don’t bother me mom–I’m learning. St. Paul, MN: Paragon House.Google Scholar
  56. Ruiz-Primo, M. A., Schultz, S. E., Li, M., & Shavelson, R. J. (2001). Comparison of the reliability and validity of scores from two concept-mapping techniques. Journal of Research in Science Teaching, 38(2), 260–278.CrossRefGoogle Scholar
  57. Scandura, J. M. (1988). Role of relativistic knowledge in intelligent tutoring. Computers in Human Behavior, 4(1), 53–64.CrossRefGoogle Scholar
  58. Scheele, B., & Groeben, N. (1984). Die Heidelberger Struktur-Lege-Technik (SLT). Eine Dialog-Konsens-Methode zur Erhebung subjektiver Theorien mittlerer Reichweite. Weinheim: Beltz.Google Scholar
  59. Schön, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. San Francisco: Jossey-Bass.Google Scholar
  60. Seel, N. M. (1999). Educational diagnosis of mental models: Assessment problems and technology-based solutions. Journal of Structural Learning and Intelligent Systems, 14(2), 153–185.Google Scholar
  61. Seel, N. M., Ifenthaler, D., & Pirnay-Dummer, P. (2009). Mental models and problem solving: Technological solutions for measurement and assessment of the development of expertise. In P. Blumschein, W. Hung, D. H. Jonassen, & J. Strobel (Eds.), Model-based approaches to learning: Using systems models and simulations to improve understanding and problem solving in complex domains (pp. 17–40). Rotterdam: Sense.Google Scholar
  62. Shaffer, D. W. (2006). How computer games help children learn? New York: Palgrave Macmillan.CrossRefGoogle Scholar
  63. Shute, V. J., & Spector, J. M. (2010). Stealth assessment in virtual worlds. Retrieved April 22, 2010, from http://www.adlnet.gov/Technologies/Evaluation/Library/Additional%20Resources/LETSI%20White%20Papers/Shute%20-%20Stealth%20Assessment%20in%20Virtual%20Worlds.pdf
  64. Spector, J. M., Dennen, V. P., & Koszalka, T. A. (2005). Individual and collaborative construction of causal concept maps: an online technique for learning and assessment. In G. Chiazzese, M. Allegra, A. Chifari, & S. Ottaviano (Eds.), Methods and technologies for learning (pp. 223–227). Southampton: WIT Press.Google Scholar
  65. Spector, J. M., & Koszalka, T. A. (2004). The DEEP methodology for assessing learning in complex domains (Final report to the National Science Foundation Evaluative Research and Evaluation Capacity Building). Syracuse, NY: Syracuse University.Google Scholar
  66. Squire, K. (2004). Replaying history. Unpublished doctoral dissertation, Indiana University, Bloomington, IN.Google Scholar
  67. Sterman, J. D. (1994). Learning in and about complex systems. System Dynamics Review, 10(2–3), 291–330.CrossRefGoogle Scholar
  68. Swearingen, D., & Eseryel, D. (2010). The effects of massively multiplayer online games on high school students’ mathematics achievement. Paper presented at the Annual Meeting of the American Educational Research Association, Denver, CO, USA.Google Scholar
  69. Tarciani, E. M., & Clariana, R. B. (2006). A technique for automatically scoring open-ended concept maps. Educational Technology, Research, & Development, 54(1), 65–82.CrossRefGoogle Scholar
  70. Thomas, D., & Brown, J. S. (2007). The play of imagination: Extending the literary mind. Games & Culture, 2(2), 149–172.CrossRefGoogle Scholar
  71. Tuzun, H. (2004). Motivating learners in educational computer games. Unpublished doctoral dissertation. Indiana University, Bloomington, IN.Google Scholar
  72. Wagner, W., & Wagner, S. U. (1985). Presenting questions, processing responses, and providing feedback in CAI. Journal of Instructional Development, 8(4), 2–8.CrossRefGoogle Scholar
  73. Warren, S. J., & Dondlinger, M. J. (2009). Designing games for learning. In R. E. Ferdig (Ed.), Handbook of research on effective electronic gaming in education (pp. 1183–1203). Hershey, PA: IGI Global.Google Scholar
  74. Willett, J. B. (1988). Questions and answers in the measurement of change. Review of Research in Education, 15, 345–422.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.College of Education, University of OklahomaNormanUSA
  2. 2.Albert-Ludwigs-University FreiburgFreiburgGermany
  3. 3.University of OklahomaNormanUSA

Personalised recommendations