Abstract
The number of educational or serious games (SGs) available to educators has increased in recent years as the cost of game development has been reduced. A benefit of SGs is that they employ not only lesson content but also knowledge contexts where learners can connect information to its context of use with active participation and engagement. This, in turn, improves learners’ ability to recall, integrate, and apply what they learn. Much of the research on game analytics has examined learner in-game trails to build predictive models that identify negative learner actions (e.g., systematic guessing after the fact). However, analytics can also be used in the game design and development phases. Drawing on evidence-centered design (ECD), the chapter outlines ways that analytics can drive the development of scenarios and activities in a game and thus allows SGs to function as contextual apprenticeships, providing robust assessment opportunities. We describe how ECD theory was applied in a project to develop and test a SG that trains people to reduce their reliance on cognitive biases. We describe instances during the design process where our team encountered obstacles due to differing psychological and learning/teaching orientations, a topic rarely explored in the SG or ECD literature. Furthermore, we describe the final analytics-based game design features. We propose an additional element (persona) and how we anticipate incorporating that ECD extension into future projects.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Al-Smadi, M., Wesiak, G., Guetl, C., & Holzinger, A. (2012, July). Assessment for/as learning: Integrated automatic assessment in complex learning resources for self-directed learning. In 2012 Sixth International Conference on Complex, Intelligent and Software Intensive Systems (CISIS) (pp. 929–934). Palermo, Italy: IEEE.
Barab, S. A., Gresalfi, M., & Ingram-Goble, A. (2010). Transformational play using games to position person, content, and context. Educational Researcher, 39(7), 525–536.
Behrens, J. T., Mislevy, R. J., Bauer, M., Williamson, D. M., & Levy, R. (2004). Introduction to evidence centered design and lessons learned from its application in a global e-learning program. International Journal of Testing, 4(4), 295–301.
Behrens, J. T., Frezzo, D., Mislevy, R., Kroopnick, M., & Wise, D. (2007). Structural, functional and semiotic symmetries in simulation-based games and assessments. Assessment of problem solving using simulations. Simulation-Based Games and Assessments, 4, 59–80.
Boyd, J. R. (1995). The essence of winning and losing. A five slide set.
Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132, 354–338.
DiCerbo, K. E. (2014). Game-based assessment of persistence. Educational Technology & Society, 17(1), 17–28.
Fischhoff, B. (1982). Debiasing. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 422–444). Cambridge, MA: Cambridge University Press.
Gee, J. P. (2007). What video games have to teach us about learning and literacy. Revised and updated edition. New York: Macmillan.
Harvey, J. H., Town, J. P., & Yarkin, K. L. (1981). How fundamental is “the fundamental attribution error”? Journal of Personality and Social Psychology, 40, 346–357.
Jones, E. E., & Harris, V. A. (1967). The attribution of attitudes. Journal of Experimental Social Psychology, 3(1), 1–24.
Kahneman, D. (2003). A perspective on judgment and choice: Mapping bounded rationality. American Psychologist, 58, 697–720.
Kahneman, D. (2011). Thinking, fast and slow. New York: Macmillan.
Loh, C. S., Anantachai, A., Byun, J., & Lenox, J. (2007, July). Assessing what players learned in serious games: in situ data collection, information trails, and quantitative analysis. In Proceedings of the computer games: AI, animation, mobile, educational & serious games conference. Wolverhampton: University of Wolverhampton.
Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13–23.
Mezirow, J. (1997). Transformative learning: Theory to practice. New directions for adult and continuing education, 74, 5–12.
Michael, D. R., & Chen, S. L. (2005). Serious games: Games that educate, train, and inform. Boston: Thomson Course Technology.
Mislevy, R. J., Behrens, J. T., Dicerbo, K. E., Frezzo, D. C., & West, P. (2012). Three things game designers need to know about assessment. In Assessment in game-based learning (pp. 59–81). New York: Springer.
Mislevy, R. J., & Riconscente, M. M. (2006). Evidence-centered assessment design: Layers, concepts, and terminology. In S. Downing & T. Haladyna (Eds.), Handbook of test development (pp. 61–90). Mahwah, NJ: Erlbaum.
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). Focus article: On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1(1), 3–62.
Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220.
Presser, A. L., Vahey, P., & Zanchi, C. (2013, June). Designing early childhood math games: A research-driven approach. In Proceedings of the 12th International Conference on Interaction Design and Children (pp. 376–379). New York: ACM.
Pronin, E., Lin, D. Y., & Ross, L. (2002). The bias blind spot: Perceptions of bias in self versus others. Personality and Social Psychology Bulletin, 28, 369–381.
Pruitt, J., & Adlin, T. (2010). The persona lifecycle: Keeping people in mind throughout product design. Amsterdam: Elsevier.
Reese, D. D., Tabachnick, B. G., & Kosko, R. E. (2014). Video game learning dynamics: Actionable measures of multidimensional learning trajectories. British Journal of Educational Technology, 46(1), 98–122
.
Rupp, A. A., Gushta, M., Mislevy, R. J., & Shaffer, D. W. (2010). Evidence-centered design of epistemic games: Measurement principles for complex learning environments. The Journal of Technology, Learning and Assessment, 8(4), 1–48.
Sanna, L. J., Schwarz, N., & Stocker, S. L. (2002). When debiasing backfires: Accessible content and accessibility experiences in debiasing hindsight. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28(3), 497–501.
Shaffer, D. W., Hatfield, D., Svarovsky, G. N., Nash, P., Nulty, A., & Bagley, E. (2009). Epistemic network analysis: A prototype for 21st-century assessment of learning. International Journal of Learning and Media, 1(2), 33–53.
Shaw, A., Kenski, K., Stromer-Galley, J., Martey, R., Clegg, B., Lewis, J., Folkestad, J. E., Strzalkowski, T. (2013). Serious Efforts at Bias Reduction: The Effects of Digital Games and Avatar Customization on Three Cognitive Biases. Washington, D.C.: National Communication Association.
Shute, V. J. (2011a). Stealth assessment in computer-based games to support learning. Computer Games and Instruction, 55(2), 503–524.
Shute, V. J., & Ke, F. (2012). Games, learning, and assessment. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning (pp. 43–58). New York: Springer.
Shute, V. J., Ventura, M., Bauer, M., & Zapata-Rivera, D. (2009). Melding the power of serious games and embedded assessment to monitor and foster learning. In Serious games: Mechanisms and effects (Vol. 2, pp. 295–321). Philadelphia, PA: Routledge/LEA.
Shute, V. J., Masduki, I., & Donmez, O. (2010). Conceptual framework for modeling, assessing, and supporting competencies within game environments. Technology, Instruction, Cognition, and Learning, 8(2), 137–161.
Susi, T., Johannesson, M., & Backlund, P. (2007). Serious games: An overview. Technical report HS-IKI-TR-07-001. Sweden: University of Skövde.
Sweet, S. J., & Rupp, A. A. (2012). Using the ECD framework to support evidentiary reasoning in the context of a simulation study for detecting learner differences in epistemic games. Journal of Educational Data Mining, 4(1), 183–223.
Tetlock, P. E. (1985). Accountability: A social check on the fundamental attribution error. Social Psychology Quarterly, 48(3), 227–236.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131.
Winn, W. (2002). Current trends in educational technology research: The study of learning environments. Educational Psychology Review, 14, 331–351.
Acknowledgements
This work was supported by the Intelligence Advanced Research Projects Activity (IARPA) via the Air Force Research Laboratory contract number FA8650-11-C-7176. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, AFRL, or the U.S. Government.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Folkestad, J.E. et al. (2015). Analytics-Driven Design: Impact and Implications of Team Member Psychological Perspectives on a Serious Games (SGs) Design Framework. In: Loh, C., Sheng, Y., Ifenthaler, D. (eds) Serious Games Analytics. Advances in Game-Based Learning. Springer, Cham. https://doi.org/10.1007/978-3-319-05834-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-05834-4_12
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-05833-7
Online ISBN: 978-3-319-05834-4
eBook Packages: Humanities, Social Sciences and LawEducation (R0)