Abstract
Telemetry opens new possibilities for the assessment of serious games through the continuous, unobtrusive, monitoring of in-game behavior. Data obtained through telemetry thus not only contains information about the outcomes but also about the intermediate processes. In this sense, telemetry data can be of value for various stakeholders of serious games, including developers, educators, and learners themselves to increase the effectiveness of the intervention. In doing so, particular significance should be attached to differences among individuals and demographic groups in order to understand and better accommodate for these variations. However, the large amounts of data gathered via telemetry can make it challenging to derive meaningful information from it. Visualizations can support this process by providing a means to explore, to compare, and to draw insights from the data sets. In this chapter, we discuss three common visual design strategies that facilitate comparative data analysis. Several examples, drawn from the game-based learning literature and related areas as well as two detailed case studies are used to illustrate how these strategies can be leveraged in the context of serious game analytics.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Andersen, E., Liu, Y.-E., Apter, E., Boucher-Genesse, F., & Popović, Z. (2010). Gameplay analysis through state projection. In Proceedings of the 5th International Conference on the Foundations of Digital Games (pp. 1–8). doi:10.1145/1822348.1822349.
Andrews, K., Wohlfahrt, M., & Wurzinger, G. (2009). Visual graph comparison. In Proceedings of the 13th International Conference Information Visualisation (pp. 62–67). doi:10.1109/IV.2009.108.
Anolli, L., & Confalonieri, L. (2011). Learning, dynamic assessment and serious games. In A. Méndez-Vilas (Ed.), Education in a technological world: Communicating current and emerging research and technological efforts (pp. 279–287). Badajoz, Spain: Formatex.
Archambault, D., Purchase, H. C., & Pinaud, B. (2011). Animation, small multiples, and the effect of mental map preservation in dynamic graphs. IEEE Transactions on Visualization and Computer Graphics, 17(4), 539–552. doi:10.1109/TVCG.2010.78.
Arthur, W., Jr., Strong, M. H., Jordan, J. A., Williamson, J. E., Shebilske, W. L., & Regian, J. W. (1995). Visual attention: Individual differences in training and predicting complex task performance. Acta Psychologica, 88(1), 3–23. doi:10.1016/0001-6918(94)E0055-K.
Beck, F., Burch, M., Diehl, S., & Weiskopf, D. (2014). The state of the art in visualizing dynamic graphs. In EuroVis—STARs (pp. 83–103). doi:10.2312/eurovisstar.20141174.
Beck, F., Burch, M., & Weiskopf, D. (2013). Visual comparison of time-varying athletes’ performance. In Proceedings of the 1st Workshop on Sports Data Visualization.
Becker, K., & Parker, J. R. (2011). The guide to computer simulations and games. Indianapolis, IN: Wiley.
Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., & Berta, R. (2013). Assessment in and of serious games: An overview. Advances in Human-Computer Interaction, 2013. doi:10.1155/2013/136864.
Boyandin, I., Bertini, E., & Lalanne, D. (2012). A qualitative study on the exploration of temporal changes in flow maps with animation and small-multiples. Computer Graphics Forum, 31(3pt2), 1005–1014. doi:10.1111/j.1467-8659.2012.03093.x.
Brandes, U., & Corman, S. R. (2003). Visual unrolling of network evolution and the analysis of dynamic discourse. Information Visualization, 2(1), 40–50. doi:10.1057/palgrave.ivs.9500037.
Brandes, U., Dwyer, T., & Schreiber, F. (2004). Visualizing related metabolic pathways in two and a half dimensions. In G. Liotta (Ed.), Graph drawing (pp. 111–122). Berlin: Springer.
Buendía-García, F., García-Martínez, S., Navarrete-Ibañez, E. M., & Jesús, M. (2013). Designing serious games for getting transferable skills in training settings. Interaction Design and Architecture(s), 19, 47–62.
Butler, E., & Banerjee, R. (2014). Visualizing progressions for education and game design. Retrieved from http://cse512-14w.github.io/fp-edbutler-piscean/final/paper-edbutler-piscean.pdf
Dawson, S. (2010). “Seeing” the learning community: An exploration of the development of a resource for monitoring online student networking. British Journal of Educational Technology, 41(5), 736–752. doi:10.1111/j.1467-8535.2009.00970.x.
de Freitas, S., & Jarvis, S. (2006). A framework for developing serious games to meet learner needs. In Proceedings of the Interservice/Industry Training, Simulation & Education Conference.
de Freitas, S., & Oliver, M. (2006). How can exploratory learning with games and simulations within the curriculum be most effectively evaluated? Computers & Education, 46(3), 249–264. doi:10.1016/j.compedu.2005.11.007.
Desmarais, M. C., & Lemieux, F. (2013). Clustering and visualizing study state sequences. In Proceedings of 6th International Conference on Educational Data Mining (pp. 224–227).
Drachen, A., & Canossa, A. (2011). Evaluating motion: Spatial user behaviour in virtual environments. International Journal of Arts and Technology, 4(3), 294–314. doi:10.1504/IJART.2011.041483.
Duval, E. (2011). Attention please!: Learning analytics for visualization and recommendation. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge (pp. 9–17). doi:10.1145/2090116.2090118.
Dwyer, T., Hong, S.-H., Koschützki, D., Schreiber, F., & Xu, K. (2006). Visual analysis of network centralities. In Proceedings of the Asia-Pacific Symposium on Information Visualisation (pp. 189–197).
Erten, C., Kobourov, S. G., Le, V., & Navabi, A. (2003). Simultaneous graph drawing: Layout algorithms and visualization schemes. In Proceedings of the 11th Symposium on Graph Drawing (pp. 437–449).
Fardinpour, A., Reiners, T., & Dreher, H. (2013). Action-based learning assessment method (ALAM) in virtual training environments. In Proceedings of the 30th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (pp. 267–277).
Federico, P., Aigner, W., Miksch, S., Windhager, F., & Zenk, L. (2011). A visual analytics approach to dynamic social networks. In Proceedings of the 11th International Conference on Knowledge Management and Knowledge Technologies (pp. 47:1–47:8). doi:10.1145/2024288.2024344.
Ferster, B., & Shneiderman, B. (2012). Interactive visualization: Insight through inquiry. Cambridge, MA: MIT Press.
Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64–71. doi:10.1007/s11528-014-0822-x.
Gelderblom, H., & Kotzé, P. (2009). Ten design lessons from the literature on child development and children’s use of technology. In Proceedings of the 8th International Conference on Interaction Design and Children (pp. 52–60). doi:10.1145/1551788.1551798.
Gleicher, M., Albers, D., Walker, R., Jusufi, I., Hansen, C. D., & Roberts, J. C. (2011). Visual comparison for information visualization. Information Visualization, 10(4), 289–309. doi:10.1177/1473871611416549.
Govaerts, S., Verbert, K., Duval, E., & Pardo, A. (2012). The student activity meter for awareness and self-reflection. In CHI’12 Extended Abstracts on Human Factors in Computing Systems (pp. 869–884). doi:10.1145/2212776.2212860.
Govaerts, S., Verbert, K., Klerkx, J., & Duval, E. (2010). Visualizing activities for self-reflection and awareness. In X. Luo, M. Spaniol, L. Wang, Q. Li, W. Nejdl, & W. Zhang (Eds.), Advances in web-based learning—ICWL 2010 (pp. 91–100). Berlin: Springer.
Graham, M., & Kennedy, J. (2010). A survey of multiple tree visualisation. Information Visualization, 9(4), 235–252. doi:10.1057/ivs.2009.29.
Griffin, A. L., MacEachren, A. M., Hardisty, F., Steiner, E., & Li, B. (2006). A comparison of animated maps with static small-multiple maps for visually identifying space-time clusters. Annals of the Association of American Geographers, 96(4), 740–753. doi:10.1111/j.1467-8306.2006.00514.x.
Guerra-Gómez, J. A., Buck-Coleman, A., Pack, M. L., Plaisant, C., & Shneiderman, B. (2013). TreeVersity: Interactive visualizations for comparing hierarchical data sets. Transportation Research Record: Journal of the Transportation Research Board, 2392, 48–58. doi:10.3141/2392-06.
Hartmann, T., & Klimmt, C. (2006). Gender and computer games: Exploring females’ dislikes. Journal of Computer-Mediated Communication, 11(4), 910–931. doi:10.1111/j.1083-6101.2006.00301.x.
Heeter, C., Lee, Y.-H., Magerko, B., & Medler, B. (2011). Impacts of forced serious game play on vulnerable subgroups. International Journal of Gaming and Computer-Mediated Simulations, 3(3), 34–53. doi:10.4018/jgcms.2011070103.
Holten, D., & van Wijk, J. J. (2008). Visual comparison of hierarchically organized data. In Proceedings of the 10th Joint Eurographics/IEEE—VGTC Conference on Visualization (pp. 759–766). doi:10.1111/j.1467-8659.2008.01205.x.
Houghton, S. (2011). Balance and flow maps. Retrieved April, 2015, from http://www.gamasutra.com/view/news/125213/Opinion_Balance_and_Flow_Maps.php.
Inselberg, A. (1985). The plane with parallel coordinates. The Visual Computer, 1(2), 69–91. doi:10.1007/BF01898350.
Javed, W., & Elmqvist, N. (2012). Exploring the design space of composite visualization. In Proceedings of the IEEE Pacific Visualization Symposium (pp. 1–8). doi:10.1109/PacificVis.2012.6183556.
Kayali, F., Wallner, G., Kriglstein, S., Bauer, G., Martinek, D., Hlavacs, H., et al. (2014). A case study of a learning game about the Internet. In S. Göbel & J. Wiemeyer (Eds.), Games for training, education, health and sports (pp. 47–58). Cham, Switzerland: Springer.
Ketelhut, D. J. (2007). The impact of student self-efficacy on scientific inquiry skills: An exploratory investigation in River City, a multi-user virtual environment. Journal of Science Education and Technology, 16(1), 99–111. doi:10.1007/s10956-006-9038-y.
Kiili, K., Ketamo, H., & Kickmeier-Rust, M. D. (2014). Evaluating the usefulness of eye tracking in game-based learning. International Journal of Serious Games, 1(2), 51–65.
Kirk, A. (2012). Data visualization: A successful design process. Birmingham, England: Packt Publishing.
Klerkx, J., Verbert, K., & Duval, E. (2014). Enhancing learning with visualization techniques. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 791–807). New York: Springer.
Kriglstein, S., Pohl, M., & Smuc, M. (2014). Pep up your time machine: Recommendations for the design of information visualizations of time-dependent data. In W. Huang (Ed.), Handbook of human centric visualization (pp. 203–225). New York: Springer.
Kriglstein, S., Wallner, G., & Rinderle-Ma, S. (2013). A visualization approach for difference analysis of process models and instance traffic. In Proceedings of the 11th International Conference on Business Process Management (pp. 219–226). doi:10.1007/978-3-642-40176-3_18.
Kriz, W. C., & Hense, J. U. (2006). Theory-oriented evaluation for the design of and research in gaming and simulation. Simulation & Gaming, 37(2), 268–283. doi:10.1177/1046878106287950.
Kruskal, J. B., & Wish, M. (1978). Multidimensional scaling. Beverly Hills, CA: Sage.
Linek, S. B., Öttl, G., & Albert, D. (2010). Non-invasive data tracking in educational games: Combination of logfiles and natural language processing. In Proceeding of the International Technology, Education and Development Conference (pp. 2977–2988).
Liu, Y.-E., Andersen, E., Snider, R., Cooper, S., & Popović, Z. (2011). Feature-based projections for effective playtrace analysis. In Proceedings of the 6th International Conference on Foundations of Digital Games (pp. 69–76). doi:10.1145/2159365.2159375.
Loh, C. S. (2012). Information Trails: In-process assessment of game-based learning. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning (pp. 123–144). New York: Springer.
Loh, C. S., & Sheng, Y. (2013). Measuring the (dis-)similarity between expert and novice behaviors as serious games analytics. Education and Information Technologies, 20, 5–19. doi:10.1007/s10639-013-9263-y.
Magerko, B., Heeter, C., & Medler, B. (2010). Different strokes for different folks: Tapping into the hidden potential of serious games. In R. Van Eck (Ed.), Gaming and cognition: Theories and practice from the learning sciences (pp. 255–280). Hershey, PA: IGI Global.
Mehigan, T. J., Barry, M., Kehoe, A., & Pitt, I. (2011). Using eye tracking technology to identify visual and verbal learners. In Proceedings of the IEEE International Conference on Multimedia and Expo (pp. 1–6). doi:10.1109/ICME.2011.6012036.
Miller, J. R. (2007). Attribute blocks: Visualizing multiple continuously defined attributes. IEEE Computer Graphics and Applications, 27(3), 57–69. doi:10.1109/MCG.2007.54.
Minović, M., & Milovanović, M. (2013). Real-time learning analytics in educational games. In Proceedings of the 1st International Conference on Technological Ecosystem for Enhancing Multiculturality (pp. 245–251). doi:10.1145/2536536.2536574.
Mislevy, R. J., & Riconscente, M. (2005). Evidence-centered assessment design: Layers, structures, and terminology (PADI Technical Report No. 9). Menlo Park, CA: SRI International.
Moreno-Ger, P., Torrente, J., Hsieh, Y. G., & Lester, W. T. (2012). Usability testing for serious games: Making informed design decisions with user data. Advances in Human-Computer Interaction, 2012. doi:10.1155/2012/369637.
Munzner, T., Guimbretière, F., Tasiran, S., Zhang, L., & Zhou, Y. (2003). TreeJuxtaposer: Scalable tree comparison using focus + context with guaranteed visibility. ACM Transactions on Graphics, 22(3), 453–462. doi:10.1145/1201775.882291.
O’Rourke, E., Butler, E., Liu, Y.-E., Ballweber, C., & Popović, Z. (2013). The effects of age on player behavior in educational games. In Proceedings of the 8th International Conference on the Foundations of Digital Games (pp. 158–165).
Olsen, T., Procci, K., & Bowers, C. (2011). Serious games usability testing: How to ensure proper usability, playability, and effectiveness. In A. Marcus (Ed.), Design, user experience, and usability. Theory, methods, tools and practice (pp. 625–634). Berlin: Springer.
Peirce, K., & Edwards, E. D. (1988). Children’s construction of fantasy stories: Gender differences in conflict resolution strategies. Sex Roles, 18(7–8), 393–404. doi:10.1007/BF00288391.
Perin, C., Vuillemot, R., & Fekete, J.-D. (2013). SoccerStories: A kick-off for visual soccer analysis. IEEE Transactions on Visualization and Computer Graphics, 19(12), 2506–2515. doi:10.1109/TVCG.2013.192.
Pileggi, H., Stolper, C. D., Boyle, J. M., & Stasko, J. T. (2012). SnapShot: Visualization to propel ice hockey analytics. IEEE Transactions on Visualization and Computer Graphics, 18(12), 2819–2828. doi:10.1109/TVCG.2012.263.
Piringer, H., Pajer, S., Berger, W., & Teichmann, H. (2012). Comparative visual analysis of 2D function ensembles. Computer Graphics Forum, 31(3), 1195–1204. doi:10.1111/j.1467-8659.2012.03112.x.
Plass, J. L., Homer, B. D., Kinzer, C. K., Chang, Y. K., Frye, J., Kaczetow, W., et al. (2013). Metrics in simulations and games for learning. In M. S. El-Nasr, A. Drachen, & A. Canossa (Eds.), Game analytics (pp. 697–729). London: Springer.
Ritsos, P. D., & Roberts, J. C. (2014). Towards more visual analytics in learning analytics. In Proceedings of the 5th EuroVis Workshop on Visual Analytics (pp. 61–65). doi:10.2312/eurova.20141147.
Roberts, J. C. (2005). Exploratory visualization with multiple linked views. In J. Dykes, A. M. MacEachren, & M.-J. Kraak (Eds.), Exploring geovisualization (pp. 159–180). Oxford, England: Elsevier.
Robertson, G., Fernandez, R., Fisher, D., Lee, B., & Stasko, J. (2008). Effectiveness of animation in trend visualization. IEEE Transactions on Visualization and Computer Graphics, 14(6), 1325–1332. doi:10.1109/TVCG.2008.125.
Rowe, J. P., Shores, L. R., Mott, B. W., & Lester, J. C. (2010). Individual differences in gameplay and learning: A narrative-centered learning perspective. In Proceedings of the 5th International Conference on the Foundations of Digital Games (pp. 171–178). doi:10.1145/1822348.1822371.
Scarlatos, L. L., & Scarlatos, T. (2010). Visualizations for the assessment of learning in computer games. In Proceedings of the 7th International Conference & Expo on Emerging Technologies for a Smarter World.
Serrano-Laguna, Á., Torrente, J., Moreno-Ger, P., & Fernández-Manjón, B. (2012). Tracing a little for big improvements: Application of learning analytics and videogames for student assessment. Procedia Computer Science, 15, 203–209. doi:10.1016/j.procs.2012.10.072.
Shah, P., & Hoeffner, J. (2002). Review of graph comprehension research: Implications for instruction. Educational Psychology Review, 14(1), 47–69. doi:10.1023/A:1013180410169.
Shute, V., Ventura, M., Bauer, M., & Zapata-Rivera, D. (2009). Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In U. Ritterfield, M. J. Cody, & P. Vorderer (Eds.), The social science of serious games: Theories and applications (pp. 295–321). New York: Routledge.
Steiner, C. M., Kickmeier-Rust, M. D., & Albert, D. (2009). Little big difference: Gender aspects and gender-based adaptation in educational games. In M. Chang, R. Kuo, K. Kinshuk, G.-D. Chen, & M. Hirose (Eds.), Learning by playing. Game-based education system design and development (pp. 150–161). Berlin: Springer.
Stewart, C. A., Hart, D., Berry, D. K., Olsen, G. J., Wernert, E. A., & Fischer, W. (2001). Parallel implementation and performance of fastDNAml: A program for maximum likelihood phylogenetic inference. In Proceedings of the ACM/IEEE Conference on Supercomputing. doi:10.1145/582034.582054.
Tominski, C., Schulze-Wollgast, P., & Schumann, H. (2005). 3D information visualization for time dependent data on maps. In Proceedings of the nineth International Conference on Information Visualisation (pp. 175–181). doi:10.1109/IV.2005.3.
Tufte, E. R. (1990). Envisioning information. Cheshire, UK: Graphics Press.
Urness, T., Interrante, V., Marusic, I., Longmire, E., & Ganapathisubramani, B. (2003). Effectively visualizing multi-valued flow data using color and texture. In Proceedings of the IEEE Visualization (pp. 115–121). doi:10.1109/VISUAL.2003.1250362.
van Wijk, J. J. (2005). The value of visualization. In Proceedings of the IEEE Visualization (pp. 79–86). doi:10.1109/VISUAL.2005.1532781.
Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Assche, F. V., Parra, G., et al. (2014). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499–1514. doi:10.1007/s00779-013-0751-2.
Wallner, G. (2013). Play-Graph: A methodology and visualization approach for the analysis of gameplay data. In Proceedings of the 8th International Conference on the Foundations of Digital Games (pp. 253–260).
Wallner, G., & Kriglstein, S. (2012a). DOGeometry: Teaching geometry through play. In Proceedings of the 4th International Conference on Fun and Games (pp. 11–18). doi:10.1145/2367616.2367618.
Wallner, G., & Kriglstein, S. (2012b). A spatiotemporal visualization approach for the analysis of gameplay data. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1115–1124). doi:10.1145/2207676.2208558.
Wallner, G., & Kriglstein, S. (2013). Visualization-based analysis of gameplay data—A review of literature. Entertainment Computing, 4(3), 143–155. doi:10.1016/j.entcom.2013.02.002.
Ward, M. O. (2002). A taxonomy of glyph placement strategies for multidimensional data visualization. Information Visualization, 1(3/4), 194–210. doi:10.1057/palgrave.ivs.9500025.
Warren, S., Jones, G., & Lin, L. (2011). Usability and play testing. In L. Annetta & S. C. Bronack (Eds.), Serious educational game assessment (pp. 131–146). Rotterdam, The Netherlands: Sense.
Westera, W., Nadolski, R., Hummel, H. (2014). Serious gaming analytics: What students’ log files tell us about gaming and learning. International Journal of Serious Games, 1(2). doi:10.17083/ijsg.v1i2.9.
Wouters, P., van der Spek, E. D., & Van Oostendorp, H. (2009). Current practices in serious game research: A review from a learning outcomes perspective. In T. M. Connolly, M. Stansfield, & L. Boyle (Eds.), Games-based learning advancements for multisensory human computer interfaces: Techniques and effective practices. Hershey, PA: IGI Global.
Yau, N. (2011). Visualize this: The FlowingData guide to design, visualization, and statistics. Indianapolis, IN: Wiley.
Acknowledgments
We would like to thank Helmut Hlavacs from the Entertainment Computing group of the University of Vienna for his permission to use the telemetry data from Internet Hero for examples in this chapter. Internet Hero was funded by netidee (www.netidee.at, project number 326).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Wallner, G., Kriglstein, S. (2015). Comparative Visualization of Player Behavior for Serious Game Analytics. In: Loh, C., Sheng, Y., Ifenthaler, D. (eds) Serious Games Analytics. Advances in Game-Based Learning. Springer, Cham. https://doi.org/10.1007/978-3-319-05834-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-05834-4_7
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-05833-7
Online ISBN: 978-3-319-05834-4
eBook Packages: Humanities, Social Sciences and LawEducation (R0)