Abstract
Evaluating your content generator is a very important task, but difficult to do well. Creating a game content generator in general is much easier than creating a good game content generator—but what is a “good” content generator? That depends very much on what you are trying to create and why. This chapter discusses the importance and the challenges of evaluating content generators, and more generally understanding a generator’s strengths and weaknesses and suitability for your goals. In particular, we discuss two different approaches to evaluating content generators: visualizing the expressive range of generators, and using questionnaires to understand the impact of your generator on the player. These methods could broadly be called top-down and bottom-up methods for evaluating generators.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Boden, M.A.: What is creativity? In: M.A. Boden (ed.) Dimensions of Creativity, pp. 75–117. MIT Press (1994)
Camilleri, E., Yannakakis, G.N., Dingli, A.: Platformer level design for player believability. In: Proceedings of the IEEE Conference on Computational Intelligence and Games (2016)
Chernova, S., Orkin, J., Breazeal, C.: Crowdsourcing HRI through online multiplayer games. In: AAAI Fall Symposium: Dialog with Robots, pp. 14–19 (2010)
Colton, S.: Creativity versus the perception of creativity in computational systems. In: Proceedings of the AAAI Spring Symposium on Creative Intelligent Systems, pp. 14–20 (2008)
Drachen, A., Thurau, C., Togelius, J., Yannakakis, G.N., Bauckhage, C.: Game data mining. In: M. Seif El-Nasr, A. Drachen, A. Canossa (eds.) Game Analytics, pp. 205–253. Springer (2013)
Holmg°ard, C., Yannakakis, G.N., Mart´ınez, H.P., Karstoft, K.I.: To rank or to classify? Annotating stress for reliable PTSD profiling. In: Proceedings of the International Conference on Affective Computing and Intelligent Interaction, pp. 719–725 (2015)
IJsselsteijn,W., De Kort, Y., Poels, K., Jurgelionis, A., Bellotti, F.: Characterising and measuring user experiences in digital games. In: Proceedings of the ACE 2007Workshop on Methods for Evaluating Games (2007)
8. Li, B., Lee-Urban, S., Appling, D.S., Riedl, M.O.: Crowdsourcing narrative intelligence. Advances in Cognitive Systems 2, 25–42 (2012)
Liapis, A., Yannakakis, G.N., Togelius, J.: Towards a generic method of evaluating game levels. In: Proceedings of the Artificial Intelligence for Interactive Digital Entertainment Conference, pp. 30–36 (2013)
Mart´ınez, H.P., Yannakakis, G.N., Hallam, J.: Don’t classify ratings of affect; rank them! IEEE Transactions on Affective Computing 5(3), 314–326 (2014)
Metallinou, A., Narayanan, S.: Annotation and processing of continuous emotional attributes: Challenges and opportunities. In: Proceedings of the IEEE Conference on Automatic Face and Gesture Recognition (2013)
Nelson, M.J.: Game metrics without players: Strategies for understanding game artifacts. In: Proceedings of the First AIIDE Workshop on AI in the Game-Design Process, pp. 14–18 (2011)
Shaker, N., Nicolau, M., Yannakakis, G.N., Togelius, J., ONeill, M.: Evolving levels for Super Mario Bros. using grammatical evolution. In: Proceedings of the IEEE Conference on Computational Intelligence and Games, pp. 304–311 (2012)
14. Shaker, N., Yannakakis, G., Togelius, J.: Crowd-sourcing the aesthetics of platform games. IEEE Transactions on Computational Intelligence and AI in Games 5(3), 276–290 (2013)
Smith, G., Whitehead, J.: Analyzing the expressive range of a level generator. In: Proceedings of the First Workshop on Procedural Content Generation in Games (2010)
16. Smith, G., Whitehead, J., Mateas, M., Treanor, M., March, J., Cha, M.: Launchpad: A rhythmbased level generator for 2-d platformers. IEEE Transactions on Computational Intelligence and AI in Games 3(1), 1–16 (2011)
Yannakakis, G.N.: Preference learning for affective modeling. In: Proceedings of the International Conference on Affective Computing and Intelligent Interaction (2009)
Yannakakis, G.N., Hallam, J.: Ranking vs. preference: A comparative study of self-reporting. In: Proceedings of the International Conference on Affective Computing and Intelligent Interaction, pp. 437–446 (2011)
Yannakakis, G.N., Liapis, A., Alexopoulos, C.: Mixed-initiative co-creativity. In: Proceedings of the 9th Conference on the Foundations of Digital Games (2014)
Yannakakis, G.N., Mart´ınez, H.P.: Grounding truth via ordinal annotation. In: Proceedings of the International Conference on Affective Computing and Intelligent Interaction, pp. 574–580 (2015)
Yannakakis, G.N., Mart´ınez, H.P.: Ratings are overrated! Frontiers in ICT 2, 13 (2015)
Yannakakis, G.N., Mart´ınez, H.P., Garbarino, M.: Psychophysiology in games. In: K. Karpouzis, G.N. Yannakakis (eds.) Emotion in Games: Theory and Praxis. Springer (2016)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Shaker, N., Smith, G., Yannakakis, G.N. (2016). Evaluating content generators. In: Procedural Content Generation in Games. Computational Synthesis and Creative Systems. Springer, Cham. https://doi.org/10.1007/978-3-319-42716-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-42716-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-42714-0
Online ISBN: 978-3-319-42716-4
eBook Packages: Computer ScienceComputer Science (R0)