Deep Reinforcement Learning in Serious Games: Analysis and Design of Deep Neural Network Architectures

  • Aline DobrovskyEmail author
  • Cezary W. Wilczak
  • Paul Hahn
  • Marko Hofmann
  • Uwe M. Borghoff
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10672)


Serious games present a noteworthy research area for artificial intelligence, where automated adaptation and reasonable NPC behaviour present essential challenges. Deep reinforcement learning has already been successfully applied to game-playing. We aim to expand and improve the application of deep learning methods in SGs through investigating their architectural properties and respective application scenarios. In this paper, we examine promising architectures and conduct first experiments concerning CNN design and analysis for game-playing. Although precise statements about the applicability of different architectures are not yet possible, our findings allow for concluding some general recommendations for the choice of DL architectures in different scenarios. Furthermore, we point out promising prospects for further research.


Deep learning Serious games Convolutional neural networks Neural network visualization 


  1. 1.
    Brisson, A., Pereira, G., Prada, R., Paiva, A., Louchart, S., Suttie, N., Lim, T., Lopes, R., Bidarra, R., Bellotti, F., et al.: Artificial intelligence and personalization opportunities for serious games. In: Eighth AIIDE Conference, vol. 2012, pp. 51–57 (2012)Google Scholar
  2. 2.
    Dobrovsky, A., Borghoff, U.M., Hofmann, M.: Applying and augmenting deep reinforcement learning in serious games through interaction. Periodica Polytech. Electr. Eng. Comput. Sci. 61(2), 198–208 (2017). ISSN 2064-5279CrossRefGoogle Scholar
  3. 3.
    Doujak, G.: Serious Games und Digital Game Based Learning. Spielebasierte E-Learning Trends der Zukunft. GRIN Verlag, Munich (2015)Google Scholar
  4. 4.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. Adaptive Computation and Machine Learning Series. MIT Press, Cambrige (2017)zbMATHGoogle Scholar
  5. 5.
    Goodfellow, I.J.: NIPS 2016 tutorial: generative adversarial networks. CoRR abs/1701.00160 (2017).
  6. 6.
    Hahn, P.: Visualisierung von Convolutional Neural Networks. Bachelorarbeit, Universität der Bundeswehr München (2017)Google Scholar
  7. 7.
    Harley, A.W.: An interactive node-link visualization of convolutional neural networks. In: Bebis, G., et al. (eds.) ISVC 2015. LNCS, vol. 9474, pp. 867–877. Springer, Cham (2015). CrossRefGoogle Scholar
  8. 8.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc. (2012)Google Scholar
  9. 9.
    Lample, G., Chaplot, D.S.: Playing FPS games with deep reinforcement learning. arXiv preprint arXiv:1609.05521 (2016)
  10. 10.
    Lara-Cabrera, R., Nogueira-Collazo, M., Cotta, C., Fernández-Leiva, A.J.: Game artificial intelligence: challenges for the scientific community. In: Proceedings 2st Congreso de la Sociedad Española para las Ciencias del Videojuego Barcelona, Spain (2015)Google Scholar
  11. 11.
    Martinez, H.P., Bengio, Y., Yannakakis, G.N.: Learning deep physiological models of affect. IEEE Comput. Intell. Mag. 8(2), 20–33 (2013)CrossRefGoogle Scholar
  12. 12.
    Min, W., Ha, E., Rowe, J.P., Mott, B.W., Lester, J.C.: Deep learning-based goal recognition in open-ended digital games. In: AIIDE. Citeseer (2014)Google Scholar
  13. 13.
    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)CrossRefGoogle Scholar
  14. 14.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)Google Scholar
  15. 15.
    Vinyals, O.: DeepMind and Blizzard to release Starcraft II as an AI research environment (2016). Accessed 23 May 2017
  16. 16.
    Wang, X., Gupta, A.: Unsupervised learning of visual representations using videos. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2794–2802 (2015)Google Scholar
  17. 17.
    Wilczak, C.: Convolutional Neural Networks: Betrachtung und Vergleich verschiedener Architekturen und ihrer Merkmale in Computerspielen. Bachelorarbeit, Universität der Bundeswehr München (2017)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Aline Dobrovsky
    • 1
    Email author
  • Cezary W. Wilczak
    • 1
  • Paul Hahn
    • 1
  • Marko Hofmann
    • 1
  • Uwe M. Borghoff
    • 1
  1. 1.Fakultät für InformatikUniversität der Bundeswehr MünchenNeubibergGermany

Personalised recommendations