Abstract
Reinforcement Learning (RL) algorithms have been promising methods for designing intelligent agents in games. Although their capability of learning in real time has been already proved, the high dimensionality of state spaces in most game domains can be seen as a significant barrier. This paper studies the popular arcade video game Ms. Pac-Man and outlines an approach to deal with its large dynamical environment. Our motivation is to demonstrate that an abstract but informative state space description plays a key role in the design of efficient RL agents. Thus, we can speed up the learning process without the necessity of Q-function approximation. Several experiments were made using the multiagent MASON platform where we measured the ability of the approach to reach optimum generic policies which enhances its generalization abilities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Galway, L., Charles, D., Black, M.: Machine learning in digital games: A survey. Artificial Intelligence Review 29, 123–161 (2008)
Sutton, R.: Learning to predict by the method of temporal differences. Machine Learning 3(1), 9–44 (1988)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Inteligence Research 4, 237–285 (1996)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Szita, I.: Reinforcement learning in games. In: Reinforcement Learning, pp. 539–577 (2012)
Szita, I., Lorincz, A.: Learning to play using low-complexity rule-based policies: Illustrations through ms. pac-man. Journal of Artificial Intelligence Research 30, 659–684 (2007)
Lucas, S.M.: Evolving a neural network location evaluator to play ms. pac-man. In: Proc. of IEEE Symposium on Computational Intelligence and Games (CIG 2005), pp. 203–210 (2005)
Bom, L., Henken, R., Wiering, M.A.: Reinforcement learning to train ms. pac-man using higher-order action-relative inputs. In: Proc. of IEEE Intern. Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp. 156–163 (2013)
Alhejali, A.M., Lucas, S.M.: Evolving diverse ms. pac-man playing agents using genetic programming. In: Proc. of IEEE Symposium on Computational Intelligence and Games (CIG 2010), pp. 53–60 (2010)
Samothrakis, S., Robles, D., Lucas, S.: Fast approximate max-n monte-carlo tree search for ms. pac-man. IEEE Trans. on Computational Intelligence and AI in Games 3(2), 142–154 (2011)
Nguyen, K.Q., Thawonmas, R.: Monte carlo tree search for collaboration control of ghosts in ms. pac-man. IEEE Trans. on Computational Intelligence and AI in Games 5(1), 57–68 (2013)
Torrey, L., Taylor, M.: Teaching on a budget: Agents advising agents in reinforcement learning. In: Intern. Conferecene on Autonomous Agents and Multi-agent Systems (AAMAS), pp. 1053–1060 (2013)
Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley (2005)
Singh, S., Sutton, R.S., Kaelbling, P.: Reinforcement learning with replacing eligibility traces, pp. 123–158 (1996)
Hart, P.E., Nilsson, N.J., Raphael, B.: A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems, Science, and Cybernetics SSC-4(2), 100–107 (1968)
Luke, S., Cioffi-Revilla, C., Panait, L., Sullivan, K., Balan, G.: Mason: A multiagent simulation environment. Simulation 81(7), 517–527 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Tziortziotis, N., Tziortziotis, K., Blekas, K. (2014). Play Ms. Pac-Man Using an Advanced Reinforcement Learning Agent. In: Likas, A., Blekas, K., Kalles, D. (eds) Artificial Intelligence: Methods and Applications. SETN 2014. Lecture Notes in Computer Science(), vol 8445. Springer, Cham. https://doi.org/10.1007/978-3-319-07064-3_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-07064-3_6
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07063-6
Online ISBN: 978-3-319-07064-3
eBook Packages: Computer ScienceComputer Science (R0)