Abstract
Modeling learning agents in the context of Multi-agent Systems requires an adequate understanding of their dynamic behaviour. Usually, these agents are modeled similar to the different players in a standard game theoretical model. Unfortunately traditional Game Theory is static and limited in its usefelness.
Evolutionary Game Theory improves on this by providing a dynamics which describes how strategies evolve over time. In this paper, we discuss three learning models whose dynamics are related to the Replicator Dynamics(RD). We show how a classical Reinforcement Learning(RL) technique, i.e. Q-learning relates to the RD. This allows to better understand the learning process and it allows to determine how complex a RL model should be. More precisely, Occam’s Razor applies in the framework of games, i.e. the simplest model (Cross) suffices for learning equilibria. An experimental verification in all three models is presented.
Author funded by a doctoral grant of the institute for advancement of scientific technological research in Flanders (IWT).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Börgers, T., Sarin, R., Learning Through Reinforcement and Replicator Dynamics. Journal of Economic Theory, Volume 77, Number 1, November 1997.
Hofbauer, J., Sigmund, K., Evolutionary Games and Population Dynamics, Cambridge University Press, 1998.
Narendra, K., Thathachar, M., Learning Automata: An Introduction. Prentice-Hall (1989).
Redondo, F.V., Game Theory and Economics, Cambridge University Press, (2001).
Schneider, T.D., Evolution of biological information. journal of NAR, volume 28, pages 2794–2799, 2000.
Stauffer, D., Life, Love and Death: Models of Biological Reproduction and Aging. Institute for Theoretical physics, Köln, Euroland, 1999.
Sutton, R. S., Barto, A.G.: Reinforcement Learning: An introduction. Cambridge, MA: MIT Press (1998).
Tuyls, K., Lenaerts, T., Verbeeck, K., Maes, S. and Manderick, B, Towards a Relation Between Learning Agents and Evolutionary Dynamics. Proceedings of BNAIC 2002. KU Leuven, Belgium.
Tuyls, K., Verbeeck, K. and Lenaerts, T., A Selection-Mutation model for Qlearning in MAS. Accepted at AAMAS 2003. Melbourne, Australia.
Weibull, J.W., Evolutionary Game Theory, MIT Press, (1996).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tuyls, K., Verbeeck, K., Maes, S. (2003). On a Dynamical Analysis of Reinforcement Learning in Games: Emergence of Occam’s Razor. In: Mařík, V., Pěchouček, M., Müller, J. (eds) Multi-Agent Systems and Applications III. CEEMAS 2003. Lecture Notes in Computer Science(), vol 2691. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45023-8_32
Download citation
DOI: https://doi.org/10.1007/3-540-45023-8_32
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40450-7
Online ISBN: 978-3-540-45023-8
eBook Packages: Springer Book Archive