Abstract
In reinforcement learning, complicated applications require involving multiple agents to handle different kinds of tasks simultaneously. However, increasing the number of agents brings in the challenges on managing the interactions among them. In this chapter, according to the optimization problem for each agent, equilibrium concepts are put forward to regulate the distributive behaviors of multiple agents. We further analyze the cooperative and competitive relations among the agents in various scenarios, combining with typical multi-agent reinforcement learning algorithms. Based on all kinds of interactions, a game theoretical framework is finalized for general modeling in multi-agent scenarios. Analyzing the optimization and equilibrium situation for each component of the framework, the optimal multi-agent reinforcement learning policy for each agent can be guided and explored.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Aumann RJ (1987) Correlated equilibrium as an expression of Bayesian rationality. Econometrica J Econ Soc 55:1–18
Bjorn PA, Vuong QH (1985) Econometric modeling of a stackelberg game with an application to labor force participation. Working Papers no. 577, California Institute of Technology, Division of the Humanities and Social Sciences. https://ideas.repec.org/p/clt/sswopa/577.html
Fudenberg D, Tirole J (1991) Game theory, 1991. Cambridge, Mass. 393(12):80
Hu J, Wellman MP (1998) Multiagent reinforcement learning: theoretical framework and an algorithm. In: International conference on robotics and automation (ICRA)
Raichuk, Stańczyk, Zaj ąc, Bachem, Espeholt, Riquelme, Vincent, Michalski, Bousquet, and Gellykurach2019google Kurach K, Raichuk A, Stańczyk P, Zaj ąc M, Bachem O, Espeholt L, Riquelme C, Vincent D, Michalski M, Bousquet O, Gelly S (2019) Google research football: a novel reinforcement learning environment. Preprint. arXiv:1907.11180
Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the international conference on machine learning (ICML), pp 157–163. https://doi.org/10.1016/b978-1-55860-335-6.50027-1
Lowe R, Wu Y, Tamar A, Harb J, Abbeel OP, Mordatch I (2017) Multi-agent actor-critic for mixed cooperative-competitive environments. In: Advances in neural information processing systems
Nash JF, et al (1950) Equilibrium points in n-person games. Proc Natl Acad Sci 36(1):48–49
Palanisamy P (2019) Multi-agent connected autonomous driving using deep reinforcement learning. Preprint. arXiv:1911.04175
Rapoport A, Chammah AM (1966) The game of chicken. Am Behav Sci 10(3):10–28
Vincent P (1974) Learning the optimal strategy in a zero-sum game. Econometrica 42(5):885–891
Zhang H, Khairy S, Cai LX, Han Z (2018) Resource allocation in unlicensed long term evolution HetNets. Springer, Berlin
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Zhang, H., Zhang, S. (2020). Multi-Agent Reinforcement Learning. In: Dong, H., Ding, Z., Zhang, S. (eds) Deep Reinforcement Learning. Springer, Singapore. https://doi.org/10.1007/978-981-15-4095-0_11
Download citation
DOI: https://doi.org/10.1007/978-981-15-4095-0_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-4094-3
Online ISBN: 978-981-15-4095-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)