Abstract
Although the presence of free communication reduces the complexity of multi-agent POMDPs to that of single-agent POMDPs, in practice, communication is not free and reducing the amount of communication is often desirable. We present a novel approach for using centralized “single-agent” policies in decentralized multi-agent systems by maintaining and reasoning over the possible joint beliefs of the team. We describe how communication is used to integrate local observations into the team belief as needed to improve performance. We show both experimentally and through a detailed example how our approach reduces communication while improving the performance of distributed xecution.
This work has been supported by several grants, including NASA NCC2-1243, and by Rockwell Scientific Co., LLC under subcontract no. B4U528968 and prime contract no. W91 1W6-04-C-0058 with the US Army. This material was based upon work supported under a National Science Foundation Graduate Research Fellowship. The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, by the sponsoring institutions, the U.S. Government or any other entity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Becker, R., Zilberstein, S., Lesser, V., and Goldman, C. V. (2003). Transition-independent decentralized Markov Decision Processes. In International Joint Conference on Autonomous Agents and Multi-agent Systems.
Bernstein, D. S., Zilberstein, S., and Immerman, N. (2000). The complexity of decentralized control of Markov Decision Processes. In Uncertainty in Artificial Intelligence.
Cassandra, A. R. POMDP solver software. http://www.cassandra.org/pomdp/code/index.shtml.
Emery-Montemerlo, R., Gordon, G., Schneider, J., and Thrun, S. (2004). Approximate solutions for partially observable stochastic games with common payoffs. In International Joint Conference on Autonomous Agents and Multi-Agent Systems.
Hansen, E. A., Bernstein, D. S., and Zilberstein, S. (2004). Dynamic programming for partially observable stochastic games. In National Conference on Artificial Intelligence.
Kaelbling, L. P., Littman, M. L., and Cassandra, A. R. (1998). Planning and acting in partially observable domains. Artificial Intelligence.
Littman, M. L., Cassandra, A. R., and Kaelbling, L. P. (1995). Learning policies for partially observable environments: Scaling up. In International Conference on Machine Learning.
Nair, R., Pynadath, D., Yokoo, M., Tambe, M., and Marsella, S. (2003). Taming decentralized POMDPs: Towards efficient policy computation for multiagent settings. In International Joint Conference on Artificial Intelligence.
Nair, R., Roth, M., Yokoo, M., and Tambe, M. (2004). Communication for improving policy computation in distributed POMDPs. In International Joint Conference on Autonomous Agents and Multi-agent Systems.
Papadimitriou, C. H. and Tsitsiklis, J. N. (1987). The complexity of Markov Decision Processes. Mathematics of Operations Research.
Peshkin, L., Kim, K.-E., Meuleau, N., and Kaelbling, L. P. (2000). Learning to cooperate via policy search. In Uncertainty in Artificial Intelligence.
Poupart, P., Ortiz, L. E., and Boutilier, C. (2001). Value-directed sampling methods for monitoring pomdps. In Uncertainty in Artificial Intelligence.
Pynadath, D. V. and Tambe, M. (2002). The communicative Multiagent Team Decision Problem: Analyzing teamwork theories and models. Journal of AI Research.
Thrun, S. (2000). Monte carlo pomdps. In Neural Information Processing Systems.
Xuan, P. and Lesser, V. (2002). Multi-agent policies: From centralized ones to decentralized ones. In International Joint Conference on Autonomous Agents and Multi-agent Systems.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer
About this paper
Cite this paper
Roth, M., Simmons, R., Veloso, M. (2005). Decentralized Communication Strategies for Coordinated Multi-Agent Policies. In: Parker, L.E., Schneider, F.E., Schultz, A.C. (eds) Multi-Robot Systems. From Swarms to Intelligent Automata Volume III. Springer, Dordrecht. https://doi.org/10.1007/1-4020-3389-3_8
Download citation
DOI: https://doi.org/10.1007/1-4020-3389-3_8
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-3388-9
Online ISBN: 978-1-4020-3389-6
eBook Packages: EngineeringEngineering (R0)