Abstract
Real world multiagent coordination problems are important issues for reinforcement learning techniques. In general, these problems are partially observable and this characteristic makes the solution computation intractable. Most of the existing approaches calculate exact or approximate solutions using the world model for only one agent. To handle a special case of partial observability, this article presents an approach to approximate the policy measuring a degree of observability for pure cooperative vehicle coordination problem. We compare empirically the performance of the learned policy for totally observable problems and performances of policies for different degrees of observability. If each degree of observability is associated with communication costs, multiagent system designers are able to choose a compromise between the performance of the policy and the cost to obtain the associated degree of observability of the problem. Finally, we show how the available space, surrounding an agent, influence the required degree of observability for near-optimal solution.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Xuan, P., Lesser, V., Zilberstein, S.: Communication decisions in multi-agent cooperation: Model and experiments. In: Müller, J.P., Andre, E., Sen, S., Frasson, C. (eds.) Fifth International Conference on Autonomous Agents, Montreal, Canada, pp. 616–623. ACM Press, New York (2001)
Pynadath, D.V., Tambe, M.: The communicative multiagent team decision problem: Analyzing teamwork theories and models. Journal of AI Research 16, 389–423 (2002)
Aras, R., Dutech, A., Charpillet, F.: Cooperation in Stochastic Games through Communication. In: fourth Internantional Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2005) (poster), Utrecht, Nederlands (2005)
Verbeeck, K.: Exploring Selfish Reinforcement Learning in Stochastic Non-Zero Sum Games. PhD thesis, Vrije Universiteit Brussel (2004)
Bui, H.H.: An Approach to Coordinating Team of Agents under Incomplete Information. PhD thesis, Curtin University of Technology (1998)
Littman, M.: Friend-or-Foe Q-learning in General-Sum Games. In: Kaufmann, M. (ed.) Eighteenth International Conference on Machine Learning, pp. 322–328 (2001)
Wang, X., Sandholm, T.W.: Reinforcement Learning to Play An Optimal Nash Equilibrium in Team Markov Games. In: 16th Neural Information Processing Systems: Natural and Synthetic conference (2002)
Moriarty, D.E., Langley, P.: Distributed learning of lane-selection strategies for traffic management. Technical report, Palo Alto, CA, 98-2 (1998)
Varaiya, P.: Smart cars on smart roads: Problems of control. IEEE Transactions on Automatic Control 38(2), 195–207 (1993)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Basar, T., Olsder, G.J.: Dynamic Noncooperative Game Theory, 2nd edn. Classics In Applied Mathematics (1999)
Emery-Montermerlo, R.: Game-theoretic control for robot teams. Technical Report CMU-RI-TR-05-36, Robotics Institute, Carnegie Mellon University (2005)
Kok, J.R., Vlassis, N.: Sparse Cooperative Q-learning. In: Greiner, R., Schuurmans, D. (eds.) Proc. of the 21st Int. Conf. on Machine Learning, Banff, Canada, pp. 481–488. ACM, New York (2004)
Fulda, N., Ventura, D.: Dynamic Joint Action Perception for Q-Learning Agents. In: 2003 International Conference on Machine Learning and Applications (2003)
Dolgov, D., Durfee, E.H.: Graphical models in local, asymmetric multi-agent Markov decision processes. In: Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS-2004) (2004)
Ünsal, C., Kachroo, P., Bay, J.S.: Simulation study of multiple intelligent vehicle control using stochastic learning automata. IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans 29(1), 120–128 (1999)
Pendrith, M.D.: Distributed reinforcement learning for a traffic engineering application. In: Fourth International Conference on Autonomous Agents, pp. 404–411 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Laumonier, J., Chaib-draa, B. (2006). Partial Local FriendQ Multiagent Learning: Application to Team Automobile Coordination Problem. In: Lamontagne, L., Marchand, M. (eds) Advances in Artificial Intelligence. Canadian AI 2006. Lecture Notes in Computer Science(), vol 4013. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11766247_31
Download citation
DOI: https://doi.org/10.1007/11766247_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34628-9
Online ISBN: 978-3-540-34630-2
eBook Packages: Computer ScienceComputer Science (R0)