Abstract
This paper investigates the effect of different rationality assumptions on the performance of co-learning by multiple agents in extensive games. Extensive games involve sequences of steps and close interactions between agents, and are thus more difficult than more commonly investigated (one-step) strategic games. Rationality assumptions may thus have more complicated influences on learning, e.g., improving performance sometimes while hurting performance some other times. In testing different levels of rationality assumptions, a “double estimation” method for reinforcement learning suitable for extensive games is developed, whereby an agent learns not only its own value function but also those of other agents. Experiments based on such a reinforcement learning method are carried out using several typical examples of games. Our results indeed showed a complex pattern of effects resulting from (different levels of) rationality assumptions.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
R. Bellman, (1957). Dynamic Programming. Princeton University Press, Princeton, NJ.
D. Bertsekas and J. Tsitsiklis, (1996). Neuro-Dynamic Programming. Athena Scientific, Belmont, MA.
C. Claus and C. Boutilier, (1998). The dynamics of reinforcement learning in cooperative multiagent systems. Proceedings of A AAI’ 98. AAAI Press, San Mateo, CA.
D. Fudenberg and D. Levine, (1998). The Theory of Learning in Games. MIT Press, Cambridge, MA.
T. Haynes and S. Sen, (1996). Co-adaptation in a team. International Journal of Computational Intelligence and Organizations.
J. Hu and M. Wellman, (1998 a). Multiagent reinforcement learning: theore-tical framework and an algorithm. Proceedings of International Conference on Machine Learning, 242–250. Morgan Kaufmann, San Francisco, CA.
J. Hu and M. Wellman, (1998 b). Online learning about other agents in a dynamic multiagent system. Second International Conference on Autonomous Agents. ACM Press, New York.
M. Littman, (1994). Markov games as a framework for multi-agent reinfocement learning. Proc. of the 11th International conference on Machine Learning, 157–163. Morgan Kaufmann, San Francisco, CA.
M. Osborne and A. Rubinstein, (1994). A Course on Game Theory. MIT Press, Cambridge, MA.
R. Salustowicz, M. Wiering, and J. Schmidhuber, (1998). Learning team strategies: soccer case studies. Machine Learning. 1998
S. Sen and M. Sekaran, (1998). Individual learning of coordination knowledge. Journal of Experimental and Theoretical Artificial Intelligence, 10, 333–356.
Y. Shoham and M. Tennenholtz, (1994). Co-learning and the evolution of social activity. Technical Report STAN-CS-TR-94-1511, Stanford University.
S. Singh, T. Jaakkola, and M. Jordan, (1994). Reinforcement learning with soft state aggregation. In: S.J. Hanson J. Cowan and C. L. Giles, eds. Advances in Neural Information Processing Systems 7. Morgan Kaufmann, San Mateo, CA.
R. Sun and T. Peterson, (1999). Multi-agent reinforcement learning: weighting and partitioning. Neural Networks, Vol. 12, No. 4–5. pp. 127–153.
R. Sun and C. Sessions, (1999). Bidding in reinforcement learning: a paradigm for multi-agent systems. Proc. of The Third International Conference on Autonomous Agents (AGENTS’99), Seattle, WA.
M. Tan, (1993). Multi-agent reinforcement learning: independent vs. cooperative agents. Proceedings of Machine Learning Conference. Morgan Kaufmann, San Francisco, CA.
C. Tham, (1995). Reinforcement learning of multiple tasks using a hierarchical CMAC architecture. Robotics and Autonomous Systems. 15, 247–274.
M. Vidal and E.H. Durfee, (1998). Learning nested models in an information economy. Journal of Experimental and Theoretical Artificial Intelligence, 10(3), 291–308.
C. Watkins, (1989). Learning with Delayed Rewards. Ph.D Thesis, Cambridge University, Cambridge, UK.
G. Weiss, (1995). Distributed reinforcement learning. Robotics and Autonomous Systems, 15, 135–142.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sun, R., Qi, D. (2000). Rationality Assumptions and Optimality of Co-learning. In: Zhang, C., Soo, VW. (eds) Design and Applications of Intelligent Agents. PRIMA 2000. Lecture Notes in Computer Science(), vol 1881. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44594-3_5
Download citation
DOI: https://doi.org/10.1007/3-540-44594-3_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67911-0
Online ISBN: 978-3-540-44594-4
eBook Packages: Springer Book Archive