Can Agents Acquire Human-Like Behaviors in a Sequential Bargaining Game? – Comparison of Roth’s and Q-Learning Agents –

  • Keiki Takadama
  • Tetsuro Kawai
  • Yuhsuke Koyama
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4442)


This paper addresses agent modeling in multiagent-based simulation (MABS) to explore agents who can reproduce human-like behaviors in the sequential bargaining game, which is more difficult to be reproduced than in the ultimate game (i.e., one time bargaining game). For this purpose, we focus on the Roth’s learning agents who can reproduce human-like behaviors in several simple examples including the ultimate game, and compare simulation results of Roth’s learning agents and Q-learning agents in the sequential bargaining game. Intensive simulations have revealed the following implications: (1) Roth’s basic and three parameter reinforcement learning agents with any type of three action selections (i.e., ε-greed, roulette, and Boltzmann distribution selections) can neither learn consistent behaviors nor acquire sequential negotiation in sequential bargaining game; and (2) Q-learning agents with any type of three action selections, on the other hand, can learn consistent behaviors and acquire sequential negotiation in the same game. However, Q-learning agents cannot reproduce the decreasing trend found in subject experiments.


agent-based simulation agent modeling sequential bargaining game human-like behaviors reinforcement learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Axelrod, R.M.: The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration. Princeton University Press, Princeton (1997)Google Scholar
  2. 2.
    Binmore, K.G.: Game Theory and the Social Contract: Just Playing, vol. 2. The MIT Press, Cambridge (1998)Google Scholar
  3. 3.
    Bosse, T., Jonker, C.M.: Human vs. Computer Behaviour in Multi-Issue Negotiation. In: First International Workshop on Rational, Robust, and Secure Negotiations in Multi-Agent Systems (RRS 2005), pp. 11–24. IEEE Computer Society Press, Los Alamitos (2005)CrossRefGoogle Scholar
  4. 4.
    Erev, I., Roth, A.E.: Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria. The American Economic Review 88(4), 848–881 (1998)Google Scholar
  5. 5.
    Friedman, D., Sunder, S.: Experimental Methods: A Primer for Economists. Cambridge University Press, Cambridge (1994)zbMATHGoogle Scholar
  6. 6.
    Iwasaki, A., Ogawa, K., Yokoo, M., Oda, S.: Reinforcement Learning on Monopolistic Intermediary Games: Subject Experiments and Simulation. The fourth international workshop on Agent-based Approaches in Economic and Social Complex Systems (AESCS 2005), pp. 117–128 (2005)Google Scholar
  7. 7.
    Kagel, J.H., Roth, A.E.: Handbook of Experimental Economics. Princeton University Press, Princeton (1995)Google Scholar
  8. 8.
    Kawai, T., Koyama, Y., Takadama, K.: Modeling Sequential Bargaining Game Agents Towards Human-like Behaviors: Comparing Experimental and Simulation Results. In: The First World Congress of the International Federation for Systems Research (IFSR 2005), pp. 164–166 (2005)Google Scholar
  9. 9.
    Moss, S., Davidsson, P.: Multi-Agent-Based Simulation. In: Moss, S., Davidsson, P. (eds.) MABS 2000. LNCS (LNAI), vol. 1979, Springer, Heidelberg (2001)Google Scholar
  10. 10.
    Muthoo, A.: Bargaining Theory with Applications. Cambridge University Press, Cambridge (1999)zbMATHGoogle Scholar
  11. 11.
    Muthoo, A.: A Non-Technical Introduction to Bargaining Theory. World Economics, 145–166 (2000)Google Scholar
  12. 12.
    Nydegger, R.V., Owen, G.: Two-Person Bargaining: An Experimental Test of the Nash Axioms. International Journal of Game Theory 3(4), 239–249 (1974)zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Ogawa, K., Iwasaki, A., Oda, S., Yokoo, M.: Analysis on the Price-Formation-Process of Monopolistic Broker: Replication of Subject-Experiment by Computer-Experiment. The, JAFEE (Japan Association for Evolutionary Economics) Annual Meeting, 2005 (in Japanese) (2005)Google Scholar
  14. 14.
    Osborne, M.J., Rubinstein, A.: A Course in Game Theory. MIT Press, Cambridge (1994)Google Scholar
  15. 15.
    Roth, A.E., Prasnikar, V., Okuno-Fujiwara, M., Zamir, S.: Bargaining and Market Behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: An Experimental Study. American Economic Review 81(5), 1068–1094 (1991)Google Scholar
  16. 16.
    Roth, A.E., Erev, I.: Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term. Games and Economic Behavior 8(1), 164–212 (1995)zbMATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Rubinstein, A.: Perfect Equilibrium in a Bargaining Model. Econometrica 50(1), 97–109 (1982)zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Schelling, T.C.: The Strategy of Conflict. Harvard University Press (1960)Google Scholar
  19. 19.
    Ståhl, I.: Bargaining Theory, Economics Research Institute at the Stockholm School of Economics (1972)Google Scholar
  20. 20.
    Spulber, D.F.: Market Microstructure -Intermediaries and the theory of the firm. Cambridge University Press, Cambridge (1999)Google Scholar
  21. 21.
    Sutton, R.S., Bart, A.G.: Reinforcement Learning – An Introduction –. The MIT Press, Cambridge (1998)Google Scholar
  22. 22.
    Watkins, C.J.C.H., Dayan, P.: Technical Note: Q-Learning. Machine Learning 8, 55–68 (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Keiki Takadama
    • 1
  • Tetsuro Kawai
    • 2
  • Yuhsuke Koyama
    • 3
  1. 1.The University of Electro-Communications, 1-5-1, Chofugaoka, Chofu, Tokyo 182-8585Japan
  2. 2.Sony Corporation, 6-7-35, Kita-Shinagawa, Shinagawa-ku, Tokyo 141-0001Japan
  3. 3.Tokyo Institute of Technology, 4259 Nagatsuta-cho, Midori-ku, Yokohama 226-8503Japan

Personalised recommendations