Advertisement

The Role of Execution Errors in Populations of Ultimatum Bargaining Agents

  • Fernando P. SantosEmail author
  • Jorge M. Pacheco
  • Ana Paiva
  • Francisco C. Santos
Conference paper
First Online:
  • 348 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10238)

Abstract

The design of artificial intelligent agents is frequently accomplished by equipping individuals with mechanisms to choose actions that maximise a subjective utility function. This way, the implementation of behavioural errors, that systematically prevent agents from using optimal strategies, often seems baseless. In this paper, we employ an analytical framework to study a population of Proposers and Responders, with conflicting interests, that co-evolve by playing the prototypical Ultimatum Game. This framework allows to consider an arbitrary discretisation of the strategy space, and allows us to describe the dynamical impact of individual mistakes by Responders, on the collective success of this population. Conveniently, this method can be used to analyse other continuous strategy interactions. In the case of Ultimatum Game, we show analytically how seemingly disadvantageous errors empower Responders and become the source of individual and collective long-term success, leading to a fairer distribution of gains. This conclusion remains valid for a wide range of selection pressures, population sizes and mutation rates.

Keywords

Multiagent System Average Fitness Subgame Perfect Equilibrium Ultimatum Game Replicator Dynamic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

References

  1. 1.
    Azaria, A., Richardson, A., Rosenfeld, A.: Autonomous agents and human cultures in the trust-revenge game. Auton. Agents Multi Agent Syst. 30(3), 1–20 (2015)Google Scholar
  2. 2.
    Bloembergen, D., Tuyls, K., Hennes, D., Kaisers, M.: Evolutionary dynamics of multiagent learning: a survey. J. Artif. Intell. Res. 53, 659–697 (2015)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Börgers, T., Sarin, R.: Learning through reinforcement and replicator dynamics. J. Econ. Theor. 77(1), 1–14 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Camerer, C.: Behavioral Game Theory: Experiments in Strategic Interaction. Princeton University Press, Princeton (2003)zbMATHGoogle Scholar
  5. 5.
    Encarnao, S., Santos, F.P., Santos, F.C., Blass, V., Pacheco, J.M., Portugali, J.: Paradigm shifts and the interplay between state, business and civil sectors. R. Soc. Open Sci. 3, 160753 (2016)CrossRefGoogle Scholar
  6. 6.
    Erev, I., Roth, A.E.: Maximization, learning, and economic behavior. Proc. Natl. Acad. Sci. USA 111, 10818–10825 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Fehr, E., Fischbacher, U.: The nature of human altruism. Nature 425(6960), 785–791 (2003)CrossRefGoogle Scholar
  8. 8.
    Fehr, E., Schmidt, K.M.: A theory of fairness, competition, and cooperation. Q. J. Econ. 114(3), 817–868 (1999)CrossRefzbMATHGoogle Scholar
  9. 9.
    Fudenberg, D., Imhof, L.A.: Imitation processes with small mutations. J. Econ. Theor. 131(1), 251–262 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Gale, J., Binmore, K.G., Samuelson, L.: Learning to be imperfect: the ultimatum game. Game Econ. Behav. 8(1), 56–90 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Güth, W., Schmittberger, R., Schwarze, B.: An experimental analysis of ultimatum bargaining. J. Econ. Behav. Organ. 3(4), 367–388 (1982)CrossRefGoogle Scholar
  12. 12.
    Heyes, C.M.: Social learning in animals: categories and mechanisms. Biol. Rev. 69(2), 207–231 (1994)CrossRefGoogle Scholar
  13. 13.
    Imhof, L.A., Fudenberg, D., Nowak, M.A.: Evolutionary cycles of cooperation and defection. Proc. Natl. Acad. Sci. USA 102(31), 10797–10800 (2005)CrossRefGoogle Scholar
  14. 14.
    Jennings, N.R., Faratin, P., Lomuscio, A.R., Parsons, S., Wooldridge, M.J., Sierra, C.: Automated negotiation: prospects, methods and challenges. Group Decis. Negot. 10(2), 199–215 (2001)CrossRefGoogle Scholar
  15. 15.
    Kahneman, D.: Maps of bounded rationality: psychology for behavioral economics. Am. Econ. Rev. 93(5), 1449–1475 (2003)CrossRefGoogle Scholar
  16. 16.
    Kraus, S.: Strategic Negotiation in Multiagent Environments. MIT press, Cambridge (2001)zbMATHGoogle Scholar
  17. 17.
    Lin, R., Kraus, S.: Can automated agents proficiently negotiate with humans? Commun. ACM 53(1), 78–88 (2010)CrossRefGoogle Scholar
  18. 18.
    Maynard-Smith, J., Price, G.: The logic of animal conflict. Nature 246, 15 (1973)CrossRefGoogle Scholar
  19. 19.
    Miller, J.H., Page, S.E.: Complex Adaptive Systems: An Introduction to Computational Models of Social Life. Princeton University Press, Princeton (2009)Google Scholar
  20. 20.
    Mitchell, M.: Complexity: A Guided Tour. Oxford University Press, Oxford (2009)zbMATHGoogle Scholar
  21. 21.
    Nowak, M.A.: Evolutionary Dynamics: Exploring the Equations of Life. Harvard University Press, Cambridge (2006)zbMATHGoogle Scholar
  22. 22.
    Nowak, M.A., Page, K.M., Sigmund, K.: Fairness versus reason in the ultimatum game. Science 289(5485), 1773–1775 (2000)CrossRefGoogle Scholar
  23. 23.
    Nowak, M.A., Sasaki, A., Taylor, C., Fudenberg, D.: Emergence of cooperation and evolutionary stability in finite populations. Nature 428(6983), 646–650 (2004)CrossRefGoogle Scholar
  24. 24.
    Pinheiro, F.L., Pacheco, J.M., Santos, F.C.: From local to global dilemmas in social networks. PLoS ONE 7(2), e32114 (2012)CrossRefGoogle Scholar
  25. 25.
    Pinheiro, F.L., Santos, M.D., Santos, F.C., Pacheco, J.M.: Origin of peer influence in social networks. Phys. Rev. Lett. 112(9), 098702 (2014)CrossRefGoogle Scholar
  26. 26.
    Rand, D.G., Tarnita, C.E., Ohtsuki, H., Nowak, M.A.: Evolution of fairness in the one-shot anonymous ultimatum game. Proc. Natl. Acad. Sci. USA 110(7), 2581–2586 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Rosenfeld, A., Kraus, S.: Modeling agents through bounded rationality theories. In: IJCAI 2009 Proceedings of the 21st International Joint Conference on Artificial Intelligence, vol. 9, pp. 264–271 (2009)Google Scholar
  28. 28.
    Rosenfeld, A., Zuckerman, I., Azaria, A., Kraus, S.: Combining psychological models with machine learning to better predict people’s decisions. Synthese 189(1), 81–93 (2012)CrossRefGoogle Scholar
  29. 29.
    Santos, F.C., Pacheco, J.M.: Risk of collective failure provides an escape from the tragedy of the commons. Proc. Natl. Acad. Sci. USA 108(26), 10421–10425 (2011)CrossRefGoogle Scholar
  30. 30.
    Santos, F.C., Pacheco, J.M., Skyrms, B.: Co-evolution of pre-play signaling and cooperation. J. Theor. Biol. 274(1), 30–35 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Santos, F.P., Santos, F.C., Melo, F.S., Paiva, A., Pacheco, J.M.: Dynamics of fairness in groups of autonomous learning agents. In: Osman, N., Sierra, C. (eds.) AAMAS 2016. LNCS (LNAI), vol. 10002, pp. 107–126. Springer, Cham (2016). doi: 10.1007/978-3-319-46882-2_7 CrossRefGoogle Scholar
  32. 32.
    Santos, F.P., Santos, F.C., Melo, F.S. Paiva, A., Pacheco, J.M.: Learning to be fair in multiplayer ultimatum games. In: Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems, International Foundation for Autonomous Agents and Multiagent Systems, pp. 1381–1382 (2016)Google Scholar
  33. 33.
    Santos, F.P., Santos, F.C., Pacheco, J.M.: Social norms of cooperation in small-scale societies. PLoS Comput. Biol. 12(1), e1004709 (2016)CrossRefGoogle Scholar
  34. 34.
    Santos, F.P., Santos, F.C., Paiva, A., Pacheco, J.M.: Evolutionary dynamics of group fairness. J. Theor. Biol. 378, 96–102 (2015)CrossRefzbMATHGoogle Scholar
  35. 35.
    Santos, F.P., Santos, F.C., Paiva, A., Pacheco, J.M.: Execution errors enable the evolution of fairness in the ultimatum game. In: Proceedings of the 22nd European Conference on Artificial Intelligence (ECAI 2016), vol. 285, p. 1592. IOS Press (2016)Google Scholar
  36. 36.
    Schelling, T.C.: Micromotives and Macrobehavior. WW Norton & Company, New York City (2006)Google Scholar
  37. 37.
    Shoham, Y., Powers, R., Grenager, T.: If multi-agent learning is the answer, what is the question? Artif. Intell. 171(7), 365–377 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Sigmund, K.: The Calculus of Selfishness. Princeton University Press, Princeton (2010)CrossRefzbMATHGoogle Scholar
  39. 39.
    Sutton, R.S., Barto, A.G.: Introduction to Reinforcement Learning. MIT Press, Cambridge (1998)Google Scholar
  40. 40.
    Taylor, P.D., Jonker, L.B.: Evolutionary stable strategies and game dynamics. Math. Biosci. 40(1), 145–156 (1978)MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Traulsen, A., Nowak, M.A.: Evolution of cooperation by multilevel selection. Proc. Natl. Acad. Sci. USA 103(29), 10952–10955 (2006)CrossRefGoogle Scholar
  42. 42.
    Traulsen, A., Nowak, M.A., Pacheco, J.M.: Stochastic dynamics of invasion and fixation. Phys. Rev. E 74(1), 011909 (2006)CrossRefGoogle Scholar
  43. 43.
    Tuyls, K., Verbeeck, K., Lenaerts, T.: A selection-mutation model for q-learning in multiagent systems. In: AAMAS 2003 Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 693–700 (2003)Google Scholar
  44. 44.
    Van Segbroeck, S., Pacheco, J.M., Lenaerts, T., Santos, F.C.: Emergence of fairness in repeated group interactions. Phys. Rev. Lett. 108(15), 158104 (2012)CrossRefGoogle Scholar
  45. 45.
    Vasconcelos, V.V., Santos, F.C., Pacheco, J.M., Levin, S.A.: Climate policies under wealth inequality. Proc. Natl. Acad. Sci. USA 111(6), 2212–2216 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Fernando P. Santos
    • 1
    • 2
    Email author
  • Jorge M. Pacheco
    • 2
    • 3
  • Ana Paiva
    • 1
  • Francisco C. Santos
    • 1
    • 2
  1. 1.INESC-ID and Instituto Superior TécnicoUniversidade de LisboaPorto SalvoPortugal
  2. 2.ATP-GroupPorto SalvoPortugal
  3. 3.CBMA and Departamento de Matemática e AplicaçõesUniversidade do MinhoBragaPortugal

Personalised recommendations

Cite paper

Buy options