Advertisement

Dynamics of Fairness in Groups of Autonomous Learning Agents

  • Fernando P. SantosEmail author
  • Francisco C. Santos
  • Francisco S. Melo
  • Ana Paiva
  • Jorge M. Pacheco
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10002)

Abstract

Fairness plays a determinant role in human decisions and definitely shapes social preferences. This is evident when groups of individuals need to divide a given resource. Notwithstanding, computational models seeking to capture the origins and effects of human fairness often assume the simpler case of two person interactions. Here we study a multiplayer extension of the well-known Ultimatum Game. This game allows us to study fair behaviors in a group setting: a proposal is made to a group of Responders and the overall acceptance depends on reaching a minimum number of individual acceptances. In order to capture the effects of different group environments on the human propensity to be fair, we model a population of learning agents interacting through the multiplayer ultimatum game. We show that, contrarily to what would happen with fully rational agents, learning agents coordinate their behavior into different strategies, depending on factors such as the minimum number of accepting Responders (to achieve group acceptance) or the group size. Overall, our simulations show that stringent group criteria leverage fairer proposals. We find these conclusions robust to (i) asynchronous and synchronous strategy updates, (ii) initially biased agents, (iii) different group payoff division paradigms and (iv) a wide range of error and forgetting rates.

Keywords

Nash Equilibrium Multiagent System Artificial Agent Ultimatum Game Strategy Profile 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

This research was supported by Fundação para a Ciência e Tecnologia (FCT Portugal) through grants SFRH/BD/94736/2013, PTDC/EEI-SII/5081/2014, PTDC/MAT/STA/3358/2014 and by multi-annual funding of CBMA and INESC-ID (under the projects UID/BIA/04050/2013 and UID/CEC/50021/2013 provided by FCT).

References

  1. 1.
    Bloembergen, D., Tuyls, K., Hennes, D., Kaisers, M.: Evolutionary dynamics of multi-agent learning: a survey. J. Artif. Intell. Res. 53, 659–697 (2015)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Blount, S.: When social outcomes aren’t fair: the effect of causal attributions on preferences. Organ. Behav. Hum. Decis. Process. 63(2), 131–144 (1995)CrossRefGoogle Scholar
  3. 3.
    Bornstein, G., Yaniv, I.: Individual and group behavior in the ultimatum game: are groups more rational players? Exp. Econ. 1(1), 101–108 (1998)zbMATHCrossRefGoogle Scholar
  4. 4.
    Cimini, G., Sánchez, A.: Learning dynamics explains human behaviour in prisoner’s dilemma on networks. J. R. Soc. Interface 11(94), 20131186 (2014)CrossRefGoogle Scholar
  5. 5.
    de Melo, C.M., Carnevale, P., Gratch, J.: The effect of expression of anger and happiness in computer agents on negotiations with humans. In: The 10th International Conference on Autonomous Agents and Multiagent Systems, International Foundation for Autonomous Agents and Multiagent Systems, pp. 937–944 (2011)Google Scholar
  6. 6.
    Duch, R., Przepiorka, W., Stevenson, R.: Responsibility attribution for collective decision makers. Am. J. Polit. Sci. 59(2), 372–389 (2015)CrossRefGoogle Scholar
  7. 7.
    Elbittar, A., Gomberg, A., Sour, L.: Group decision-making and voting in ultimatum bargaining: an experimental study. B.E. J. Econ. Anal. Policy 11(1), 53 (2011)Google Scholar
  8. 8.
    Erev, I., Roth, A.E.: Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Am. Econ. Rev. 88, 848–881 (1998)Google Scholar
  9. 9.
    Fischbacher, U., Fong, C.M., Fehr, E.: Fairness, errors and the power of competition. J. Econ. Behav. Organ. 72(1), 527–545 (2009)CrossRefGoogle Scholar
  10. 10.
    Forsythe, R., Horowitz, J.L., Savin, N.E., Sefton, M.: Fairness in simple bargaining experiments. Games Econ. Behav. 6(3), 347–369 (1994)zbMATHCrossRefGoogle Scholar
  11. 11.
    Fudenberg, D., Levine, D.K.: The Theory of Learning in Games. MIT press, Cambridge (1998)zbMATHGoogle Scholar
  12. 12.
    Grosskopf, B.: Reinforcement and directional learning in the ultimatum game with responder competition. Exp. Econ. 6(2), 141–158 (2003)zbMATHCrossRefGoogle Scholar
  13. 13.
    Güth, W., Schmittberger, R., Schwarze, B.: An experimental analysis of ultimatum bargaining. J. Econ. Behav. Organ. 3(4), 367–388 (1982)CrossRefGoogle Scholar
  14. 14.
    Hagan, J.D., Everts, P.P., Fukui, H., Stempel, J.D.: Foreign policy by coalition: deadlock, compromise, and anarchy. Int. Stud. Rev. 3(2), 169–216 (2001)CrossRefGoogle Scholar
  15. 15.
    Hamilton, W.D.: Innate social aptitudes of man: an approach from evolutionary genetics. In: Fox, R. (ed.) Biosocial Anthropology, pp. 133–155. Wiley, New York (1975)Google Scholar
  16. 16.
    Hoffman, E., McCabe, K., Smith, V.L.: Social distance and other-regarding behavior in dictator games. Am. Econ. Rev. 86, 653–660 (1996)Google Scholar
  17. 17.
    Iranzo, J., Román, J., Sánchez, A.: The spatial ultimatum game revisited. J. Theor. Biol. 278(1), 1–10 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Jennings, N.R., Faratin, P., Lomuscio, A.R., Parsons, S., Wooldridge, M.J., Sierra, C.: Automated negotiation: prospects, methods and challenges. Group Decis. Negot. 10(2), 199–215 (2001)CrossRefGoogle Scholar
  19. 19.
    Jing, X., Xie, J.: Group buying: a new mechanism for selling through social interactions. Manage. Sci. 57(8), 1354–1372 (2011)zbMATHCrossRefGoogle Scholar
  20. 20.
    Kauffman, R.J., Lai, H., Ho, C.-T.: Incentive mechanisms, fairness and participation in online group-buying auctions. Electron. Commer. Res. Appl. 9(3), 249–262 (2010)CrossRefGoogle Scholar
  21. 21.
    Lin, R., Kraus, S.: Can automated agents proficiently negotiate with humans? Commun. ACM 53(1), 78–88 (2010)CrossRefGoogle Scholar
  22. 22.
    Macy, M.W., Flache, A.: Learning dynamics in social dilemmas. Proc. Natl. Acad. Sci. 99, 7229–7236 (2002)CrossRefGoogle Scholar
  23. 23.
    Messick, D.M., Moore, D.A., Bazerman, M.H.: Ultimatum bargaining with a group: underestimating the importance of the decision rule. Organ. Behav. Hum. Decis. Process. 69(2), 87–101 (1997)CrossRefGoogle Scholar
  24. 24.
    Newell, A., Rosenbloom, P.S.: Mechanisms of skill acquisition and the law of practice. Cogn. Skills Acquisition 1, 1–55 (1981)Google Scholar
  25. 25.
    Nowak, M.A., Page, K.M., Sigmund, K.: Fairness versus reason in the ultimatum game. Science 289(5485), 1773–1775 (2000)CrossRefGoogle Scholar
  26. 26.
    Oosterbeek, H., Sloof, R., Van De Kuilen, G.: Cultural differences in ultimatum game experiments: evidence from a meta-analysis. Exp. Econ. 7(2), 171–188 (2004)zbMATHCrossRefGoogle Scholar
  27. 27.
    Osborne, M.J.: An Introduction to Game Theory. Oxford University Press, New York (2004)Google Scholar
  28. 28.
    Pacheco, J.M., Santos, F.C., Souza, M.O., Skyrms, B.: Evolutionary dynamics of collective action. In: Chalub, F.A.C.C., Rodrigues, J.F. (eds.) The Mathematics of Darwin’s Legacy, pp. 119–138. Springer, Basel (2011)CrossRefGoogle Scholar
  29. 29.
    Page, K.M., Nowak, M.A.: Empathy leads to fairness. Bull. Math. Biol. 64(6), 1101–1116 (2002)zbMATHCrossRefGoogle Scholar
  30. 30.
    Page, K.M., Nowak, M.A., Sigmund, K.: The spatial ultimatum game. Proc. R. Soc. Lond. B Biol. Sci. 267(1458), 2177–2182 (2000)CrossRefGoogle Scholar
  31. 31.
    Pinheiro, F.L., Santos, M.D., Santos, F.C., Pacheco, J.M.: Origin of peer influence in social networks. Phys. Rev. Lett. 112(9), 098702 (2014)CrossRefGoogle Scholar
  32. 32.
    Rand, D.G., Tarnita, C.E., Ohtsuki, H., Nowak, M.A.: Evolution of fairness in the one-shot anonymous ultimatum game. Proc. Natl. Acad. Sci. 110(7), 2581–2586 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  33. 33.
    Rendell, L., Boyd, R., Cownden, D., Enquist, M., Eriksson, K., Feldman, M.W., Fogarty, L., Ghirlanda, S., Lillicrap, T., Laland, K.N.: Why copy others? insights from the social learning strategies tournament. Science 328(5975), 208–213 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  34. 34.
    Rosenfeld, A., Zuckerman, I., Segal-Halevi, E., Drein, O., Kraus, S.: Negochat-a: a chat-based negotiation agent with bounded rationality. Auton. Agent. Multi-Agent Syst. 30(1), 60–81 (2016)CrossRefGoogle Scholar
  35. 35.
    Roth, A.E., Erev, I.: Learning in extensive-form games: experimental data and simple dynamic models in the intermediate term. Games Econ. Behav. 8(1), 164–212 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  36. 36.
    Santos, F.P., Santos, F.C., Melo, F.S., Paiva, A., Pacheco, J.M.: Learning to be fair in multiplayer ultimatum games. In: Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems, International Foundation for Autonomous Agents and Multiagent Systems, pp. 1381–1382 (2016)Google Scholar
  37. 37.
    Santos, F.P., Santos, F.C., Paiva, A.: The evolutionary perks of being irrational. In: Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, International Foundation for Autonomous Agents and Multiagent Systems, pp. 1847–1848 (2015)Google Scholar
  38. 38.
    Santos, F.P., Santos, F.C., Paiva, A., Pacheco, J.M.: Evolutionary dynamics of group fairness. J. Theor. Biol. 378, 96–102 (2015)zbMATHCrossRefGoogle Scholar
  39. 39.
    Segal-Halevi, E., Hassidim, A., Aumann, Y.: Waste makes haste: bounded time protocols for envy-free cake cutting with free disposal. In: Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, International Foundation for Autonomous Agents and Multiagent Systems, pp. 901–908 (2015)Google Scholar
  40. 40.
    Sequeira, P., Melo, F.S., Paiva, A.: Emergence of emotional appraisal signals in reinforcement learning agents. Auton. Agents Multi-Agent Syst. 29(4), 537–568 (2014)CrossRefGoogle Scholar
  41. 41.
    Sigmund, K.: The Calculus of Selfishness. Princeton University Press, Princeton (2010)zbMATHCrossRefGoogle Scholar
  42. 42.
    Sinatra, R., Iranzo, J., Gomez-Gardenes, J., Floria, L.M., Latora, V., Moreno, Y.: The ultimatum game in complex networks. J. Stat. Mech. Theory Exp. 2009(09), P09012 (2009)CrossRefGoogle Scholar
  43. 43.
    Skyrms, B.: Signals: Evolution, Learning, and Information. Oxford University Press, Oxford (2010)CrossRefGoogle Scholar
  44. 44.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  45. 45.
    Szolnoki, A., Perc, M., Szabó, G.: Defense mechanisms of empathetic players in the spatial ultimatum game. Phys. Rev. Lett. 109(7), 078701 (2012)CrossRefGoogle Scholar
  46. 46.
    Thaler, R.H.: Anomalies: the ultimatum game. J. Econ. Perspect. 2, 195–206 (1988)CrossRefGoogle Scholar
  47. 47.
    Thorndike, E.L.: Animal intelligence: an experimental study of the associative processes in animals. In: The Psychological Review: Monograph Supplements, (4), i (1898)Google Scholar
  48. 48.
    Van Segbroeck, S., De Jong, S., Nowé, A., Santos, F.C., Lenaerts, T.: Learning to coordinate in complex networks. Adapt. Behav. 18(5), 416–427 (2010)CrossRefGoogle Scholar
  49. 49.
    Van Segbroeck, S., Pacheco, J.M., Lenaerts, T., Santos, F.C.: Emergence of fairness in repeated group interactions. Phys. Rev. Lett. 108(15), 158104 (2012)CrossRefGoogle Scholar
  50. 50.
    Weibull, J.W.: Evolutionary Game Theory. MIT Press, Cambridge (1997)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Fernando P. Santos
    • 1
    • 2
    Email author
  • Francisco C. Santos
    • 1
    • 2
  • Francisco S. Melo
    • 1
  • Ana Paiva
    • 1
  • Jorge M. Pacheco
    • 2
    • 3
  1. 1.INESC-ID and Instituto Superior TécnicoUniversidade de LisboaPorto SalvoPortugal
  2. 2.ATP-GroupPorto SalvoPortugal
  3. 3.CBMA and Departamento de Matemática e AplicaçõesUniversidade do MinhoBragaPortugal

Personalised recommendations

Cite paper

Buy options