Towards Large-Scale Optimization of Iterated Prisoner Dilemma Strategies

  • Grażyna Starzec
  • Mateusz Starzec
  • Aleksander ByrskiEmail author
  • Marek Kisiel-Dorohinicki
  • Juan C. Burguillo
  • Tom Lenaerts
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11370)


The Iterated Prisoner’s Dilemma (IPD) game is a one of the most popular subjects of study in game theory. Numerous experiments have investigated many properties of this game over the last decades. However, topics related to the simulation scale did not always play a significant role in such experimental work. The main contribution of this paper is the optimization of IPD strategies performed in a distributed actor-based computing and simulation environment. Besides showing the scalability and robustness of the framework, we also dive into details of some key simulations, analyzing the most successful strategies obtained.


Iterated prisoner dilemma Parallel simulation Optimization 



This research was supported by AGH University of Science and Technology Statutory Project.


  1. 1.
    Axelrod, R.: The Evolution of Cooperation. Basic Books, New York (2006)zbMATHGoogle Scholar
  2. 2.
    Rapoport, A., Chammah, A.M.: Prisoner’s Dilemma: A Study in Conflict and Cooperation. University of Michigan Press (1965)Google Scholar
  3. 3.
    Roth, A., Murnighan, J.: Equilibrium behavior and repeated play of the prisoner’s dilemma. J. Math. Psychol. 17(2), 189–198 (1978)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Fogel, D.: Evolving behaviors in the iterated prisoner’s dilemma. Evol. Comput. 1, 77–97 (1993)CrossRefGoogle Scholar
  5. 5.
    Kendall, G., Yao, X., Chong, S.: The Iterated Prisoners’ Dilemma: 20 Years on. World Scientific, Singapore (2006)zbMATHGoogle Scholar
  6. 6.
    Van Veelen, M., Garcia, J., Rand, D., Nowak, M.: Direct reciprocity in structured populations. Proc. Natl. Acad. Sci. 109(25), 9929–9934 (2012)CrossRefGoogle Scholar
  7. 7.
    Peleteiro, A., Burguillo, J.C., Chong, S.Y.: Exploring indirect reciprocity in complex networks using coalitions and rewiring. In: Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems, AAMAS 2014, Richland, SC, pp. 669–676. International Foundation for Autonomous Agents and Multiagent Systems (2014)Google Scholar
  8. 8.
    Wellman, M.: Putting the agent in agent-based modeling. Auton. Agent. Multi-Agent Syst. 30, 1175–1189 (2016)CrossRefGoogle Scholar
  9. 9.
    Wiedenbeck, B., Wellman, M.: Scaling simulation-based game analysis through deviation- preserving reduction. In: Proceedings of 11th International Conference on Autonomous Agents and Multi-Agent Systems. ACM (2012)Google Scholar
  10. 10.
    Faber, L., Pietak, K., Byrski, A., Kisiel-Dorohinicki, M.: Agent-based simulation in AgE framework. In: Byrski, A., Oplatková, Z., Carvalho, M., Kisiel-Dorohinicki, M. (eds.) Advances in Intelligent Modelling and Simulation: Simulation Tools and Applications, vol. 416, pp. 55–83. Springer, Heidelberg (2012). Scholar
  11. 11.
    Kisiel-Dorohinicki, M.: Agent-based models and platforms for parallel evolutionary algorithms. In: Bubak, M., van Albada, G.D., Sloot, P.M.A., Dongarra, J. (eds.) ICCS 2004. LNCS, vol. 3038, pp. 646–653. Springer, Heidelberg (2004). Scholar
  12. 12.
    Hewitt, C., Bishop, P., Steiger, R.: A universal modular actor formalism for artificial intelligence. In: Proceedings of the 3rd International Joint Conference on Artificial Intelligence. IJCAI 1973, San Francisco, CA, USA, pp. 235–245. Morgan Kaufmann Publishers Inc. (1973)Google Scholar
  13. 13.
    Agha, G.: Actors: A Model of Concurrent Computation in Distributed Systems. MIT Press, Cambridge (1986)Google Scholar
  14. 14.
    Haller, P., Odersky, M.: Scala actors: unifying thread-based and event-based programming. Theoret. Comput. Sci. 410(2), 202–220 (2009)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Snijders, T.A., van de Bunt, G.G., Steglich, C.E.: Introduction to stochastic actor-based models for network dynamics. Soc. Netw. 32(1), 44–60 (2010). Dynamics of Social NetworksCrossRefGoogle Scholar
  16. 16.
    Esposito, A., Loia, V.: Integrating concurrency control and distributed data into workflow frameworks: an actor model perspective. In: 2000 IEEE International Conference on Systems, Man, and Cybernetics, vol. 3, pp. 2110–2114 (2000)Google Scholar
  17. 17.
    Skiba, G., et al.: Flexible asynchronous simulation of iterated prisoner’s dilemma based on actor model. Simul. Model. Pract. Theory 83, 75–92 (2018)CrossRefGoogle Scholar
  18. 18.
    Peleteiro, A., Burguillo, J.C., Luck, M., Arcos, J.L., Rodígruez-Aguilar, J.A.: Using reputation and adaptive coalitions to support collaboration in competitive environments. Eng. Appl. Artif. Intell. 45, 325–338 (2015)CrossRefGoogle Scholar
  19. 19.
    Peleteiro, A., Burguillo, J.C., Bazzan, A.L.C.: How coalitions enhance cooperation in the IPD over complex networks. In: 2012 Third Brazilian Workshop on Social Simulation, pp. 68–74, October 2012Google Scholar
  20. 20.
    Peleteiro, A., Burguillo, J.C., Arcos, J.L., Rodriguez-Aguilar, J.A.: Fostering cooperation through dynamic coalition formation and partner switching. ACM Trans. Auton. Adapt. Syst. 9(1), 1:1–1:31 (2014)CrossRefGoogle Scholar
  21. 21.
    Huberman, B., Glance, N.: Evolutionary games and computer simulations. Proc. Natl. Acad. Sci. USA 90, 7716–7718 (1993)CrossRefGoogle Scholar
  22. 22.
    Grilo, C., Correia, L.: What makes spatial prisoner’s dilemma game sensitive to asynchronism? In: Proceedings of 11th International Conference on the Simulation and Synthesis of Living Systems, Alife XI. MIT (2008)Google Scholar
  23. 23.
    Grilo, C., Correia, L.: The influence of asynchronous dynamics in the spatial prisoner’s dilemma game. In: Asada, M., Hallam, J.C.T., Meyer, J.-A., Tani, J. (eds.) SAB 2008. LNCS (LNAI), vol. 5040, pp. 362–371. Springer, Heidelberg (2008). Scholar
  24. 24.
    Newth, D.: Asynchronous iterated prisoner’s dilemma. Adapt. Behav. 17(2), 175–183 (2009)CrossRefGoogle Scholar
  25. 25.
    Newth, D., Cornforth, D.: Asynchronous spatial evolutionary games. Biosystems 95(2), 120–129 (2009)CrossRefGoogle Scholar
  26. 26.
    Abar, S., Theodoropoulos, G.K., Lemarinier, P., O’Hare, G.M.: Agent based modelling and simulation tools: a review of the state-of-art software. Comput. Sci. Rev. 24, 13–33 (2017)CrossRefGoogle Scholar
  27. 27.
    Collier, N., North, M.: Parallel agent-based simulation with repast for high performance computing. Simulation 89(10), 1215–1235 (2013)CrossRefGoogle Scholar
  28. 28.
    Coakley, S., Gheorghe, M., Holcombe, M., Chin, S., Worth, D., Greenough, C.: Exploitation of high performance computing in the flame agent-based simulation framework. In: 2012 IEEE 14th International Conference on High Performance Computing and Communication and 2012 IEEE 9th International Conference on Embedded Software and Systems, pp. 538–545, June 2012Google Scholar
  29. 29.
    Suryanarayanan, V., Theodoropoulos, G., Lees, M.: PDES-MAS: distributed simulation of multi-agent systems. Procedia Comput. Sci. 18, 671–681 (2013)CrossRefGoogle Scholar
  30. 30.
    Wittek, P., Rubio-Campillo, X.: Scalable agent-based modelling with cloud HPC resources for social simulations. In: 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings, pp. 355–362, December 2012Google Scholar
  31. 31.
    Allen, J.: Effective Akka. O’Reilly Media, Sebastopol (2013)Google Scholar
  32. 32.
    Piccolo, E., Squillero, G.: Adaptive opponent modelling for the iterated prisoner’s dilemma. In: 2011 IEEE Congress of Evolutionary Computation (CEC), pp. 836–841, June 2011Google Scholar
  33. 33.
    Hein, O., Schwind, M., König, W.: Scale-free networks. Wirtschaftsinformatik 48(4), 267–275 (2006)CrossRefGoogle Scholar
  34. 34.
    Axelrod, R., Axelrod, R.M.: The Evolution of Cooperation, vol. 5145. Basic Books, New York (1984)zbMATHGoogle Scholar
  35. 35.
    Boyd, R., Lorberbaum, J.P.: No pure strategy is evolutionarily stable in the repeated prisoner’s dilemma game. Nature 327(6117), 58–59 (1987)CrossRefGoogle Scholar
  36. 36.
    Friedman, J.W.: A non-cooperative equilibrium for supergames. Rev. Econ. Stud. 38(1), 1–12 (1971)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Grażyna Starzec
    • 1
  • Mateusz Starzec
    • 1
  • Aleksander Byrski
    • 1
    Email author
  • Marek Kisiel-Dorohinicki
    • 1
  • Juan C. Burguillo
    • 2
  • Tom Lenaerts
    • 3
    • 4
  1. 1.Department of Computer ScienceAGH University of Science and TechnologyKrakowPoland
  2. 2.Escuela de Ingeniería de Telecomunicación, Campus Universitario Lagoas-MarcosendeUniversity of VigoVigoSpain
  3. 3.Machine Learning GroupUniversité Libre de BruxellesBrusselsBelgium
  4. 4.Artificial Intelligence LaboratoryVrije Universiteit BrusselBrusselsBelgium

Personalised recommendations