A Formula-Driven Scalable Benchmark Model for ABM, Applied to FLAME GPU

  • Eidah Alzahrani
  • Paul Richmond
  • Anthony J. H. Simons
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10659)


Agent Based Modelling (ABM) systems have become a popular technique for describing complex and dynamic systems. ABM is the simulation of intelligent agents and how these agents communicate with each other within the model. The growing number of agent-based applications in the simulation and AI fields led to an increase in the number of studies that focused on evaluating modelling capabilities of these applications. Observing system performance and how applications behave during increases in population size is the main factor for benchmarking in most of these studies. System scalability is not the only issue that may affect the overall performance, but there are some issues that need to be dealt with to create a standard benchmark model that meets all ABM criteria. This paper presents a new benchmark model and benchmarks the performance characteristics of the FLAME GPU simulator as an example of a parallel framework for ABM. The aim of this model is to provide parameters to easily measure the following elements: system scalability, system homogeneity, and the ability to handle increases in the level of agent communications and model complexity. Results show that FLAME GPU demonstrates near linear scalability when increasing population size and when reducing homogeneity. The benchmark also shows a negative correlation between increasing the communication complexity between agents and execution time. The results create a baseline for improving the performance of FLAME GPU and allow the simulator to be contrasted with other multi-agent simulators.


Agent based modelling Benchmarking Multi-agent systems 


  1. 1.
    Aaby, B.G., Perumalla, K.S., Seal, S.K.: Efficient simulation of agent-based models on multi-GPU and multi-core clusters. In: Proceedings of 3rd International ICST Conference on Simulation Tools and Techniques, p. 29. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering) (2010)Google Scholar
  2. 2.
    Bigbee, A., Cioffi-Revilla, C., Luke, S.: Replication of sugarscape using MASON. In: Terano, T., Kita, H., Deguchi, H., Kijima, K. (eds.) Agent-Based Approaches in Economic and Social Complex Systems IV, vol. 3, pp. 183–190. Springer, Tokyo (2007). CrossRefGoogle Scholar
  3. 3.
    Borshchev, A., Filippov, A.: From system dynamics and discrete event to practical agent based modeling: reasons, techniques, tools. In: Proceedings of 22nd International Conference of the System Dynamics Society, vol. 22 (2004)Google Scholar
  4. 4.
    Deissenberg, C., Van Der Hoog, S., Dawid, H.: EURACE: a massively parallel agent-based model of the European economy. Appl. Math. Comput. 204(2), 541–552 (2008)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Dematte, L.: Parallel particle-based reaction diffusion: a GPU implementation. In: 2010 9th International Workshop on Parallel and Distributed Methods in Verification and 2nd International Workshop on High Performance Computational Systems Biology, pp. 67–77. IEEE (2010)Google Scholar
  6. 6.
    Epstein, J.M., Axtell, R.: Growing Artificial Societies: Social Science from the Bottom Up. Brookings Institution Press, Washington, DC (1996)Google Scholar
  7. 7.
    Fachada, N., Lopes, V.V., Martins, R.C., Rosa, A.C.: Towards a standard model for research in agent-based modeling and simulation. PeerJ Comput. Sci. 1, e36 (2015)CrossRefGoogle Scholar
  8. 8.
    Goldsby, M.E., Pancerella, C.M.: Multithreaded agent-based simulation. In: Proceedings of 2013 Winter Simulation Conference: Simulation: Making Decisions in a Complex World, pp. 1581–1591. IEEE Press (2013)Google Scholar
  9. 9.
    Grimm, V., Railsback, S.F.: Individual-Based Modeling and Ecology. Princeton Series in Theoretical and Computational Biology. Princeton University Press, Princeton (2005)CrossRefzbMATHGoogle Scholar
  10. 10.
    Ipate, F., Holcombe, M.: A method for refining and testing generalised machine specifications. Int. J. Comput. Math. 68(3-4), 197–219 (1998)Google Scholar
  11. 11.
    Lysenko, M., D’Souza, R.M., et al.: A framework for megascale agent based model simulations on graphics processing units. J. Artif. Soc. Soc. Simul. 11(4), 10 (2008)Google Scholar
  12. 12.
    Lytinen, S.L., Railsback, S.F.: The evolution of agent-based simulation platforms: a review of NetLogo 5.0 and ReLogo. In: Proceedings of 4th International Symposium on Agent-Based Modeling and Simulation, p. 19 (2012)Google Scholar
  13. 13.
    Macal, C.M., North, M.J.: Agent-based modeling and simulation. In: Winter Simulation Conference, pp. 86–98 (2009)Google Scholar
  14. 14.
    Macal, C.M., North, M.J.: Tutorial on agent-based modelling and simulation. J. Simul. 4(3), 151–162 (2010)CrossRefGoogle Scholar
  15. 15.
    North, M.J., Tatara, E., Collier, N.T., Ozik, J., et al.: Visual agent-based model development with repast simphony. Technical report, Argonne National Laboratory (2007)Google Scholar
  16. 16.
    de Paiva Oliveira, A., Richmond, P.: Feasibility study of multi-agent simulation at the cellular level with FLAME GPU. In: FLAIRS Conference, pp. 398–403 (2016)Google Scholar
  17. 17.
    Railsback, S., Lytinen, S., Grimm, V.: StupidModel and extensions: a template and teaching tool for agent-based modeling platforms. Swarm Development Group (2005).
  18. 18.
    Railsback, S.F., Lytinen, S.L., Jackson, S.K.: Agent-based simulation platforms: review and development recommendations. Simulation 82(9), 609–623 (2006)CrossRefGoogle Scholar
  19. 19.
    Reynolds, C.W.: Boids: background and update (2001).
  20. 20.
    Reynolds, C.W.: Big fast crowds on PS3. In: Proceedings of 2006 ACM SIGGRAPH Symposium on Videogames, pp. 113–121. ACM (2006)Google Scholar
  21. 21.
    Reynolds, C.W.: Flocks, herds and schools: a distributed behavioral model. ACM SIGGRAPH Comput. Graph. 21(4), 25–34 (1987)CrossRefGoogle Scholar
  22. 22.
    Richmond, P.: Flame GPU technical report and user guide (CS-11-03). Technical report, Department of Computer Science, University of Sheffield (2011)Google Scholar
  23. 23.
    Richmond, P., Romano, D.: Template-Driven Agent-based Modeling and Simulation with CUDA. Applications of GPU Computing Series, GPU Computing Gems Emerald Edition, pp. 313–324 (2011)Google Scholar
  24. 24.
    Robertson, D.A.: Agent-based models to manage the complex. Manag. Org. Complex.: Philos. Theory Appl. 24, 417–430 (2005)Google Scholar
  25. 25.
    Rousset, A., Herrmann, B., Lang, C., Philippe, L.: A survey on parallel and distributed multi-agent systems. In: Lopes, L., et al. (eds.) Euro-Par 2014. LNCS, vol. 8805, pp. 371–382. Springer, Cham (2014). Google Scholar
  26. 26.
    Rousset, A., Herrmann, B., Lang, C., Philippe, L.: A survey on parallel and distributed multi-agent systems for high performance computing simulations. Comput. Sci. Rev. 22, 27–46 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Standish, R.K.: Going stupid with EcoLab. Simulation 84(12), 611–618 (2008)CrossRefGoogle Scholar
  28. 28.
    Tatara, E., North, M., Howe, T., Collier, N., Vos, J., et al.: An introduction to repast simphony modeling using a simple predator-prey example. In: Proceedings of Agent 2006 Conference on Social Agents: Results and Prospects (2006)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Eidah Alzahrani
    • 1
    • 2
  • Paul Richmond
    • 1
  • Anthony J. H. Simons
    • 1
  1. 1.Department of Computer ScienceThe University of SheffieldSheffieldUK
  2. 2.Al Baha UniversityAl BahahSaudi Arabia

Personalised recommendations