Keywords

1 Introduction

Agent-based modelling (ABM) systems (also known as multi-agent systems) have become a popular technique to study complex systems in various domains, such as biology, social sciences and business complexity. ABM can be defined as a modelling paradigm used to simulate the actions and reactions of individual entities and to measure their effects on the whole system. Many phenomena, even complex ones, can be described as systems of autonomous agents following a number of rules to communicate with each other [14].

According to Macal and North [13, 14], the structure of an agent-based model is based on three elements: (1) the number of agents, their attributes and behaviours; (2) the agents’ relationships and the mechanisms with which they interact with others; and (3) each agent’s environment, the actions and reactions of the agent with respect to its environment and other agents. By identifying and programming these elements, a model designer can easily create an ABM that simulates reality.

There are a number of popular agent-based modelling and simulation frameworks that are used to build models such as Swarm, NetLogo, Repast and MASON. The limitations of scalability and performance in these systems prevent modellers from simulating complex systems at very large scales. This is because some of these frameworks were designed to be run on a single CPU architecture and some of them cannot deal with a large number of agents within one model. For this reason, a number of platforms and simulators were implemented to deal with such systems. Repast HPC [28], D-Mason and FLAME GPU [23] are examples of these kinds of platforms that use parallel and distributed approaches to run simulations.

There have been several studies in the literature reporting computational performance in most ABM frameworks [2, 7, 11] for specific models. Varying the population size to measure system scalability is the most common benchmark. A benchmarking process is an excellent method to discover the characteristics of simulator performance, but unfortunately, so far there is no standard method to benchmark simulation tools. Thus there is a need to design a benchmark model that meets complexity and scalability standards. The OpenAB communityFootnote 1 summarised a number of criteria that may affect the performance as follows:

  • Arithmetic intensity: the computational complexity of an agent or population.

  • Scale: varying population size.

  • Model memory: the internal memory requirements of an agent or population.

  • Inter-connectivity: the level of communication between agents.

  • Homogeneity: divergence of behaviour within an agent or population.

This paper proposes a benchmark model that allows each of these criteria to be tested and we have implemented this model in FLAME GPU. The main contribution of this paper is creating a benchmark model that can be a standard to measure the execution efficiency of the existing ABM systems. This new model will be able to examine the following elements: system scalability, system homogeneity, and the ability to handle an increase in the level of agents’ communications and agents’ internal memories. The results will give insight into the performance characteristics of simulations and provide a baseline for which to measure simulator improvements.

2 Related Work

Numerous ABMs have been used to address a number of issues such as testing and analysing simulation tools and comparing ABM platforms, and they have been used as teaching tools for modelling real systems. This section reviews some of these models and their purposes.

Railsback et al. [17] proposed a simple model called StupidModel that can be easily implemented on any ABM platform. This model contains a number of versions to increase simulation complexity, starting from moving agents to a full predator-prey model. StupidModel was developed to be a teaching model for ABM platforms such as NetLogo and Swarm. It is also used as a benchmark model to compare modelling capabilities and performance between several ABM platform [11, 12, 18, 27].

Predator-prey is the most commonly used model in the field of ABM and simulation. Developed by Alfred Lotka (1925) and Vito Volterra (1926), it is based on two differential equations to describe the dynamics of predator-prey behaviour. The basic rules of predator-prey in ABM can be summarised as follows: (1) two types of populations represent prey and predator agents; (2) the prey population will increase by moving to food resources and decrease by being eaten by the predators; (3) the predator population will increase by eating the prey and will decrease by starvation; and (4) both populations are moving randomly and following simple rules to communicate with the environment and with each other.

Several studies have reported comparisons of execution efficiencies between ABM platforms using predator-prey models [7, 23]. Execution efficiencies have also been used as a benchmark to show the modelling ability of Repast Simphony [28] and by Borshchev and Filippov [3] to compare three approaches to simulation modelling: System Dynamics, Discrete Events and ABM.

The Sugarscape model is an artificial society model presented by Epstein and Axtell in their book Growing Artificial Societies: Social Science from the Bottom Up [6]. This model was replicated by several ABM platforms such as NetLogoFootnote 2, MASON [2] and Repast [24]. Agents in the basic Sugarscape model follow very simple rules. They move towards deserted areas with high levels of sugar resources. The Sugarscape Wealth Distribution model, as described by Epstein and Axtell, has more complexity in the relations between agents.

Boids is an artificial life model developed by Reynolds [20, 21] that describes the behaviour of flocking of fish or birds. According to Reynolds (2001), flocking is an example of emergence, by which the interactions of simple local rules produce a complex global behaviour. There are three simple steering behaviours that an agent in the Boids model can follow: (1) alignment, which is steering towards the average heading of nearby neighbours; (2) separation, steering to avoid crowding nearest neighbours; and (3) cohesion, steering to move toward the average position of the immediate flockmates [19]. Flocking models have been used widely to measure the modelling ability of some ABM platforms [8, 15, 18, 23].

Rousset et al. [26] used their reference model [25] to benchmark 10 existing platforms that support parallel and distributed systems. The reference model they used is based on three main behaviours for each agent: (1) agent perception, (2) agent communication and (3) agent mobility. This benchmark model is used to evaluate the ability of each platform regarding their parallelism support. A large and growing body of literature has focused on the comparison between parallel and serial execution methods to run simulation [1, 4, 5, 7, 11, 16]. All ABMs reviewed above were used as benchmarks for two purposes; to evaluate modelling capabilities of platforms and/or to make comparisons between simulators. Observing system performance and how applications behave during increases in population size is the main factor for benchmarking in most of these studies. System scalability is not the only issue that may affect the overall performance, but there are some issues that need to be dealt with to create a standard benchmark model that meets all ABM criteria.

3 The Benchmark Model

Our model is based on the concept of particle-based simulation which represents each molecule in the system as an individual entity. This entity has attributes, such as position, velocity and type of molecule. Entity movements and the reactions within the system will be computed using these attributes through methods to update system behaviour. The representation of the molecule (agent) will follow Brownian Dynamics methods, where each agent is represented as a point-like particle moving randomly in the environment.

This type of model is relevant to a wider class of ABMs. For example, both cellular models and social system models have similar behaviours, when considered from the view point of mobile agents with local interactions, birth and death and binding (combining). To make this model more complex and to meet all the criteria highlighted above, we propose a reaction-diffusion like model with different rules. Our model is able to convert formula syntax (such as A + B = C) that represents a chemical reaction to a number of mobile agents that can communicate with each other and captures important characteristics of ABM.

A simple reaction will occur when one A molecule combines with one B molecule to produce a C molecule, assuming that A + B = C represents the relationship between the three molecules. The model that resulted from the given example above contains three agents A, B, and C as follows: agent A (master agent), agent B (slave agent) and agent C (combined agent). Each of these agent specifications is defined by a set of variables and functions that help to establish the simulation. At the beginning of the simulation, agents A and B are moving randomly, and both agents are communicating with each other looking for the closest complementary agent. Agent B will send its location and then agent A will choose the closest B and replied with the ID of closest B. Once the ID of B is confirmed both agents will die and produce the new agent C.

3.1 Implementation

This section consists of three parts: (1) an overview of FLAME GPU, (2) how the benchmark model is implemented using FLAME GPU and (3) model generation. The FLAME GPU framework [22] is a template for agent-based simulation on the Graphics Processing Unit (GPU). It consists of a number of X-agents (the agent representation of an X-machine [10]) specifications. Each instance of an x-agent has its own memory that holds a set of variables. All instances of x-agents have transition functions that can read and write to their memory a start state and an end state. Agents can communicate by sending and receiving messages and their functions can read and write these messages at any time between start and end states for each agent. Creating a model using FLAME GPU is very similar to the original FLAMEFootnote 3 which required writing the model specification in XML format within an X-Machine Mark-up Language (XMML) document. However, the syntax that is used to write the model in FLAME GPU uses an extended version of the FLAME XML schema. The GPUXMML extension outlines the GPU specific model description elements such as the maximum size of an agent memory [22]. This allows a formal agent specification to be transformed to optimised C-based CUDA code through GPU-specific templates.

The FLAME GPU implementation of the above example consists of three agents A, B, and C. Each agent is defined by a set of variables, transition functions, start and end states, and communication messages as shown in Table 1. The representation of agents as a state machine is shown in Fig. 1. During a single iteration of the simulation, each type of agent will move from the starting state to the end state, completing each function in turn. The diagram is divided into three parts, each part showing the agent-transition functions and the communication dependency messages (green) for each agent.

Fig. 1.
figure 1

State graph of the model that represents A + B = C. (Color figure online)

Full size image
Table 1. Agent specifications
Full size table
Fig. 2.
figure 2

Part A: Screenshot of the first iteration showing agents A (red) and B (yellow) moving randomly. Part B: Screenshot after 100 iterations showing agents C (blue) moving randomly and two of A (red) and two of B (yellow) still moving. (Color figure online)

Full size image

At the beginning of the simulation, agents A and B are moving randomly using their move functions to update their locations during each cycle, as shown in Fig. 2 Part A. Agent B will use send_locationB to output a locationB message holding all B information (agent ID, location, etc.). Agent A after that will get all B’s locations using a need_locationB function that inputs the locationB message. This function will calculate the distance between A and B and then compare it with the binding radius. If the distance is less than or equal to the binding radius, the internal memory of A will be updated (the state variable will be set equal to 2, the defined value of binding (2 is the defined value of the combined state.), and the closest ID and the closest point will be stored). The send_bindB function will output bindB messages holding the updated information for agent A (only messages that have the state variable equal to 2 as a function condition (An agent function condition indicates that the agent function should only be applied to agents which meet the defined condition which are in the correct state specified by current State [22])). In the next step, the receive_bindB function will input bindB messages to check for the closest A that is ready to combine. B’s internal memory will be updated (the state variable will be set to 3 (3 is the defined value of the dead state.), and the closest ID and closest point will be stored) after finding the closest A that is ready to combine. The send_combinedB function will output combinedB messages that meet the condition (the state variable is equal to 2), and the B agent will be removed from the simulation. The next function will be created_C. This function will input combinedB messages (only messages that meet the condition that the state is equal to 3), output agent C, and update A’s internal memory (the state variable will be updated to meet the next function condition). All A’s that meet the condition of death_A will be removed at this stage. A visualisation of the model after a number of iterations is shown in Fig. 2 Part B.

To save time and effort, and to implement several chemical reactions at the same time automatically, a model generator is needed. This section presents a FLAME GPU model generator that can easily convert lines of formula syntax into movement models of agents. This generator after parsing the syntaxes will output three files that are required to run a FLAME GPU model: (1) a FLAME GPU XML model (XMLModelFile.xml) file that consists of model specifications, (2) a function.c file that holds the scripted agent functions, and (3) initial values of each agent for the simulation state data which is stored in a FLAME GPU XML file (0.xml).

4 Benchmarking Results

The model generator helped to vary the model in a different way and allowed modelling of different types of chemical reaction. FLAME GPU version 1.4.3 was used for the performance benchmarking on a NVIDIA GeForce GTX 970 GPU with 1665 CUDA cores and 4 GB of memory. This section shows four different benchmarks to measure FLAME GPU framework performance.

Divergence within a population: The purpose of this benchmark is to observe the system performance when doubling the number of equations. This benchmark starts with a simple model with three types of agent, ten agent functions and three type of message and ends with more than 40 agent types, 150 agent functions, and 45 message types. Adding more equation input lines (every line contains three different types of agent) increases the execution time linearly with a value of regression \(\simeq \)0.9945, as shown in Fig. 3 (axis x1 against axis y1). processing time increases by \(\simeq \)0.5 a second with the addition of a new equation. This benchmark was implemented using an agent population of 2000 for each type of agent with the same environment size, and each simulation was performed for 100 iterations.

Fig. 3.
figure 3

Processing time of the same environment size against the type of agent that have been added at every step (appears with a red line). Processing time of the same environment size against the number of slave agents that has been added every time (appears with a blue line). (Color figure online)

Full size image

Divergence within an agent: This benchmark gives us the average execution time for increasing slave agent types (more chemicals per line). This experiment will increase divergence within the master agent of this line. Adding a new chemical will extend the master agent functions, and that means more functions in each layer every cycle. In FLAME GPU function layers represent the control flow of simulation processes [9]. All agent functions are executed in a sequential order to complete one iteration and by adding more functions for the same agent that will increase execution time in every iteration. This can be observed in the results in Fig. 3 (axis x2 against axis y2). The processing time is increasing linearly by increasing chemicals per line given a value of the regression equal to 0.9956. This benchmark was implemented using an agent population of 2000 for each type of agent with the same environment size, and each simulation was run for 100 iterations.

Population sizes: The goal of this benchmark is to measure the ability of this model to scale to examine ABM systems scalability. The population size of each agent type starts with 4,096 agents and ends with 262,144 agents. This benchmark uses A + B = C as an example to run this experiment for 100 iterations each time. The performance of implementing our model on FLAME GPU with respect to agent population size is shown in Fig. 4 with linear correlation coefficient equal to 0.9811.

Fig. 4.
figure 4

Increasing population size led to increased processing time.

Full size image

Level of communication and complexity: Two changes have been made to agent behaviour to slow down the simulation and add extra arithmetic intensity within agent functions: (1) decreased interaction radius and (2) decreased agent movement speed. Figure 5 (axis x1 against axis y1) shows the relationship between decreasing the interaction radius and increasing processing time to produce 50 agents C from the A + B = C equation with same movement speed. This experiment allows agents to move for a longer time until reaching the needed radius, during this movement, several operations occur such as calculating agent position, sending and receiving messages between agents looking for the nearest agent to combine with. The next experiment is shown in Fig. 5 (axis x2 against axis y2), which shows the relationship between slowing down the agent speed the number of iterations required to produce 50 agents. This experiment has been implemented with a constant radius and same environment size. Slowing down the movement speed allows additional operations during the simulation and this help to measure the ability of the system to handle many computational operations for a long time and how to manage using the resources.

Fig. 5.
figure 5

Decreasing interaction radius led to increased time to produce 50 agents (appears with a red curve). Decreasing agent movement speed led to increased time to produce 50 agents (appears with a blue curve) (Color figure online)

Full size image

5 Conclusion

This paper presents the implementation of a new benchmark model using FLAME GPU. The aim of this model is to measure the following elements: system scalability, system homogeneity, and the ability to handle increases in the level of agent communications and model complexity. Unfortunately, measuring the ability to handle increases in the internal memory requirements of an agent or population was not covered by this paper. However, it will be involved in our future work.

Four benchmark experiments have been carried out, demonstrating the ability of this benchmark model to examine each element. The first two experiments focused on increasing agent and population divergence, and this led to increased the execution time due to the additional agent functions, messages and communication information that is held by these messages. The third experiment showed that we could easily scale the population size of this model to measure the system scalability. The results showed that scaling the population size led to varying the execution time from 0.5 s per 100 iterations for 4069 agents till 72 s per 100 for 262144 agents. In the last experiment, computational complexity was added by decreasing the value of two variables that are used within agent functions to update agents behaviour. This experiment causes the model to reach a steady state at a slower rate, this allows assessment of the system capabilities.

Divergence is known to reduce performance in GPU simulations and our benchmark model confirms this. The obtained results will be used for assessing simulator improvements to achieve improved scaling with respect to divergence and better overall performance for increasing the population size. The performance results obtained indicate that our benchmark model is a suitable model to be used as an experimental tool to evaluate modelling capabilities of an ABM system if it is replicated in a suitable way.