Advertisement

GeoInformatica

, Volume 23, Issue 2, pp 169–199 | Cite as

Crossing the chasm: a ‘tube-map’ for agent-based social simulation of policy scenarios in spatially-distributed systems

  • J. Gareth PolhillEmail author
  • Jiaqi Ge
  • Matthew P. Hare
  • Keith B. Matthews
  • Alessandro Gimona
  • Douglas Salt
  • Jagadeesh Yeluripati
Open Access
Article

Abstract

Agent based models (ABMs) simulate actions and interactions of autonomous agents/groups and their effect on systems as a whole, accounting for learning without assuming perfect rationality or complete knowledge. ABMs are an increasingly popular approach to studying complex, spatially distributed socio-environmental systems, but have still to become an established approach in the sense of being one that is expected by those wanting to explore scenarios in such systems. Partly, this is an issue of awareness – ABM is still new enough that many people have not heard of it; partly, it is an issue of confidence – ABM has more to do to prove itself if it is to become a preferred method. This paper will identify advances in the craft and deployment of ABM needed if ABM is to become an accepted part of mainstream science for policy or stakeholders. The conduct of ABM has, over the last decade, seen a transition from using abstracted representations of systems (supporting theory-led thought experiments) to more accessible representations derived empirically (to deliver more applied analysis). This has enhanced the perception of potential users of ABM outputs that the latter are salient and credible. Empirical ABM is not, however, a panacea, as it demands more computing and data resources, limiting applications to domains where data exist along with suitable environmental models where these are required. Further, empirical ABM is still facing serious questions of validation and the ontology used to describe the system in the first place. Using Geoffrey A. Moore’s Crossing the Chasm as a lens, we argue that the way ahead for ABM lies in identifying the niches in which it can best demonstrate its advantages, working with collaborators to demonstrate that it can deliver on its promises. This leads us to identify several areas where work is needed.

Keywords

Agent-based modelling Policy analysis Innovation diffusion Chasm 

1 Introduction

The case for agent-based modelling (ABM) has been made by several authors. ABMs can simulate heterogeneous interacting individuals making decisions about their behaviour. It can be spatially explicit and simulate social interactions under biophysical constraints. It can capture complex dynamics that are infeasible, inelegant, or oversimplified when addressed in other modelling approaches. It can explore formalizations of theories, and is less prone to hiding assumptions, making it easier to use with stakeholders. It has been applied to a wide range of case studies from water use to traffic simulation, and can be applied to modelling proteins as agents to modelling interactions among nation states [1]. With ABMs dating back at least to the 1960s (such as Schelling and Sakoda both modelling spatial dynamics of homophily; [2]), it can hardly be said to be especially new. With all these advantages, it is strange that ABM is not routinely taught in undergraduate courses (particularly in geography, psychology, sociology, economics, politics); and policymakers, businesses, banks and research funders are not typically expecting analyses to involve ABM.

One difficulty with ABM is defining what it is authoritatively. Controversies over whether the entities in a model qualify as ‘agents’, for example, have a long history. Writing in 1999, Gilbert and Troitzsch [3] observe (p. 159) that “there is no generally agreed definition of what an ‘agent’ is.” Some of the controversy has its origins in confusing representation with implementation, with software agents [4] sometimes being used to implement ABMs. There is intersection, but not equivalence, between ABM and software agents. To us, ABM is primarily a concern with representation, and though some environments and languages are more convenient than others, it matters little how that is implemented in a computer program.

It is not the place of this paper to authoritatively answer a question that to some extent derives from the cross-disciplinary nature of ABM and hence is unlikely to be resolved to the satisfaction of everyone. However, for our purposes, ABMs must at least explicitly represent the following:
  • a ‘sufficient’ number of entities (sorites paradox notwithstanding) individually;

  • each such entity having some attributes that are, in some sense, ‘theirs’ and not others’;

  • each such entity also having some dynamics that they are, in some sense, responsible for causing; with

  • these dynamics having the potential to cause (directly or indirectly) changes to the attributes belonging to other such entities.

This paper uses Moore’s [5] Crossing the Chasm book in the marketing literature as a lens through which to examine the adoption of ABM. Moore’s book concentrates on the adoption of disruptive technology, and takes the technology adoption lifecycle from Rogers’s [6] Diffusion of Innovations as its starting point. In the technology adoption lifecycle, the population is divided according to five psychodemographic profiles, labelled in order of adoption: innovators (estimated to be 2.5% of the population), early adopters (13.5%), early majority (34%), late majority (34%) and laggards (16%). These are usually depicted using a normal curve, where the ‘mean’ sits between the early and late majorities.

Moore’s insight was that there is a qualitative difference between the challenge of appealing to innovators and early adopters, and that of engaging the early and late majorities. Moore describes the gaps between innovators and early adopters, and between early and late majorities, as ‘cracks’ that are relatively easy (but not trivial) to cross. To bridge the crack between innovators and early adopters, early adopters need a compelling use-case, rather than just trying something new for the sake of it, which is what innovators find exciting. To move from the early to late majority, the late majority just need the innovation already adopted by the early majority to be as easy as possible for them to use.

More challenging than the ‘cracks’, to Moore there is a ‘chasm’ (see Fig. 1) between the early adopters and the early majority, which must be crossed if an innovation is ever to be widely adopted. Appreciating that the chasm is more difficult to cross than the cracks requires understanding the different psychological profiles of the two groups. Moore characterizes the psychological profiles of innovators and early adopters as ‘visionaries’ who make horizontal connections with people outside their community and specialism. By contrast, the early majority are ‘pragmatists’ who are interested in what works, and primarily make vertical connections within their community. The chasm exists because the qualities of an innovation that appeal to visionaries (radicalness, transformational) are a turn-off to pragmatists, and vice versa. Further, pragmatists give higher weight to the opinions of other pragmatists in their peer group when evaluating an innovation than they do to opinions of visionaries. This means that assumptions that uptake by early adopters will automatically and linearly lead to uptake by the early majority are misplaced. Such assumptions are all-too-easy to make for communities, such as some academics, who might have the impression that all that is needed for their obviously brilliant innovation to be adopted is to amass a sufficient body of evidence that it is so.
Fig. 1

Graphical summary of the steps needed to cross the chasm in ABM in the style of the London Underground (a ‘tube-map’). The technology adoption lifecycle from Rogers’s Diffusion of Innovations is depicted using the grey segments of a normal curve. Moore’s [5] ‘cracks’ between innovators and early adopters, between early and late majority, and between late majority and laggards, are depicted as non-interchange stops, labelled accordingly. The ‘chasm’ is depicted as a sequence of four interchange stops each of which corresponds to a step in Moore’s strategy for crossing the chasm. The interchange lines are coloured according to themes that link with other areas of research. ‘I2S’ is short for the Implementation and Integration Sciences [7]

Primarily aimed at a sales audience, Moore’s [5] book adopts a somewhat aggressive tone towards the strategy for crossing the chasm, likening it to the Allies’ invasion of Normandy in the final year of World War II. Such language may not be appropriate in the more refined arena of academia, but it would be a mistake to assume that the advantages of ABM that are so clear to those who use it will be automatically observed by others; or to expect that more established approaches and entrenched epistemological positions will be dislodged without a struggle. However, in less combative terminology, the strategy comprises the following steps for ABM:
  1. 1.

    Identify niches where ABM most clearly solves problems better than other approaches. For Moore [5], this is about segmenting customers and analysing competitors in each segment. More relevantly to ABM, he also characterizes it as ‘high-risk, low-data’ situation, because the market identified is unknown. This speaks clearly to classic interdisciplinary situations: as an expert in building and using ABM, you will not have as deep a knowledge of a particular empirical application domain as someone who has spent their working life studying it.

     
  2. 2.

    Build alliances with interested parties in various disciplines and application domains. Moore [5] articulates this around Levitt’s [8] ‘Whole Product’ model, which, for us, involves thinking about how ABM will fit in to the identified niche, and who else we need to work with to make that happen. There is a more general point here, particularly for the academic community where working as an individual can have higher perceived status than collaborating in teams: applying ABM to policy scenarios in empirical contexts is probably not an undertaking best conducted alone.

     
  3. 3.

    Define the agenda; be clear in each context why ABM is the right choice for that context. Moore [5] characterizes this as ‘defining the competition’. For ABM, this entails developing clarity about where it is strong, and shifting the discourse on formal approaches to modelling systems so that it encompasses territories where ABM has strengths.

     
  4. 4.

    Deliver on what ABM promises, and make sure it is as accessible as possible. In the world of marketing, this is a question of distribution and pricing [5]. More generally, these are both issues of accessibility, about which domain specialists curious about ABM have complained. Since many high-tech products come ‘out of the box’ once delivered, there is an additional element to work with ABM for it to be more widely adopted, which is that it is perceived to have delivered on what was promised in any project in which it is applied.

     

Agent-based modelling has certainly had its visionaries. Many of the more stylized models of the 1990s were concerned with explaining complex emergent socio-political phenomena using interacting agents with ‘simple’ heterogeneous behavioural rules. Though this led to some fascinating work, there was occasionally a tendency to overstate the interpretation of observations in the model for societies in the real world – sometimes based on quite arbitrary implementation details or utilization practice with the model [9, 10]. However, it is worth noting that ABM is not the only modelling discipline to have overreached itself from time to time: game theory, neoclassical economics, cellular automata, system dynamics – all have examples of work doing the equivalent of reading tea leaves. It is precisely this sort of hyperbole that can put off the pragmatists comprising the early majority.

Reviews and introductory papers on ABM in various journals (e.g. [11, 12, 13]) chart the beginnings of the journey across the chasm to appeal to established disciplines. As part of building that appeal, work by authors such as Janssen and Ostrom [14] and Smajgl et al. [15] shifted the narrative towards empirical applications of ABM. There were also calls to address the seeming arbitrariness with which ABMs were put together and to link to established theories [16, 17]. Various researchers applied ABM in policy contexts (e.g. [18, 19, 20]), or alongside role-playing games in contexts of conflict over environmental resources [21, 22].

There is a way to go yet before the kind of acceptance among the early adopters is needed to achieve the level of adoption of ABM outlined earlier, suggesting ABM is facing, or will shortly face, the chasm. The strategy outlined by Moore implies, for ABM, that we need to demonstrate that it works in each of the niche areas of application to which it is to be applied, taking control of the agenda with the aid of key allies in various disciplines and application areas, and making ABM as accessible and easy to use as we can. At the same time, with growing interest in Integration and Implementation Sciences (I2S; [7]), and interdisciplinary and transdisciplinary research more generally (especially in policy arenas), ABM is well-placed to make a significant, practical contribution.

The rest of this paper will consider the developments needed in ABM to complete the bridge using Moore’s strategy for crossing the chasm as a lens. The structure is outlined in Fig. 1.

2 Niches

Identifying possible niches for ABM to occupy entails (a) estimating the current stage of adoption ABM has attained in a given niche; (b) understanding entry barriers to the niche; and (c) being clear about how ABM might address the niche’s issues in ways other modelling paradigms cannot. We now consider each of these in turn.

2.1 Assessing adoption

A rough impression of levels of adoption of ABM in specialist areas of research can be gleaned using bibliographic databases to compare numbers of articles mentioning ABM as a proportion of articles mentioning modelling or simulation generally. We used Clarivate Analytics’s Web of Science Core Collection (WSCC) and Journal Citation Reports (JCR) to rank journals in Web of Science Categories corresponding to research areas in Agriculture, Economics, Sociology, Urban Systems and Water Management, as well as Multidisciplinary research. These research areas were chosen subjectively for their potential in having a significant human decision-making component, with micro-macro interactions between individuals and the whole system, and a possible need to integrate the social and the environmental.

The thinking behind this approach was that in specific research areas, leading researchers would be publishing in higher-ranking journals for those areas. Based on the theory of Moore [5] about the characteristics of the late majority, domain-driven researchers would be more likely to talk with each other than with researchers in other disciplines. The level of presence of ABM among modelling and simulation papers in each of the research areas would be one way to examine the level of adoption. The results are summarized in Table 1, and show that levels of adoption are less than 1% in each area. More details on the method used are in the appendix.
Table 1

Results in each research area, after searching Clarivate Analytics’s Web of Science Core Collection for proportions of articles in the period 2008–2017 inclusive with ‘model’ or ‘simulation’ in the topic that also have ‘agent-based’, ‘agent based’, ‘multi-agent’ or ‘multi agent’ in the topic. See the appendix for more details

 

Agriculture

Economics

Sociology

Urban systems

Water management

Interdisciplinary social sciences

Multidisciplinary

Model

13,894

74,694

8687

4628

55,564

11,954

108,839

ABM

53

575

74

30

89

104

527

% ABM

0.38

0.77

0.85

0.65

0.16

0.87

0.48

This rapid appraisal of the level of adoption is open to a number of criticisms. Meanings of ‘model’ and ‘simulation’ in different disciplines can be diverse, and phrases such as ‘role model’ might appear in an abstract without any relevance to modelling or simulation. It may be that some articles were inappropriately included. On the other hand, we did not include some variants of these terms, such as ‘simulating’, which would have increased the volume of articles captured. Similarly, the search terms used to suggest the presence of an agent-based model ((‘model’ or ‘simulation’) and (‘agent-based’, ‘agent based’, ‘multi-agent’ or ‘multi agent’) are debatable, even before the complicated question of what, precisely, constitutes an agent-based model, is raised. For us to reach the conclusion that ABM had crossed the chasm in any of these areas, however, we would have needed the method yielding a more accurate estimate of the level of adoption to produce results suggesting between one and two orders of magnitude increase in the numbers of ABM papers returned over the figures in Table 1.

The levels of adoption suggest ABM is chiefly used by innovators in empirical application domains, but the margin of error from this exercise is potentially big enough that some research areas may have crossed the crack from innovators to early adopters. For example, Squazzoni and Casnici [23] show less citation of ABM work by the social sciences than vice versa, suggesting it is innovators in the social sciences that are using ABM. By contrast, O’Sullivan et al. [24], whose article is entitled, “Yet another agent-based model, whatever, never mind,” suggest a sufficient volume of work in the Land Use and Cover Change (LUCC) area1 that we might conclude that in this area, early adopters are using ABM. More generally, bibliometric studies, such as that of Squazzoni and Casnici [23], particularly where they included some form of network analysis, would be a useful way to assess the level of engagement with ABM in application domains, and where the niches are.

2.2 Timely provision of clear messages

The assumption that there is an information deficit that policy-makers want filled through the adoption of new modelling tools has proven false [25]. Policy-makers have many, varied sources of information from which to support decision-making, including expert advice, databases, reports from consultancies, spreadsheets, and existing in-house modelling tools. Information provided by new models can be seen as ranking low on the importance hierarchy of information sources to be taken into account [26]. Established, trusted sources of information (including individual experts) are also hard to supersede, especially given the high levels of personal liability that policy-makers typically have with respect to their decisions. The simplicity and clarity of message can be hard to establish if the outputs of ABM simulations are themselves complex, uncertain, or point to path dependencies and the need to differentiate policies in space or across sectors. Developing new visualization techniques to help summarize results succinctly is crucial for both salience and credibility. In policy circles, complexity is in any case a hard sell at best, if not an outright deterrent, especially if it raises issues of model output uncertainty (ibid.). Indeed, the academic community are not always especially keen on complexity either [27]. However, managing risks entails being clear about uncertainty and other limits of what is known, rather than ignoring it for hypocognitive [28] reasons that are often clothed in pragmatism.

To be able to effectively feed in to a policy process, there is also a need to achieve salience at particular points in time. In fluid, uncertain, political cycles, there will be sequential phases of interest and disinterest in what is being offered. There is thus a requirement for significant flexibility, individually and institutionally, to be in a position to deliver the results at the most propitious moment. Even with such foresight or adaptability, the impact of individual initiatives can depend as much on serendipity as it does on planning. A strategy for breaking in to a niche with high entry barriers could be to have multiple small-scale projects in play, each with plans for rapid reinforcement and exploitation of success when called upon, rather than putting effort in to monolithic flagship projects.

2.3 Unintended consequences

‘Unintended consequences’ of a policy intervention can be described in terms of its “knock-on”, “multiplier” and “unanticipated” effects ([29], p. 59). HM Treasury (ibid.) give examples of four kinds of such effect: ‘displacement’ – the benefits of an intervention are offset by negative outcomes elsewhere; ‘substitution’ – the outcomes are not Pareto-efficient; ‘leakage’ – benefits are experienced by those outside the intended group; and ‘deadweight’ – outcomes supported by the intervention would have happened without it. To these, we may add ‘gaming the system’, where actors intentionally take advantage of loopholes in an intervention to obtain material gain with relatively little effort. Through connecting individual decisions with systemic effects, we anticipate that ABM has a potential niche in the evaluation of unintended consequences.

There are several reported advantages of ABM in the literature over other paradigms (e.g. [30, 31]), typically argued on the basis of additional systemic features known to exist in the real world that are challenging for other approaches to address. A key characteristic of ABM enabling it to address these features is the high level of ‘expressivity’ of the formal languages used to implement ABMs in comparison to other modelling paradigms [32]. This expressivity allows more features to be modelled and simulated that would otherwise have to be assumed, aggregated or abstracted away for reasons of tractability. Complex systems can be modelled in greater degrees of complexity.

In terms of niche establishment for ABM, mere appeal to complexity is insufficient to disrupt established practice. Crossing the chasm within a niche with considerable entry barriers requires offering an exceptional service that includes previously unavailable features that are novel and important, but critically, also salient. If we consider the field of LUCC, it is clear that many of the problems of land use change are not amenable to tractable analysis with traditional methods, not least because of issues with explicit representation of space. More significantly, for some calculus-based approaches, the notion of change central to the concept of LUCC itself suggests a system that is not at equilibrium. Further, the significant time delays between activity and economic return for most land-based activities (and especially farming, forestry, housing and infrastructure development), and intervening uncertainties, render optimization with respect to profit infeasible for most decision-makers. There is thus a greater emphasis on individual, societal and environmental context in the factors influencing land-based decision-making. In addition to the heterogeneity of decision-makers and their contexts is the fundamental issue of spatial connectivity. Even if independent, norm-free reasoning processes are assumed, spatial connectivity creates a physical basis for the effects of one land use decision to influence future decisions of a neighbour [33, 34]. Understanding LUCC also has potentially profound implications for the governance and practice of landscape management, as it influences landscape dynamics and configuration [35], which are crucial to meeting international biodiversity obligations under the United Nations’ Convention on Biodiversity.

For LUCC modelling, the features that ABM offers include:
  • interactions among humans and with the biophysical environment in a spatially explicit environment;

  • heterogeneous decision-making of individual humans based on their individual attitudes and memories;

  • environmental and social (normative) contextual aspects of decision-making;

  • endogenous evolution of systemic change (away from equilibria);

  • greater coverage of potential system state space (especially through multiple simulation runs), including outcomes that might be low-probability, path-dependent and/or emergent.

Though we, as academics, might be persuaded that features such as these are important enough to include in models, if established practice has been to ignore them, developing the niche for ABM will need to be articulated around the perils of ignoring the features as opposed to assembling more and more evidence that they exist. It is generally recognized that policies can have unintended consequences, and there is a growing body of examples of well-intentioned policies have not had the expected effect (e.g. [36]). Hence, complexity and systemic features that do need to be modelled are better articulated in terms of unintended consequences in policy evaluation contexts.

3 Allies

The social networks of the innovators and early adopters are, as stated earlier, qualitatively different to those of the early and late majorities. The latter are much more likely to connect and listen to people within their own community. Collaborations will be needed to enable ABM to contribute to identified niches.

3.1 Inter- and trans-disciplinary integration

Appealing to target discipline and application domain practitioners requires joining forces with key individuals therein who are already receptive to ABM. There has been recognition in the social sciences of the relevance of complex systems theory (e.g. [37]), but, as noted earlier, in policy circles complexity is not an attractive attribute. However, a more fruitful line of argument to canvass allies might be that of integration.

ABM is an excellent integrator in collaborative interdisciplinary projects, providing a platform for the explicit modelling of concepts from the social, physical, economic and ecological sciences. Increasing recognition that integration is itself a specialist area [7], along with growing demand for inter- and trans-disciplinary teams to tackle so-called ‘wicked’ problems [38, 39], also means that skills in ABM are potentially sought after when putting together consortia.

The integrative benefit of ABM is, however, a double-edged sword. The risk of only ever acting as an integrator in empirical contexts is the lack of opportunity to develop methodology and theory in a strategic way, rather than in a more ad hoc manner as needed in the projects in which ABM practitioners happen to be involved. The Catch-22 is clear, and indeed a more general issue for any novel approach in any area of research: funding bodies are unlikely to part with limited resources purely to develop methodology and theory for ABM until it has become established.

Assembling consortia of people who want to help advance the cause of ABM (whether intentionally or not) requires different approaches in policy-relevant circles, in the environmental/natural sciences, and in the (other) social sciences. For the social sciences, for example, potential colleagues will want to see links with the theories and methods that they are familiar with, and evidence that ABMs are not making naïve assumptions about humans and societies that have long been discredited [16]. For the environmental and natural sciences, where formal modelling is more universally accepted, the linkages required pertain to how to meaningfully embed their models in a social simulation; a matter that is non-trivial semantically. For policy-relevance, it is important to show how ABM can operate successfully within established policy evaluation frameworks, such as, in the UK, The Magenta Book [29], which provides guidance to policy analysts in the civil service on methods and good practice for evaluating government policies, projects and programmes.2

Since many practitioners in ABM have a background in the social sciences, and/or regard the social sciences as the main audience for their work, the social sciences are important as a potential source of allies to support crossing the chasm. Squazzoni [40] gives examples of the various ways in which ABM has benefitted social sciences from economics to anthropology, particularly around the refinement of theory and facilitating transdisciplinary research. Recent formalizations of social theories in ABMs include Shove and Pantzar’s [41] Practice Theory [42], Ingelhart and Welzel’s [43] Human Development Sequence Theory [44], and Lindenberg and Steg’s [45] Goal-Framing Theory [46]. Formalization of theory is not unproblematic, however – social theories are not software requirements specifications. Poile and Safayeni [47] observe that equifinality and ‘reasonable technical assumptions’ about how to implement social theories can obscure potential insights gained from so doing.

3.2 Hybridization

The quantification of environmental impacts inevitably requires hybridization of ABM with modelling approaches used in the natural sciences. Filatova et al. [48] conclude that hybridizing ABM with other modelling approaches holds the best prospects for modelling systemic change, something that much of the policy around the most urgent global problems is trying to achieve. Connecting models together (‘coupling’) is, however, not trivial [49]. Issues with semantic heterogeneity when integrating databases also apply to integrating models. Pluempitiwiriyawej and Hammer [50] categorize these into structural (e.g. is something a class or a relationship), domain and data conflicts. Since models not only contain data, but simulate processes, there are additional semantic heterogeneity issues with model integration. Specifically, ‘algorithmic’ conflicts can occur [51] where two submodels each embed the same process in a different way – a matter that is not at all clear if models only exchange input and output variables. Using (computing science) ontologies (e.g. OWL; [52, 53]) to manage integration allows at least some of these issues to be addressed [54, 55] but such approaches are not widely used, nor at a sufficient state of maturity that they reasonably can be. Undertaking such work would be helpful; minimally, however, awareness of semantic heterogeneity and openness about the structure and functioning of (sub) models that are to be coupled together, is vital if issues raised by Voinov and Shugart [49] are to be avoided [56].

3.3 Tackling prediction

Squazzoni [40] notes in his conclusion (pp. 220–221) that for ABM to advance, it needs to develop the means to have impact on policy. He nevertheless includes a number of examples of uses of ABM in policy contexts, and indeed there are special issues on the topic (e.g. that introduced by [57]). Hamill [58], who came to ABM from a policy advice background, observes that for it to be adopted, there needs to be a clear benefit of ABM over existing approaches, in terms of the output from the model (rather than its features and mechanisms): the quantitative predictions made by whatever method is used about the impacts and costs of different policy options – “Option A will do this and cost £Xm,” as she puts it (ibid., para. 4.1). However, Schulze et al. [59], reviewing ABM in socio-ecological systems, state that there is “still a lack of predictive power of ABMs,” whilst Ahrweiler [60] refers to “weak prediction”. Aodha and Edmonds [61] are more forthright, arguing that prediction should not even be expected in policy contexts.

It is natural for a community well aware of the complexities and path dependencies of societal systems to be cautious about prediction [60]. “Prediction” is a contentious word that some only use in contexts of certainty and precision when making statements about the future. This may be one reason for the seeming caution, and no such meaning is intended here: predictions can be uncertain and imprecise. A lack of willingness to engage with prediction will not advance the cause of ABM, and represents lost opportunities for insights to be gained from the attempt, including influencing understanding of the limits to prediction [62]. It also leaves the door open for overconfident predictions based on oversimplified models to continue to be used to justify policies, with potentially harmful side-effects on individuals, society as a whole, and the environment. No reasonable person expects a crystal ball when models make predictions – we are scientists, not prophets and fortune-tellers. Rather, it is a case of presenting the prediction in ways that capture uncertainties, especially around the ‘reasonable technical assumptions’ discussed above. Making several runs, exploring implementation variants and parameter spaces, together with phase space analysis (see [63]), perhaps ABM could one day make predictions such as: “Option A will achieve goal G under scenario S P% of the time at a cost of £X±E, as long as fewer than Y people do Z.”

That said, it is worth noting Edmonds’s [64] warnings that policymakers make unrealistic demands of modellers, and are impatient with caveats. Arguably, such situations suggest dysfunctional relationships between scientists and policymakers. Gilbert et al. [65] drawing on their experience in computational policy modelling, stress the importance of the iterative engagement and strong mutual understanding between scientists and policymakers. Even then, they note (para. 2.5) that predicting specific values of variables at specific future times is typically not possible; more realistic are qualitative predictions about whether an event will occur, the direction of change of variables, or the possibility that unexpected outcomes could emerge. Matthews et al. [66] describes the use of Quantitative Storytelling [67] in the context of a project using system dynamics to inform policy around food-water-energy-waste-biodiversity nexus issues. Such may be a more appropriate way in which to couch prediction using ABMs.

4 Defining the agenda

Psychologically, the early majority (our target at the other side of the chasm) prefer evolution to revolution [5]. Arguing the revolutionary nature of Agent-Based Modelling is therefore potentially off-putting. There are nevertheless capabilities introduced by ABM that mean traditional ways of assessing models are not entirely appropriate.

4.1 Validation

Traditional methods for developing confidence in models and their predictions (e.g. out-of-sample validation) do not do justice to ABMs, meaning that they are not competing on a level playing field when statistics such as L2 norm or Akaike Information Criterion are the sole metrics by which such confidence is established. There is a clear, qualitative difference between adjusting parameters of a mathematical function until it fits some data, and replicating those data as emergent phenomena of the interactions among heterogeneous agents. The latter is arguably much harder to do, and has more meaningfully captured the underlying dynamics. None of this is captured in the sum-of-squared error and similar fit-to-data metrics, and research is needed to create new metrics that do address structural matters. Whether revolutionary or evolutionary, the advent of ABM raises important questions about how models are assessed.

Polhill and Salt [32] attempt to articulate the difference drawing on the concept of expressiveness [68, 69] of the formal logics used to describe the state of a model. Essentially, ABMs have richer ontologies to describe the states of a system than does, for example, a high-order polynomial. Since the question of ontology does not really arise in more traditional modelling approaches, methods for assessing their performance have naturally focused on fit-to-data, and penalising models for unnecessary parameters.

The important point, if ABM is to define the agenda, is therefore to develop the means to argue more formally what benefits it brings to the exploration of policy scenarios in spatially distributed systems, beyond lists of system features it implements. Ideally, it would then be possible to demonstrate that, under such a broader set of criteria by which confidence in a model is to be assessed, ABM performs just as well or better. Such an ideal set of circumstances is not necessarily always realistic, but by being clearer about the multiplicity of ways in which models are judged, those evaluating models are then at least in a position of having to trade off criteria, and modellers are in less of a position of being able to play to a single established metric. It also avoids circumstances in which assessing models is reduced to a rather trivial debate about which fit-to-data metric should be used to establish the supremacy of one model over another (see [70]).

4.2 Representation

ABM needs to move beyond mere appeal to complexity as a basis for trusting that ABM captures real-world features that other models cannot. We are still not in a position where complex systems are understood sufficiently that there are standardized tests and metrics to demonstrate the statistical signatures of complexity in observed data, and whether a model exhibits behaviours that adequately reproduce those signatures. There are other features of real-world systems that have been emphasized in articulating the benefits of ABM, most notably the influences of space and networks in mediating dynamics, and more realistic representations of decision-making. Here, it is less an issue of such features being poorly understood, than it is the questions of whether they matter, and if they do matter, how they ‘should’ be represented.

Similarly, in the case of space and networks, there has been work showing that they make a difference (e.g. [71]). Most ABMs, where they represent space, use rasterized representations. There has been increasing use of Geographic Information Systems (GIS) in ABMs, providing easier access to vector representations, especially with the embedding of GIS tools in popularly-used platforms such as Repast [72] and Netlogo [73]. Besides rasters and vectors, there are also qualitative spatial representations, such as the Region Connection Calculus [74], which have only very rarely been applied in ABM contexts.

Where social networks are fixed rather than evolving, established artificial algorithms such as Barabasi-Albert [75] or Watts-Strogatz [76] are commonly used, though these can have artefacts [77]. A further issue for both spatial and network representation is that alternative approaches to ABM can argue they include them (e.g. [78, 79]), meaning that, in the absence of suitable metrics reflecting how the representation is achieved, arguments about whether ABM or more established methods are more faithfully representing systems to which they are applied become mired in technical detail. One way around this returns to the diversification of quantitative metrics discussed above. For both space and, to a lesser extent, networks, there are methods for making comparisons between models and the empirical world, and these should be used more when empirically evaluating ABMs – even if they are not central to the objective of the model, they are relevant in reflecting the model’s representation of the underlying system.

Representing the individual explicitly also places an emphasis on means to initialize the attributes of a population of agents. Where this cannot be achieved directly from data, it will be necessary to synthesize these populations such that they have characteristics matching those data that are available. Though there are methods in microsimulation (such as [80]) that ABMs can and should draw on, research is needed to bring these together with methods for initializing social networks and locations, or at least to study the extent to which systemic outcomes are sensitive to the principles underlying the initialization of agents’ networks, locations and attributes.

4.3 Decision-making

The ability of ABM to simulate decision-making by agents that have limited computational and information resources is one of its most commonly mooted advantages. Such capability does not preclude the possibility of representing decision-making using traditional utility maximization approaches; but it is useful that the latter are not necessary in order for ABM to work. The project of relaxing assumptions made in traditional domains to explore the effect they have on macro-level dynamics is on-going, both in ABM and heterodox economics [81]. There are also examples where non-optimizing decision-making by agents has been shown to make an important difference to the macro level dynamics (e.g. [82]).

The question of how to represent decision-making is almost a perennial topic of discussion among those interested in ABM, and indeed boundedly and non-rational decision-making more generally (e.g. [83, 84]), but Groeneveld et al. [85] found most researchers in ABM of LUCC are still using utility maximization, whilst Huber et al. [86] note a lack of consideration of values, norms and social interactions in ABMs of European agriculture. There are a plethora of theories and algorithms for implementing decision-making in specific and general contexts. Part of the issue is in increasing awareness of them, and access to implementations – especially for those with strong disciplinary backgrounds who are unlikely to have been exposed to ideas outwith their narrow specialism. However, it is also important to establish and build collaborations with people in the AI community, since they provide a body of expertise in the formalization of behaviour. Jager [87], reviewing implementations of psychological theories in ABMs, observes the potential of drawing on cognitive modelling despite its tendency to focus on a single individual, citing Sun’s [88] recommendation of enhancing collaboration between cognitive models and social simulations.

If, as part of exploiting a potential niche for ABM in evaluating unintended consequences, we are to tackle the exploration of agents’ abilities to ‘game the system’, decision-making algorithms will need to be sophisticated and adaptive. Orr et al. [89] posit that ABMs of social systems that fail to account for cognition will not generalize to new situations. Balke and Gilbert [90] review approaches to implementing richer cognitive architectures in ABMs. Various researchers have also used machine learning algorithms such as reinforcement learning [91], genetic algorithms [92], and neural networks [93].

5 Delivery and accessibility

The people we might be trying to engage with ABM are not especially concerned with the approach itself; they just need to know that it will help them answer the questions they are interested in. Established methodologies are being implicitly (sometimes explicitly) criticized when ABM is introduced to an area of research. If that is not hard enough to swallow, a lack of standardization and uniformity of approach in ABM lends it an impression of lacking rigour: building an ABM is still seen as something of a dark art. This is not necessarily a question of developing such standards, though undoubtedly they would appeal to the early majority; however, minimally, the accessibility and transparency of ABM needs to be improved. Developing methodology will make it easier to put together education courses, and will increase the openness of ABM. Empirical ABM in policy contexts also raises ethical issues that need to be considered.

5.1 Developing methodology

Squazzoni ([40] p. 220) concludes that ABM needs standards and methodologies if it is to be more widely adopted. Richiardi et al. [94], writing 4 years before that, suggested a three-step strategy working towards such standards involving a meta-analysis of current practice, whilst Alessa et al. [95] propose the development of e-infrastructure to support a community of practice in modelling complex social-ecological systems. In the same year, Grimm et al. [96] published the first version of their Overview, Design concepts and Details (ODD) protocol for documenting the structure and implementation of ABMs in ecology, which has subsequently been revised to appeal more to social simulation [97] and had various suggested extensions to it, most notably ODD+D [98] with its emphasis on recording decision-making. Grimm et al. [97] state that following the ODD protocol is also suitable for ordering the model design and implementation stages, whilst Schmolke et al. [99] propose the TRACE (TRansparent And Comprehensive Ecological modelling documentation) standard as a way of documenting the full modelling lifecycle, including sensitivity testing, calibration and validation.

The approaches of both TRACE and ODD suggest a linear modelling process, starting with purpose and problem formulation, through stages of design, implementation and testing very much akin to the traditional ‘waterfall’ model of software engineering. Such an approach is no longer preferred in the software development community, with ‘agile’ approaches now much more prominent. Agile approaches are fundamentally more iterative, and are aimed at situations where stakeholders in the software are not sufficiently clear about their requirements that they can agree on precise specifications for the developers to implement; neither are the developers sufficiently conversant with the stakeholders’ contexts that they are necessarily able to implement something appropriate. Maintaining an ongoing relationship with the stakeholders, agile software development projects are able to respond to evolving user requirements, and maximize the opportunity to deliver value.

An agile approach3 to the development of ABM is arguably more appropriate in inter- and trans-disciplinary research contexts, including addressing policy [65]. Interestingly, the emphasis on iteration in the companion modelling approach (see [100]), where modelling is used with stakeholders to resolve issues of conflict over environmental resources, is more akin to agile. Broader consideration of ways in which the principles of agile software development can be translated into policy-relevant ABM contexts would be useful.

There is considerable variation from one area of application to another in the standards by which an approach is assessed to have ‘worked’. Hence, expecting uniformity and consistency among ABM methodologies may be unrealistic, even if it is desirable to advance ABM as a whole. However, Richiardi et al.’s [94] approach, involving the meta-study of ABMs and practices, would be a promising way of eliciting emergent standards and methodologies, and their applicability in different contexts, should funding be made available enabling such a study. A similar approach would also apply to addressing issues with ‘arbitrariness’ in the ways in which models are built (see [101], para. 3.9), and in synthesizing and generalizing knowledge from the application of ABMs. With the late majority in mind, it would also be a good idea to make the standards and methodologies easy to use and understand, rather than burdensome constraints on research.

5.2 Education

A common complaint about ABM is the steep learning curve associated with taking it on, and these challenges are often underestimated. Partly, this is a question of forming collaborations rather than necessarily expecting a single individual to take on all the work. However, from the perspective of the field researcher, who might already find learning statistics an unwelcome, if necessary, distraction from their primary area of interest, ABM will surely seem an insurmountable obstacle. On the other hand, if such an individual has been taught about ABM during their studies, then arguably that familiarity will make it easier to engage with, individually or collaboratively, when appropriate later on. Even so, producing an ABM requires more investment of time than, say, loading survey data into a statistical package and running a standard statistical test.

Standards and methodologies make ABM easier to teach, and define some boundaries on the knowledge needed. The latter is significant for working with collaborators in the traditional social scientists, especially in terms of specialist requirements ABM has for data [58]. Knowledge elicitation techniques, such as those of Pahl-Wostl and Hare [22], are needed that attend to these requirements – traditional methods in the social sciences: interviews, focus groups, and questionnaire surveys, do not necessarily deliver data useful for configuring ABMs. Neither is it necessarily the case that the knowledge elicitation methods from artificial intelligence (e.g. [102]) will automatically be suitable, since AI has at least traditionally had a focus on the individual rather than the social. Methodological development is thus not just about how to build an ABM, but how this is done in collaboration with field research.

Education also plays a vital role in forming expectations of future policy analysts, stakeholders, businesses and funding bodies. One of the main obstacles is learning to program. Hamill [58] describes this as being a significant challenge, even in NetLogo, which is based on a family of languages designed to teach children programming. If only programming were more routinely taught in schools, and seen as being as important as arithmetic, reading and writing, there would be no need for this matter to be such a hurdle. ABM will find useful allies in the computing sciences and in industry if it joins voices with their calls for computing to be taken more seriously in the education of children. Further, students should not expect to be able to escape from formality by studying the social sciences in tertiary education. Squazzoni ([40], p. 221) has argued that ABM needs to be embedded in education in the social sciences at the undergraduate level.

5.3 Openness

Open Science is a term covering diverse views on the everyday practice of science with a view to fostering reuse, collaboration and accessibility of scientific knowledge [103]. There are various dimensions to Open Science with relevance to ABM. Harnad [104], for example, has been a long-term advocate of Open Access to scientific publications. There are also emerging principles for provision of Open access to Data: ‘Findable, Accessible, Interoperable and Reusable’ (FAIR; [105]). Though these both apply to ABM, of more specific relevance to the social simulation community is what might be termed ‘Open Modelling’ [106].

Poile and Safayeni [47] recommend code sharing and documentation to address issues with assumptions in formalization of theory. A lot of programmers learn by example from what others have done; such practices would be beneficial from an education perspective. Whilst there is a long-established tradition of sharing code in specialist repositories such as CoMSES-Net’s openabm.org website [107], tools and techniques such as literate programming [108] and provenance [109] to record metadata about software implementations are not typically applied.

Kremmydas et al. [110] also emphasize the importance of transparency when using ABMs in agricultural policy contexts. Their survey of literature finds a rapid growth of ABMs used to analyse agricultural policies since 2008, but only 11% of the 32 papers they surveyed released model source code and data enabling interested parties to reproduce the results and adjust assumptions to test sensitivity. Their recommendation to release source code and data emphasizes the points made earlier in the context of social theory, but provenance metadata around simulation structure and outputs would help even further.

Metadata, source code release and documentation may not suffice, however. Edmonds and Polhill [106] suggest a number of activities needed to make a model open. Releasing the source code is not necessarily enough: the code itself would ideally be neatly laid out, structured and commented. Version management should be used to keep track of which version of the source code is associated with a specific article. Typical behaviour should be documented, and tables of input-output data provided to characterize the important behaviour of the model, with accompanying visualizations.

A further issue with openness has to do with the potential ‘complicatedness’ of ABMs, which can make them incomprehensible to anyone except their developers. This is a matter raised by Couclelis [111] at Parker et al.’s [112] seminal workshop on ABM of LUCC. To Couclelis [111], ABM-LUCC models would need both environmental and social dynamics fitted to empirical observations – a matter that would introduce a degree of complicatedness to the model such that questions would legitimately be asked about whether the benefits of such a model outweigh the costs of building and calibrating it and analysing its outputs. Moss [113], however, argues that real social systems are “messy systems” (p. 2) in that there is not enough knowledge about them to make them amendable to elegant analysis; indeed, in ‘wicked problems’ what knowledge there is may well be contested [39]. Complexity and complicatedness cannot be ignored just because they are inelegant, difficult, or computationally expensive to process.

Sun et al. [27] draw on Loehle’s [114] concept of the Medawar zone to argue that, in comparison with conceptual ABMs, empirical ABMs have a relatively high level of complicatedness at which optimum model utility occurs. McDowall and Geels’s [115] response to Holtz et al. [116] article on modelling transitions draws on Andersson et al. [117] to characterize social-ecological systems as both complex (bottom-up, self-organized) and complicated (in structure), and hence only amenable to narrative analysis. In fact, Andersson et al. [117] draw on Edmonds and Moss [118] to argue that ‘descriptive’ agent-based simulations push the boundary of applications of ABM into higher complicatedness than the more traditional, Santa Fe Institute school of thought entailing high complexity but low complicatedness. Andersson et al. subsequently observe ([117], p. 155) that, though methods are not developed, simulation and narrative analysis should complement each other in wicked contexts. Such complementarity, particularly in collaborative contexts, may provide the means by which complicated, descriptive ABMs become more comprehensible.

5.4 Ethics

With increasing scrutiny over the use of personal data, we should be prepared that ABM will pose challenges when applied in empirical contexts that are less of an issue otherwise. In some cases, models may be simulating the future of specific households based on their data, for example: their employment prospects, health, marital status, beliefs, and other attributes most would quite reasonably consider an intrusion of their privacy. Whilst Fienberg and Makov [119] provide a method for detecting whether a release of population classification data will compromise confidentiality, we need methods for anonymization that allow data sharing and visualization of results. An alternative when publishing work is always to use synthetic populations with similar statistical properties to the empirical population. Methodologically this has utility at least in assessing the robustness of observed outcomes in the simulation, and any contingencies on the specific situation to which it is applied.

A further matter pertains to the involvement of ABMs in the policy process and the democratic accountability thereof. This has not traditionally been considered necessary – nobody was asked whether they wanted policy evaluated using General Equilibrium Models or even whether they agreed with the underlying assumptions – but there are good arguments for increasing openness to facilitate this [106].

It is also important from an ethical standpoint that ABM delivers on its promises, bearing in mind McDowall and Geels’s [115] sixth challenge to would-be modellers of societal transitions that they avoid being over-confident. This is significant where outcomes from simulation models might be used to influence policy decisions. Gilbert et al. [65], considering the ethics of applying models in policy contexts, emphasize the potential for modellers to derogate the uncertainty of their results for various reasons, including expert bias, a belief that highlighting the uncertainty will not be well-received, failure to find effective ways to communicate the uncertainty, and pressure from stakeholders. They note (para. 5.26) that such risks are increased when models have not been developed collaboratively with those who will use the results.

Ethics aside, over-confidence is also a concern for empirical applications of ABM in policy contexts per se. Matthews et al. [120] draw on Nissen’s [121] depiction of the way in which expectations of the potential of new technologies in comparison with established ones develop over time from unbelief (lower expectations) through euphoria (maximum expectations) and disappointment (lower expectations) to maturity (higher expectations). Matthews et al.’s [120] depiction augments Nissen’s [121] with a bifurcation at the trough of disappointment, adding a branch leading to abandonment.

To cross the chasm, ABM cannot rely on its ‘newness’ or ‘radicalness’, but on its ability to show, across the spectrum of metrics and qualitative assessments resulting from the research in section 5, that it can deliver better (more robust [122], more comprehensive) policy insights, helping to create policy that avoids the kinds of adverse outcome in section 3, through integrating diverse knowledge as described in section 4.

6 Discussion and conclusion

Figure 1 summarizes the discussion in the preceding sections as a ‘tube-map’ showing the steps needed for ABM to cross the chasm as interchange stations between the early adopters and early majority. Various themes emerge, which are visualized in Fig. 1 as the colours of the lines intersecting with each station. We now discuss these themes in turn.

A meta-study of ABM would be useful in assessing the level of adoption of ABM in various disciplines to look for niches that may not have been exploited, but also for areas where a shift in approach may be needed to appeal to pragmatists. A meta-study would also help with the development of methodology, and synthesizing knowledge from ABM applications. In addition to various reviews, cited earlier, examples of syntheses do exist, such as Cioffi-Revilla and Gotts’s [123] TRAP2 classification, and Hare and Deadman’s [12] taxonomy. However, a much larger-scale, comprehensive activity should be undertaken, not only of the models themselves, but of the contexts in which they are built.

Though we have said repeatedly that complexity is not attractive to policymakers, a critical examination of it would help in being clearer how issues with complexity are important to policymakers in avoiding unintended consequences and gaming the system. We need to formalize further the statistical signatures [20] of complexity so that we can broaden the means by which we argue our models have captured dynamics. We should anticipate that there are various classifications of complex system (see [124]) rather than attempting a grand unified theory of complexity at this stage, and undertake research in empirical and theoretical contexts that allow us to discover what they are, learn how to detect them, and what the ontological structures are that lead to associated observed statistical signatures.

Transparency is important in developing clear visualizations from ABMs and summarizing outputs for policymakers, and links through to the open science agenda and ethics. It may be easy for some in ABM to believe that model transparency is fully addressed through adopting ABM in the first place. Tesfatsion [125], for example, has argued (rightly) that ABM representations are more transparent than equations. However, as Lorscheid et al. [126] point out, the way the model is used to develop results is also an issue for transparency, as well as the results themselves being so [127].

Ethics is sufficiently important to merit a theme in its own right, and a matter that has not received a great deal of attention in the ABM literature ([65] being an exception). Hansson [128], notes that, because it has regarded them as being in the domain of the decision sciences, ethics has not treated risk appropriately, neither has it considered individual action in the context of other actions that, in effect, reduce the degree to which consequences of the individual action can be exclusively related to it.

Semantics is integral to developing meaningful hybridized models of social-ecological systems, as well as providing the basis for structural validation of models through ontological interoperability [32]. There is a sense in which ontologies can also facilitate transparency [129], through separating the representation of the model from its implementation, and ontologies have been argued by Livet et al. [130] and Gotts and Polhill [131] to have important roles as intermediaries in inter- and trans-disciplinary research. Troitzsch [132] has argued that extracting ontologies from models can lead to suggestions for how to improve them.

The Integration and Implementation Sciences (I2S) are a potentially useful community for ABM to network with, a basis for gathering together disparate views on decision-making, and provide a context in which ABM might be taught. Too many post-graduate students of ABM find themselves in situations where they are the only student in their department (or even their whole university) using the approach. Creating accredited on-line courses, summer schools and training materials is one way to address the education issue. More generally, interdisciplinary collaborations face institutional obstacles discussed by various authors (e.g. [133, 134, 135, 136]) that need to be overcome. If current institutions are not working, we will need to experiment with new ones.

Applying ABM in the empirical world means facing up to prediction and validation. Edmonds [137] outlines five purposes for modelling: prediction, explanation, theoretical exposition, description and illustration. As he defines them, prediction is reliably anticipating unknown data with useful accuracy (p. 42), explanation is about identifying plausible causal chains (p. 45), theoretical exposition concerns the establishment of general principles (p. 48) that need not necessarily apply in the real world (p. 49), description entails the (partial) representation of a real-world situation (p. 50), and illustration is, in essence, using ABM for the purposes of visualization (p. 53). We should not be naïve about which of these is going to be of most interest to policymakers. Getting predictions wrong should be seen as opportunities to learn, rather than the consequences of poor science by bad researchers. More important, however, is developing feasible criteria with which we can assess whether someone was in a position with respect to their model that they could reasonably have made the predictions they did.

The tube-map metaphor highlights that many of the themes cut across endeavours in areas of work not purely of interest to agent-based modellers (whilst also, it could be said, reflecting the fact that ABM is still something of an ‘underground’ activity). As Squazzoni [40] has said, ABM is fundamentally a cross-disciplinary endeavour. Tress et al. [138], clarifying some of the terminology around integrated research, emphasize that inter- and transdisciplinary work is about building new knowledge at boundaries. It is not uncommon for monodisciplinary researchers to expect their work to be usable without them having to adapt their methods, but this belief is mistaken: new knowledge is generated by working together. For ABM, this is important because it has specialist requirements that mean existing work may not be completely reusable. To take one example, hypothesis evaluation using significance testing with simulation experiments has been questioned [139]. This does not mean that statistical testing is irrelevant, just that we need new methods that are appropriate. Crossing the chasm should therefore be expected to entail active inter- and trans-disciplinary collaborations. It is clear that ABM will remain a cross-disciplinary endeavour, requiring researchers to be willing to work together in ways that do not necessarily speak to their core disciplinary audiences. ABM work will be done by coalitions, and part of the work of the chasm will involve training collaborators to work with agent-based modellers.

Footnotes

  1. 1.

    Itself insufficiently established that it has a clear Web of Science Category.

  2. 2.

    Pages 65–67 cover simulation modelling, but more of the kind that can be implemented in a spreadsheet. No mention is made of agent-based modelling!

  3. 3.
  4. 4.

Notes

Acknowledgements

Agent-based modelling work at The James Hutton Institute has been supported by the Scottish Government Environment, Agriculture and Food Strategic Research Portfolio (2005-2010, 2011-2016, 2016-2021), the European Commission (Grant Agreement nos. 12186, 225383, 265155, 613420), the Macaulay Development Trust (Fellowship E000677), the Economic and Social Research Council (grant reference RES-149-25-1075), and the Norwegian Research Council (Biosmart, project no. 244608; Newpath, project no. 235670). The authors gratefully acknowledge constructive feedback on an earlier draft of this article from two anonymous reviewers and from Dr. Tony Craig.

References

  1. 1.
    Walbert HJ, Caton JL, Norgaard JR (2018) Countries as agents in a global-scale computational model. JASSS 21(2):4 http://jasss.soc.surrey.ac.uk/21/3/4.html. Accessed 6 Dec 2018Google Scholar
  2. 2.
    Hegselmann R (2017) Thomas C. Schelling and James M. Sakoda: the intellectual, technical, and social history of a model. JASSS 20(3):15 http://jasss.soc.surrey.ac.uk/20/3/15.html. Accessed 6 Dec 2018Google Scholar
  3. 3.
    Gilbert N, Troitzsch LG (1999) Simulation for the social scientist. Open University Press, BuckinghamGoogle Scholar
  4. 4.
    Wooldridge M, Jennings NR (1995) Intelligent agents, theory and practice. Knowl Eng Rev 10(2):115–152Google Scholar
  5. 5.
    Moore GA (2014) Crossing the chasm: marketing and selling disruptive products to mainstream customers, Third edn. Harper Collins, New YorkGoogle Scholar
  6. 6.
    Rogers EM (2003) Diffusion of innovations, 5th edn. Free Press, New YorkGoogle Scholar
  7. 7.
    Bammer G (2017) Should we discipline interdisciplinarity. Palgr Commun 3:30.  https://doi.org/10.1057/s41599-017-0039-7 Google Scholar
  8. 8.
    Levitt T (1983) The marketing imagination. The Free Press, New YorkGoogle Scholar
  9. 9.
    Edmonds B, Hales D (2003) Replication, replication and replication: some hard lessons from model alignment. JASSS 6(4):11 http://jasss.soc.surrey.ac.uk/6/4/11.html. Accessed 6 Dec 2018Google Scholar
  10. 10.
    Galan JM, Izquierdo LR (2005) Appearances can be deceiving: lessons learned re-implementing Axelrod’s ‘evolutionary approach to norms’. JASSS 8(3):2 http://jasss.soc.surrey.ac.uk/8/3/2.html <Accessed 30 April 2018>Google Scholar
  11. 11.
    Bousquet F, Le Page C (2004) Multi-agent simulations and ecosystem management: a review. Ecol Model 176(3–4):313–332Google Scholar
  12. 12.
    Hare M, Deadman P (2004) Further towards a taxonomy of agent-based simulation models in environmental management. Math Comput Simul 64(1):25–40Google Scholar
  13. 13.
    Parker DC, Manson SM, Janssen MA, Hoffmann MJ, Deadman P (2003) Multi-agent systems for the simulation of land-use and land-cover change: a review. Ann Assoc Am Geogr 93(2):314–337Google Scholar
  14. 14.
    Janssen MA, Ostrom E (2006) Empirically based, agent-based models. Ecol Soc 11(2):37 http://www.ecologyandsociety.org/vol11/iss2/art37/ <Accessed 30 April 2018>Google Scholar
  15. 15.
    Smajgl A, Brown DG, Valbuena D, Huigen MGA (2011) Empirical characterisation of agent behaviours in socio-ecological systems. Environ Model Softw 26(7):837–844Google Scholar
  16. 16.
    Agar M (2003) My kingdom for a function: Modeling misadventures of the innumerate. JASSS 6(3):8 http://jasss.soc.surrey.ac.uk/6/3/8.html. Accessed 6 Dec 2018Google Scholar
  17. 17.
    Crooks A, Castle C, Batty M (2008) Key challenges in agent-based modelling for geo-spatial simulation. Comput Environ Urban Syst 32(6):417–430Google Scholar
  18. 18.
    Downing TE, Moss S, Pahl-Wostl C (2000) Understanding climate policy using participatory agent-based social simulation. Lect Notes Comput Sci 1979:198–213Google Scholar
  19. 19.
    Happe K, Kellerman K, Balmann A (2006) Agent-based analysis of agricultural policies: an illustration of the agricultural policy simulator AgriPoliS, its adaptation and behavior. Ecol Soc 11(1):49 http://www.ecologyandsociety.org/vol11/iss1/art49/ <Accessed 30 April 2018>Google Scholar
  20. 20.
    Moss S (2002) Policy analysis from first principles. Proc Natl Acad Sci U S A 99(suppl. 3):7267–7274Google Scholar
  21. 21.
    Étienne M (2014) Companion modelling: a participatory approach to support sustainable development. Springer, DordrechtGoogle Scholar
  22. 22.
    Pahl-Wostl C, Hare M (2004) Processes of social learning in integrated resources management. J Community Appl Soc Psychol 14(3):193–206Google Scholar
  23. 23.
    Squazzoni F, Casnici N (2013) Is social simulation a social science outstation? A bibliometric analysis of the impact of JASSS. JASSS 16(1):10 http://jasss.soc.surrey.ac.uk/16/1/10.html. Accessed 6 Dec 2018Google Scholar
  24. 24.
    O’Sullivan D, Evans T, Manson S, Metcalf S, Ligmann-Zielinska A, Bone C (2016) Strategic directions for agent-based modeling: avoiding the YAAWN syndrome. J Land Use Sci 11(2):177–187Google Scholar
  25. 25.
    Matthews KB, Rivington M, Blackstock KL, McCrum G, Buchan K, Miller DG (2011) Raising the bar? – the challenges of evaluating the outcomes of environmental modelling and software. Environ Model Softw 26:247–257Google Scholar
  26. 26.
    Borowski I, Hare M (2007) Exploring the gap between water managers and researchers: difficulties of model-based tools to support practical water management. Water Resour Manag 21(7):1049–1074Google Scholar
  27. 27.
    Sun Z, Lorscheid I, Millington JD, Lauf S, Magliocca NR, Groeneveld J, Balbi S, Nolzen H, Müller B, Schulze J, Buchmann CM (2016) Simple or complicated agent-based models? A complicated issue. Environ Model Softw 86:55–67Google Scholar
  28. 28.
    Wu K, Dunning D (2018) Hypocognition: making sense of the landscape beyond one’s conceptual reach. Rev Gen Psychol 22(1):25–35Google Scholar
  29. 29.
    HM Treasury (2011) The Magenta Book: guidance for evaluation. https://www.gov.uk/government/publications/the-magenta-book <Accessed 23 April 2018>
  30. 30.
    Bonabeau E (2002) Agent-based modeling: methods and techniques for simulating human systems. Proc Natl Acad Sci U S A 99(suppl. 3):7280–7287Google Scholar
  31. 31.
    Matthews RB, Gilbert NG, Roach A, Polhill JG, Gotts NM (2007) Agent-based land-use models: a review of applications. Landsc Ecol 22(10):1447–1459Google Scholar
  32. 32.
    Polhill G, Salt D (2017) The importance of ontological structure: why validation by ‘fit-to-data’ is insufficient. In: Edmonds B, Meyer R (eds) Simulating social complexity: a handbook, 2nd edn. Springer, Cham, p 141–172Google Scholar
  33. 33.
    Parker DC, Meretsky V (2004) Measuring pattern outcomes in an agent-based model of edge-effect externalities using spatial metrics. Agric Ecosyst Environ 101(2–3):233–250Google Scholar
  34. 34.
    Parker DC, Munroe DK (2007) The geography of market failure: edge-effect externalities and the location and production patterns of organic farming. Ecol Econ 60(4):821–833Google Scholar
  35. 35.
    Verhagen W, Van Teeffelen AJA, Compagnucci AB, Poggio L, Gimona A, Verburg PH (2016) Effects of landscape configuration on mapping ecosystem service capacity: a review of evidence and a case study in Scotland. Landsc Ecol 31(7):1457–1479Google Scholar
  36. 36.
    Allen J, Pryke M (2013) Financialising household water: Thames water, MEIF, and ‘ring-fenced’ politics. Camb J Reg Econ Soc 6(3):419–439Google Scholar
  37. 37.
    Byrne D (1998) Complexity theory and the social sciences: an introduction. Routledge, LondonGoogle Scholar
  38. 38.
    Jahn T, Bergmann M, Keil F (2012) Transdisciplinarity: between mainstreaming and marginalization. Ecol Econ 79:1–10Google Scholar
  39. 39.
    Rittel HWJ, Webber MM (1973) Dilemmas in a general theory of planning. Policy Sci 4(2):155–169Google Scholar
  40. 40.
    Squazzoni F (2010) The impact of agent-based models in the social sciences after 15 years of incursions. Hist Econ Ideas 18(2):197–233Google Scholar
  41. 41.
    Shove E, Pantzar M (2005) Consumers, producers and practices. J Consum Cult 5(1):43–64Google Scholar
  42. 42.
    Holtz G (2014) Generating social practices. JASSS 17(1):17 http://jasss.soc.surrey.ac.uk/17/1/17.html. Accessed 6 Dec 2018
  43. 43.
    Ingelhart R, Welzel C (2005) Modernization, cultural change and democracy: the human development sequence. Cambridge University Press, CambridgeGoogle Scholar
  44. 44.
    Spaiser V, Sumpter DJT (2016) Revising the human development sequence theory using an agent-based approach and data. JASSS 19(3):1 http://jasss.soc.surrey.ac.uk/19/3/1.html. Accessed 6 Dec 2018Google Scholar
  45. 45.
    Lindenberg S, Steg L (2007) Normative, gain and hedonic goal frames guiding environmental behavior. J Soc Issues 63(1):117–137Google Scholar
  46. 46.
    Rangoni R, Jager W (2017) Social dynamics of littering and adaptive cleaning strategies explored using agent-based modelling. JASSS 20(2):1 http://jasss.soc.surrey.ac.uk/20/2/1.html. Accessed 6 Dec 2018Google Scholar
  47. 47.
    Poile C, Safayeni F (2016) Using computational modeling for building theory: A double-edged sword. JASSS 19(3):8 http://jasss.soc.surrey.ac.uk/19/3/8.html. Accessed 6 Dec 2018Google Scholar
  48. 48.
    Filatova T, Polhill JG, van Ewijk S (2016) Regime shifts in coupled socio-environmental systems: review of modelling challenges and approaches. Environ Model Softw 75:333–347Google Scholar
  49. 49.
    Voinov A, Shugart HH (2013) ‘Integronsters’, integral and integrated modeling. Environ Model Softw 39:149–158Google Scholar
  50. 50.
    Pluempitiwiriyawej C, Hammer J (2000) A classification scheme for semantic and schematic heterogeneities in XML data sources. Technical Report TR00–004, University of Florida, Gainesville, FL, USAGoogle Scholar
  51. 51.
    Polhill G, Gotts N (2010) Semantic model integration: an application for OWL. ESSA 2010, Seventh Conference of the European Social Simulation Association, Montpellier, France, September 19–23, 2011Google Scholar
  52. 52.
    Cuenca Grau B, Horrocks I, Motik B, Parsia B, Patel-Scheider P, Sattler U (2008) OWL 2: the next step for OWL. Web Semant Sci Serv Agents World Wide Web 6(4):309–322Google Scholar
  53. 53.
    Horrocks I, Patel-Schneider PF, van Harmelen F (2003) From SHIQ and RDF to OWL: the making of a web ontology language. Web Semant Sci Serv Agents World Wide Web 1(1):7–26Google Scholar
  54. 54.
    Polhill G, Gotts N, Sánchez-Maroño N, Pignotti E, Fontenla-Romero Ó, Rodríguez-García M, Alonso-Bentanzos A, Edwards P, Craig T (2012) An ontology-based design for modelling case studies of everyday proenvironmental behaviour in the workplace. 6th International Congress on Environmental Modelling and Software, Leipzig, Germany, July 2012. https://scholarsarchive.byu.edu/iemssconference/2012/Stream-B/231/ <Accessed 30 April 2018>
  55. 55.
    Villa F, Athanasiadis IN, Rizzoli AE (2009) Modelling with knowledge: a review of emerging semantic approaches to environmental modelling. Environ Model Softw 24(5):577–587Google Scholar
  56. 56.
    Laniak GF, Olchin G, Goodall J, Voinov A, Hill M, Glynn P, Whelan G, Geller G, Quinn N, Blind M, Peckham S, Reaney S, Gaber N, Kennedy R, Hughes A (2013) Integrated environmental modelling: a vision and roadmap for the future. Environ Model Softw 39:3–23Google Scholar
  57. 57.
    Dechesne F, Ghorbani A, York-Smith N (2015) Introduction to the special issue on agent-based modelling for policy engineering. AI & Soc 30(3):311–313Google Scholar
  58. 58.
    Hamill L (2010) Agent-based modelling: the next 15 years. JASSS 13(4):7 http://jasss.soc.surrey.ac.uk/13/4/7.html. Accessed 6 Dec 2018Google Scholar
  59. 59.
    Schulze J, Müller B, Groeneveld J, Grimm V (2017) Agent-based modelling of social-ecological systems: achievements, challenges, and a way forward. JASSS 20(2):8 http://jasss.soc.surrey.ac.uk/20/2/8.html. Accessed 6 Dec 2018Google Scholar
  60. 60.
    Ahrweiler P (2017) Agent-based simulation for science, technology and innovation policy. Scientometrics 110(1):391–415Google Scholar
  61. 61.
    Aodha L n, Edmonds B (2017) Some pitfalls to beware when applying models to issues of policy relevance. In: Edmonds B, Meyer R (eds) Simulating social complexity: a handbook, Second edn. Springer, Cham, p 801–822Google Scholar
  62. 62.
    Polhill G (2018) Why the social simulation community should tackle prediction. Review of Artificial Societies and Social Simulation, 17 August 2018. https://rofasss.org/2018/08/17/gp/ <Accessed 6 November 2018>
  63. 63.
    Thomas SA, Lloyd DJB, Skeldon AC (2016) Equation-free analysis of agent-based models and systematic parameter determination. Physica A 464:27–53Google Scholar
  64. 64.
    Edmonds B (2016) The Aqua Book: guidance on producing quality analysis for government, HM treasury. JASSS 19(3):7 http://jasss.soc.surrey.ac.uk/19/3/reviews/7.html. Accessed 6 Dec 2018Google Scholar
  65. 65.
    Gilbert N, Ahrweiler P, Barbrook-Johnson P, Narasimhan KP, Wilkinson H (2018) Computational modelling of public policy: reflections on practice. JASSS 21(1):14 http://jasss.soc.surrey.ac.uk/21/1/14.html. Accessed 4 May 2018Google Scholar
  66. 66.
    Matthews KB, Blackstock KL, Rivington M, Waylen K, Miller DG, Wardell-Johnson D, Kovacic Z, Renner A, Ripa M, Giampietro M (2017) Delivering more than the “sum of the parts”: using Quantitative Storytelling to address the challenges of conducting science for policy in the EU land, water and energy nexus. 22nd International Congress on Modelling and Simulation, Hobart, Tasmania, Australia, 3–8 December 2017. https://www.mssanz.org.au/modsim2017/Keynote/matthews.pdf <Accessed 4 May 2018>
  67. 67.
    Saltelli A, Giampietro M (2017) What is wrong with evidence based policy, and how can it be improved? Futures 91:62–71Google Scholar
  68. 68.
    Nardi D, Brachman RJ (2003) Introduction to description logics. In: Baader F, Calvanese D, McGuinness DL, Nardi D, Patel-Schneider PF (eds) The description logic handbook: theory, implementation and applications. Cambridge University Press, Cambridge, pp 1–40Google Scholar
  69. 69.
    Wand Y, Weber R (1993) On the ontological expressiveness of information systems analysis and design grammars. J Inf Syst 3:217–237Google Scholar
  70. 70.
    Brewer MJ, Butler A, Cooksley S (2016) The relative performance of AIC, AICC and BIC in the presence of unobserved heterogeneity. Methods Ecol Evol 7:679–692Google Scholar
  71. 71.
    Flache A, Hegselmann R (2001) Do irregular grids make a difference? Relaxing the spatial regularity assumption in cellular models of social dynamics. JASSS 4(4):6 http://jasss.soc.surrey.ac.uk/4/4/6.html. Accessed 6 Dec 2018Google Scholar
  72. 72.
    North MJ, Collier NT, Ozik J, Tatara ER, Macal CM, Bragen M, Sydelko P (2013) Complex adaptive systems modeling with repast Simphony. Com Adap Sy Model 1:3Google Scholar
  73. 73.
    Wilensky U, Rand W (2015) An introduction to agent-based modeling: modeling natural, social, and engineered complex systems with NetLogo. MIT Press, MA, CambridgeGoogle Scholar
  74. 74.
    Cohn AG, Bennett B, Gooday J, Gotts NM (1997) Qualitative spatial representation and reasoning with the Region Connection Calculus. Geoinformatica 1(3):275–316Google Scholar
  75. 75.
    Barabási A-L, Albert R (1999) Emergence of scaling in random networks. Science 286(5439):509–512Google Scholar
  76. 76.
    Watts DJ, Strogatz SH (1998) Collective dynamics of ‘small world’ networks. Nature 393:440–442Google Scholar
  77. 77.
    Hamill L, Gilbert N (2009) Social circles: A simple structure for agent-based social network models. JASSS 12(2):3 http://jasss.soc.surrey.ac.uk/12/2/3.html. Accessed 6 Dec 2018Google Scholar
  78. 78.
    Gagnon J, Goyal S (2017) Networks, markets, and inequality. Am Econ Rev 107(1):1–30Google Scholar
  79. 79.
    Jackson MO, Rogers BW (2007) Meeting strangers and friends of friends: how random are social networks? Am Econ Rev 97(3):890–915Google Scholar
  80. 80.
    Beckman RJ, Baggerly KA, McKay MD (1996) Creating synthetic baseline populations. Transp Res A 30(6):415–429Google Scholar
  81. 81.
    Hamill L, Gilbert N (2016) Agent-based modelling in economics. John Wiley & Sons, ChichesterGoogle Scholar
  82. 82.
    Polhill JG, Gimona A, Gotts NM (2013) Nonlinearities in biodiversity incentive schemes: a study using an integrated agent-based and metacommunity model. Environ Model Softw 45:74–91Google Scholar
  83. 83.
    An L (2012) Modeling human decisions in coupled human and natural systems: review of agent-based models. Ecol Model 229:25–36Google Scholar
  84. 84.
    Schlüter M, Baeza A, Dressler G, Frank K, Groenwveld J, Jager W, Janssen MA, McAllister RRJ, Müller B, Orach K, Schwarz N, Wijermans N (2017) A framework for mapping and comparing behavioural theories in models of social-ecological systems. Ecol Econ 131:21–35Google Scholar
  85. 85.
    Groeneveld J, Müller B, Buchmann CM, Dressler G, Guo C, Hase N, Hoffmann F, John F, Klassert C, Lauf T, Liebelt V, Nolzen H, Pannicke N, Schulze J, Weise H, Schwarz N (2017) Theoretical foundations of human decision-making in agent-based land use models – a review. Environ Model Softw 87:39–48Google Scholar
  86. 86.
    Huber R, Bakker M, Balmann A, Berger T, Bithell M, Brown C, Grêt-Regamey A, Xiong H, Le QB, Mack G, Meyfroidt P, Millington J, Müller B, Polhill JG, Sun Z, Seidl R, Troost C, Finger R (2018) Representation of decision-making in European agricultural agent-based models. Agric Syst 167:143–160Google Scholar
  87. 87.
    Jager W (2017) Enhancing the Realism Of Simulation (EROS): Implementing and developing psychological theory in social simulation. JASSS 20(3):14 http://jasss.soc.surrey.ac.uk/20/3/14.html. Accessed 6 Dec 2018Google Scholar
  88. 88.
    Sun R (2012) Grounding social sciences in cognitive sciences. MIT Press, LondonGoogle Scholar
  89. 89.
    Orr MG, Lebiere C, Stocco A, Pirolli P, Bianica P, Kennedy WG (2018) Multi-scale resolution of cognitive architectures: a paradigm for simulating minds and society. In Thomson R, Dancy C, Hyder A, Bisgin H (eds) Social, Cultural, and Behavioral Modeling. SBP-BRiMS 2018. Lecture Notes in Computer Science 10899, pp. 3–15Google Scholar
  90. 90.
    Balke T, Gilbert N (2014) How do agents make decisions? A survey. JASSS 17(4):13 http://jasss.soc.surrey.ac.uk/17/4/13.html. Accessed 6 Dec 2018Google Scholar
  91. 91.
    Macy MW, Flache A (2002) Learning dynamics in social dilemmas. Proc Natl Acad Sci U S A 99(suppl. 3):7229–7236Google Scholar
  92. 92.
    Manson SM, Evans T (2007) Agent-based modeling of deforestation in southern Yucatán, Mexico, and reforestation in the Midwest United States. Proc Natl Acad Sci U S A 104(52):20678–20683Google Scholar
  93. 93.
    Salle IL (2015) Modeling expectations in agent-based models – an application to central bank’s communication and monetary policy. Econ Model 46:130–141Google Scholar
  94. 94.
    Richiardi M, Leombruni R, Saam N, Sonnessa M (2006) A common protocol for agent-based social simulation. JASSS 9(1):15. http://jasss.soc.surrey.ac.uk/9/1/15.html. Accessed 6 Dec 2018
  95. 95.
    Alessa LN, Laituri M, Barton M (2006) An “all hands” call to the social science community: Establishing a community framework for complexity modeling using agent based models and cyberinfrastructure. JASSS 9(4):6. http://jasss.soc.surrey.ac.uk/9/4/6.html. Accessed 6 Dec 2018
  96. 96.
    Grimm V, Berger U, Bastiansen F, Eliassen S, Ginot V, Giske J, Goss-Custard J, Grand T, Heinz SK, Huse G, Huth A, Jepsen JU, Jørgensen C, Mooij WM, Müller B, Pe’er G, Piou C, Railsback SF, Robbins AM, Robbins MM, Rossmanith E, Rüger N, Strand E, Souissi S, Stillman RA, Vabø R, Visser U, DeAngelis DL (2006) A standard protocol for describing individual-based and agent-based models. Ecol Model 198(1–2):115–126Google Scholar
  97. 97.
    Grimm V, Berger U, DeAngelis DL, Polhill JG, Giske J, Railsback SF (2010) The ODD protocol: a review and first update. Ecol Model 221(23):2760–2768Google Scholar
  98. 98.
    Müller B, Bohn F, Dreßler G, Groeneveld J, Klassert C, Martin R, Schlüter M, Schulze J, Weise H, Schwarz N (2013) Describing human decisions in agent-based models – ODD + D, an extension of the ODD protocol. Environ Model Softw 48:37–48Google Scholar
  99. 99.
    Schmolke A, Thorbek P, DeAngelis DL, Grimm V (2010) Ecological models supporting environmental decision making: a strategy for the future. Trends Ecol Evol 25(8):479–486Google Scholar
  100. 100.
    Barreteau O, Bousquet F, Étienne M, Souchère V, d’Aquino P (2014) Companion modelling: a method of adaptive and participatory research. In: Étienne M (ed) Companion modelling: a participatory approach to support sustainable development. Springer, Dordrecht, p 13–40Google Scholar
  101. 101.
    Waldherr A, Wijermans N (2013) Communicating social simulation models to sceptical minds. JASSS 16(4):13 http://jasss.soc.surrey.ac.uk/16/4/13.html. Accessed 6 Dec 2018Google Scholar
  102. 102.
    Gonzalez AJ, Dankel DD (1993) The engineering of knowledge-based systems: theory and practice. Prentice-Hall, EnglewoodGoogle Scholar
  103. 103.
    Fecher B, Friesike S (2014) Open science: one term, five schools of thought. In: Bartling S, Friesike S (eds) Opening science. Springer, Cham, pp 17–47Google Scholar
  104. 104.
    Harnad S (2015) Open access: what, where, when, how and why. In: Holbrook JB, Mitcham C (eds) Ethics, science, technology, and engineering: an international resource. Macmillan, Farmington Hills https://eprints.soton.ac.uk/361704/ <Accessed 5 November 2018>Google Scholar
  105. 105.
    Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak. A., Blomberg. A., Boiten, J.-W., Bonino da Silva Santos, L., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., Gonzalez-Beltran, A., Gray, A. J. G., Groth, P., Goble, C., Grethe, J. S., Heringa, J., ‘t Hoen, P. A. C., Hooft, R., Kuhn, T., Kok, R., Kok, J., Lusher, S. J., Martone, M. E., Mons, A., Packer, A. L., Persson, B., Rocca-Serra, P., Roos, M., van Schaik, R., Sansone, S.-A., Schultes, E., Sengstag, T., Slater, T., Strawn, G., Swertz, M. A., Thompson, M., van der Lei, J., van Milligen, E., Velterop, J., Waagmeester, A., Wittenburg, P., Wolstencroft, K., Zhao, J. and Mons, B. (2016) The FAIR guiding principles for scientific data management and stewardship. Scientific Data 3, 160018. doi: https://doi.org/10.1038/sdata/2016.18
  106. 106.
    Edmonds B, Polhill G (2015) Open modelling for simulators. In: Terán O, Aguilar J (eds) Societal benefits of freely accessible technologies and knowledge resources. IGI Global, Hershey, PA, USA, pp 237–254Google Scholar
  107. 107.
    Janssen MA, Alessa LN, Barton M, Bergin S, Lee A (2008) Towards a community framework for agent-based modelling. JASSS 11(2):6 http://jasss.soc.surrey.ac.uk/11/2/6.html. Accessed 6 Dec 2018Google Scholar
  108. 108.
    Knuth DE (1984) Literate programming. Comput J 27(2):97–111Google Scholar
  109. 109.
    Missier P, Belhajjame K, Cheney J (2013) The W3C PROV family of specifications for modelling provenance metadata. EDBT ‘13 Proceedings of the 16th International Conference on Extending Database Technology, Genoa, Italy, March 18–22, 2013, pp. 773–776Google Scholar
  110. 110.
    Kremmydas D, Athanasiadis IN, Rozakis S (2018) A review of agent based modeling for agricultural policy evaluation. Agric Syst 164:95–106Google Scholar
  111. 111.
    Couclelis H (2002) Why I no longer work with agents: a challenge for ABMs of human-environment interactions. In Parker DC, Berger T, Manson SM, McConnell WJ (eds) Agent-based models of land-use and land-cover change: report and review of an International Workshop, October 4–7, 2001, Irvine, California, USA. LUCC Focus 1 Office, Indiana University, USA, pp. 3–5. http://www.csiss.org/events/other/agent-based/papers/couclelis.pdf <Accessed 10 October 2018>
  112. 112.
    Parker DC, Berger T, Manson SM, McConnell WJ (2002) Agent-based models of land-use and land-cover change. Report and review of an international workshop October 4-7, 2001, Irvine, California, USA. LUCC Report Series No. 6. LUCC Focus 1 Office, Anthropological Center for Training and Research on Global Envrionmental Change, Indiana UniversityGoogle Scholar
  113. 113.
    Moss S (2001) Editorial introduction: messy systems – the target for multi agent based simulation. In Moss. S. and Davidsson, P. (eds.) Multi-Agent-Based Simulation: Second International Workshop MABS 2000, Boston, MA, USA, July. Revised and Additional Papers. Lecture Notes in Artificial Intelligence 1979, pp. 1–14Google Scholar
  114. 114.
    Loehle C (1990) A guide to increased creativity in research – inspiration or perspiration. BioScience 40(2):123–129Google Scholar
  115. 115.
    McDowall W, Geels FW (2017) Ten challenges for computer models in transitions research: commentary on Holtz et al. Environ Innov Soc Trans 22:41–69Google Scholar
  116. 116.
    Holtz G, Alkemade F, de Haan F, Köhler J, Trutnevyte E, Luthe T, Halbe J, Papachristos G, Chappin E, Kwakkel J, Ruutu S (2015) Prospects of modelling societal transitions: position paper of an emerging community. Environ Innov Soc Trans 17:41–58Google Scholar
  117. 117.
    Andersson C, Törnberg A, Törnberg P (2014) Societal systems – complex or worse? Futures 63:145–157Google Scholar
  118. 118.
    Edmonds B, Moss S (2005) From KISS to KIDS – an ‘anti-simplistic’ modelling approach. In Davidsson P, Logan B, Takadama K (eds) Multi-agent and multi-agent-based simulation. MABS 2004. Lecture Notes in Computer Science 3415, pp. 130–144Google Scholar
  119. 119.
    Fienberg SE, Makov UE (1998) Confidentiality, uniqueness, and disclosure limitation for categorical data. J Off Stat 14(4):385–397Google Scholar
  120. 120.
    Matthews KB, Schwarz G, Buchan K, Rivington M, Miller D (2008) Wither agricultural DSS? Comput Electron Agric 61:149–159Google Scholar
  121. 121.
    Nissen V (1995) An overview of evolutionary algorithms in management applications. In: Biethahn J, Nissen V (eds) Evolutionary algorithms in management applications. Springer-Verlag, Berlin, pp 44–97Google Scholar
  122. 122.
    Bankes SC (2002) Tools and techniques for developing policies for complex and uncertain systems. Proc Natl Acad Sci U S A 99(suppl. 3):7263–7266Google Scholar
  123. 123.
    Cioffi-Revilla C, Gotts N (2003) Comparative analysis of agent-based social simulation: GeoSim and FEARLUS models. JASSS 6(4):10 http://jasss.soc.surrey.ac.uk/6/4/10.html. Accessed 6 Dec 2018Google Scholar
  124. 124.
    Gotts NM, van Voorn GAK, Polhill JG, de Jong E, Edmonds B, Hofstede GJ, Meyer R (in press) Modelling socio-ecological systems. Ecol Complex.  https://doi.org/10.1016/j.ecocom.2018.07.007
  125. 125.
    Tesfatsion L (2006) Agent-based computational modeling and macroeconomics. In: Colander D (ed) Post Walrasian macroeconomics: beyond the dynamic stochastic general equilibrium model. Cambridge University Press, Cambridge, pp 175–202Google Scholar
  126. 126.
    Lorscheid I, Heine B-O, Meyer M (2012) Opening the ‘black box’ of simulations: increased transparency and effective communication through the systematic design of experiments. Comput Math Organ Theory 18(1):22–62Google Scholar
  127. 127.
    Lee J-S, Filatova T, Ligmann-Zielinksa A, Hassani-Mahmooei B, Stonedahl F, Lorscheid I, Voinov A, Polhill G, Sun Z, Parker DC (2015) The complexities of agent-based modeling output analysis. JASSS 18(4):4 http://jasss.soc.surrey.ac.uk/18/4/4.html. Accessed 6 Dec 2018Google Scholar
  128. 128.
    Hansson SO (2010) The harmful influence of decision theory on ethics. Ethical Theory Moral Pract 13(5):585–593Google Scholar
  129. 129.
    Polhill JG, Gotts NM (2009) Ontologies for transparent integrated human-natural system modelling. Landsc Ecol 24(9):1255–1267Google Scholar
  130. 130.
    Livet P, Muller J-P, Phan D, Sanders L (2010) Ontology, a mediator for agent-based modeling in social science. JASSS 13(1):3 http://jasss.soc.surrey.ac.uk/13/1/3.html. Accessed 6 Dec 2018Google Scholar
  131. 131.
    Gotts NM, Polhill JG (2009) Narrative scenarios, mediating formalisms, and the agent-based simulation of land use change. In Squazzoni F (ed) Epistemological aspects of computer simulation in the social sciences. Second International Workshop, EPOS 2006, Brescia, Italy, October 5–6, 2006. Revised selected and invited papers. Lecture Notes in Artificial Intelligence 5466. Berlin: Springer. pp. 99–116Google Scholar
  132. 132.
    Troitzsch KG (2015) What can from learn from extracting OWL ontologies from a NetLogo model that was not designed from such an exercise. JASSS 18(2):14 http://jasss.soc.surrey.ac.uk/18/2/14.html. Accessed 6 Dec 2018Google Scholar
  133. 133.
    Campbell LM (2005) Overcoming obstacles to interdisciplinary research. Conserv Biol 19(2):574–577Google Scholar
  134. 134.
    Gardner SK (2013) Paradigmatic differences, power, and status: a qualitative investigation of faculty in one interdisciplinary research collaboration on sustainability science. Sustain Sci 8(2):241–252Google Scholar
  135. 135.
    Podesta GP, Natenzon CE, Hidalgo C, Toranzo FR (2013) Interdisciplinary production of knowledge with participation of stakeholders: a case study of a collaborative project on climate variability, human decisions and agricultural ecosystems in the Argentine Pampas. Environ Sci Pol 26:40–48Google Scholar
  136. 136.
    Schuitema G, Sintov ND (2017) Should we quit our jobs? Challenges, barriers and recommendations for interdisciplinary energy research. Energy Policy 101:246–250Google Scholar
  137. 137.
    Edmonds B (2017) Different modelling purposes. In: Edmonds B, Meyer R (eds) Simulating social complexity. Springer, Cham, p 39–58Google Scholar
  138. 138.
    Tress G, Tress B, Fry G (2005) Clarifying integrative research concepts in landscape ecology. Landsc Ecol 20(4):479–493Google Scholar
  139. 139.
    White JW, Rassweiler A, Samhouri JF, Stier AC, White C (2014) Ecologists should not use statistical significance tests to interpret simulation results. Oikos 123(4):385–388Google Scholar

Copyright information

© The Author(s) 2019

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.The James Hutton InstituteAberdeenUK

Personalised recommendations