Keywords

1 Introduction

A large body of public policy analysis is devoted to retrospective analysis (evaluation), which tries to understand the causes and consequences of policies after they have been implemented (Tsoukiàs et al. 2013). Equally important in policy analysis is the role of ex ante evaluations, i.e. prescriptive analysis involving impact assessment carried out at the early stages of policy development, cf., e.g., (Turnpenny et al. 2009). Ex ante evaluation encompasses forecasting of consequences if policies were to be implemented and prescriptions about which policies should be implemented. One important aim of ex ante decision support within the context of policy making therefore involve developing ways of facilitating for policy-makers to create policies that is consistent with their preferences while at the same time being accepted by other stakeholders, cf. (Bryson 2007). However, there is a lack of operational approaches for addressing the cognitive activity of designing or finding policy options. The long term implications of policy making imply the need to consider a vast range of possible futures, often characterized by the presence of multiple stakeholders and sometimes characterized by large uncertainties as well.

Problem structuring are now widely acknowledged as an important feature of strategic decision support tools and there is a growing but still small body of research and practice on how to integrate such methods with other formal and/or quantitative methods for decision evaluation, see (Tsoukiàs et al. 2013). The aim of this paper is therefore to provide a basis for the selection and use of modeling and decision support tools from the operational research field in a common toolkit for ex ante policy analysis, such that there is a logical relationship between the tool used to the stage and sub-stage of the policy making process and how the modeling tools provide decision value in the policy making process.

2 Policy Making Process and Policy Formulation

Herein public policy is defined as a purposeful, goal-oriented action that is taken by government to deal with societal problems or to improve societal conditions for the wellbeing of its population. Decision making is the cognitive process of selecting a course of action from many possible ones which might be initially known or not known. According to Bero and Jadad (1997) policy making has some distinguishing features having an impact on the decision process:

  1. (i)

    A population level decision making context;

  2. (ii)

    Explicit justification is required, as policies are formally and informally evaluated by government agencies, by outside consultants, by interest groups, by the mass media, and by the public;

  3. (iii)

    Effect of the existing political ideology and governance; and

  4. (iv)

    Evidence of systematic reviews on public policy decisions is hard to come by.

We relate to three-stage process model for policy-making conforming to the well-established policy cycle models of Lindblom (1968) and Howlett et al. (2009). The process model includes the following steps; (i) problem identification stage, (ii) policy formulation stage, and (iii) policy implementation stage, see Fig. 1.

Fig. 1.
figure 1

Policy-making process model.

Full size image

This process model acknowledges that policy-making is an ongoing, continuous process that requires continuous assessment, evaluation, and reaction. Each stage has a set of activities according to Table 1 below.

Table 1. Stages and sub-stages of the policy-making process model.
Full size table

Hence, a large part of the activities occurring within a policy cycle is about understanding, formulating and structuring “problems”, see also (Mingers and Rosenhead 2004; Tsoukiàs 2007). Thus, problem structuring is a key element of the public policy making process and in particular for the policy formulation stage. Modeling for problem structuring in order to find feasible policy options and decision evaluation of these options is of prime concern in the policy formulation stage, since in this stage “policy makers must ensure that there is a meaningful definition of the problem” and “need a comprehensive understanding of the problem” in order to identify different policy options (Hamilton 2010). Having this said, in the next section we focus on operational research and the decision sciences within the context of supporting policy formulation and a selection of methods for an integrated modeling approach to policy formulation decision support is suggested.

3 Modeling for Policy Formulation

3.1 Problem Analysis

Problem analysis calls for problem structuring methods (PSM) which aim to enable a better understanding of unstructured problems characterized by the existence of multiple actors, multiple perspectives, incommensurable and/or conflicting interests, intangibles and key uncertainties. The methods rely heavily on engaging with policy makers, adopting a facilitative mode of engagement, and simple, often qualitative, models (Franco and Montibeller 2010). See (Mingers and Rosenhead 2001) for an overview of the characteristics of a wide range of PSM’s.

The use of cognitive maps is among the early approaches to enable for problem understanding, originally intended for representing social science knowledge, see (Axelrod 1976). A cognitive map is a directed graph where nodes represents societal concepts or variables (such as subsidy size, traffic intensity, CO2 emission rates) and links represent causal connections between these concepts. With respect to the simple example, two nodes “car traffic” and “emissions” and a link from the former to the latter would then mean that the former causally has an impact on the latter, negative or positive.

Multiple perspectives of the problem from different actors can be represented and debated for improving the policy-makers’ understanding of the problem situation and sharing it with associates. Several “dialects” of cognitive maps have emerged, which share focus on the qualitative aspects of problem structuring, see, e.g., (Acar and Druckenmiller 2006).

Although a cognitive map serves its purpose as a problem structuring method, its content is too informal to be used as an underlying model for simulation of how changes in one node affects other nodes through causal relations. A causal map in the sense of Acar (1983) is a diagram that maps problem variables, or factors, and defines their interdependencies in a quantitative manner. Goals for different actors of the problem can be defined to facilitate group decision making and negotiation analysis. The primitive elements used in the causal mapping method are: Independent variables (sources of change), dependent variables (intermediate and outcome/goal variables), change transmission channels, change transfer coefficients, time lags, minimum thresholds (minimum change to be transferred), status-quo level of the map (the initial state of the system, considering 0 % relative change), and goal vector representing the targeted changes in goal variables relative to status-quo values.

As a simple example, consider a local government wanting to reduce CO2 emissions in order to contribute to climate targets and improve central town air quality. The upfront objectives are focused on the environmental issue. One policy option is to increase the subsidy of bus tickets, where the underlying hypothesis in terms of causality is that higher subsidies (lower ticket prices) will cause more people to take the bus instead of cars when going into town, thereby reducing emissions also enabling for more citizens to access the central town area increasing town commerce, which is the main concern of an influential stakeholder group. However, there is also a positive causal relationship between car traffic and town commerce, increasing the complexity of the policy option making. In the problem analysis, it becomes clear that for many citizens it is not the bus ticket price that cause them to avoid taking the bus, but rather the frequency of buses. Further, reducing emissions can also be done by increasing the proportion of gas driven buses. A causal map of the policy problem can then be illustrated in Fig. 2 above.

Fig. 2.
figure 2

Causal map model of bus ticket price subsidy policy problem with four actors, three controlling resources on the left hand side and one actor having an interest in the town commerce factor. The goal vector consists of that the three controlling actors want to reduce CO2 emissions by 20 % while the third actor wants to increase town commerce with 5 %. Each link has a change transfer coefficient and a time lag. For instance, the link from “Subsidy bus tickets” into “Car traffic” has a coefficient of −0.2 and a time lag of 1 month.

Full size image

The richer computational semantics of the causal mapping approach support simulation in ways that other varieties of cognitive mapping and causal mapping do not, see (Druckenmiller and Acar 2009). The method includes indications not only of the signs of the presumed causal influences, but also of their intensities, minimum threshold values and the time lags. Still, it is possible to utilize the non-quantitative elements of the map analogously to how cognitive maps are employed. Three classes of analyses are supported, see (Acar 1983) for detailed description:

  1. 1-

    Backward analysis:

    Clarifying, testing and reassessing assumptions about the cause-effect relationships of the situation. It can be divided into:

    1. (a)

      Major assumptional analysis (validity of the major aspects of the graph elements and relationships); and

    2. (b)

      Minor assumptional analysis (validity of detailed qualifications and quantifications of the graph).

  2. 2-

    Structural analysis:

    It includes: Graph scope, connectivity analysis, reachability analysis and goal comparative analysis (qualitative and quantitative).

  3. 3-

    Forward analysis:

    Implications of a “change scenario” through simulating the transfer of change, includes:

    1. (a)

      Scenario simulation: running change scenarios on the graph – simulating transfer of change over time from origins throughout the map.

    2. (b)

      Goal negotiation analysis: Goal feasibility and compatibility of goals, identify scenarios realizing a goal or goals jointly). Effectiveness or efficiency – expression of outcomes in comparison with objectives and relative to change needed at origins.

Within policy formulation, a policy option would be represented by a change scenario. Feasible scenarios would be the ones who satisfy the actors by reaching their goals in different ways, and in this way scenarios can be generated/discovered (Bryant and Lempert 2010). The method cannot capture all the intricacies of a situation to its minutest details, but can sufficiently map out its principal elements and their relationships.

3.2 Design and Generation of Policy Options

The identification and creation of new alternatives is one of the most important aspects of any decision support. If the decision alternatives under consideration are weak, it will lead to a poor choice (Brown 2005). Thus, support in the generation of feasible options is important for policy formulation.

A policy option is represented by a scenario of change from the status-quo level of the causal map. A base line scenario is defined with initial values for the problem’s key variables with zero initial relative changes. The desired state of the system is represented by a goal vector (targeted relative changes in outcome variables compared to the base line scenario). The causal map allows triggering change transfer by a ‘Pure scenario’, a single change at one source, or a ‘Mixed scenario’, change in several sources all at once or with a time lag.

Graph change analysis allows us to investigate the dynamic consequences of entering a change in one of the graph origins, thus simulating the propagation of change throughout the causal map. Several decision analytic concepts are readily usable for scenario generation:

  • “Optimality”, relating scenarios to the level or degree to which the goal vector is realised.

  • “Dominance”, reducing the number of scenarios by discarding dominated ones.

  • “Resource constraint”, cost of input triggers of certain magnitude at a node. Triggering a scenario may impose on the controlling actor the supply of funds and resources.

  • “Goal compatibility”, goal vectors of different actors is compatible if a scenario can be found to realize them jointly.

The simulation is run upon the causal map modeled, whereas the set of objectives and their target values are used for identifying feasible scenarios. Based on the simulation results of an initially large sample of scenarios generated by, e.g., full factorial design or Latin hypercube sampling, unsatisfactory scenarios not realizing the goal vector or being dominated are filtered out, while scenarios deemed efficient and “interesting” according to some predefined decision rule based upon resource constraints and goal compatibility concepts mentioned above are suggested as policy options for further evaluation.

3.3 Evaluation of Policy Options

Arriving at a set of feasible options which are non-dominated, further discrimination between them will call for taking preference information into account. Although, the scenario-based dynamic simulation allows decision makers to identify alternative policy options and verify their effects over time, it does not provide the explicit decision evaluation of policy options. Multi-criteria decision analysis (MCDA) has been extensively used to support a wide variety of complex decision problems as a tool for evaluating options where decisions involve the achievement of multiple objectives and considering multiple decision makers and stakeholders, see, e.g., (Zavadskas et al. 2014). MCDA typically distinguish between a decision maker which has control of some decision variable, and stakeholders, which are affected by the consequences of a decision. This distinction is straightforwardly analogous to the actor representations of the causal map. Applying MCDA should thus provide the relative global performance of each alternative, and is particularly useful when selecting one out of a finite set of feasible alternatives.

When evaluation of policy options is guided by a decision analysis approach, a relevant MCDA model is required equipped with preference elicitation methods for capturing policy makers’ and stakeholders’ preferences. Given that we have identified feasible policy options from scenario generation, there are two main tasks remaining in structuring MCDA evaluation models; (i) representation of objectives in a structure, commonly a value tree, and (ii) the definition of attributes to measure the achievement of objectives (Franco and Montibeller 2011).

Traditionally in MCDA, the decision process starts by structuring the problem as an attribute tree hierarchically ordering the decision makers’ aims at different abstraction levels, from fundamental objectives (such as “improved environment”) to lower level attributes (such as “NOx reduction”) where the latter contributes to the former in a hierarchical value tree. It is generally assumes that each criterion can be operationalized by a set of measurable attributes allowing for assessing the consequences arising from the implementation of any particular alternative. In the next step preferential information is elicited. The relative importance of criteria is captured in weights for each criterion at each abstraction level. At the lowest level of the value tree these objectives are translated into attributes, with each one of them evaluating a given characteristic of the decision options (for example, an objective ‘efficiency’ may be measured by the attribute ‘operating cost’). The performance of each decision option against each attribute is determined and weights reflecting acceptable trade-offs of performance among objectives are elicited from the decision-makers. Given the presence of many decision makers or stakeholders, for a group decision a decision has to be made and there might be disagreement with respect to what the best option is, but the group choice still has sufficient support from the group, i.e. there is a way of representing the collective preferences of the group. This differs from negotiation in that in such as process each stakeholder might simply abandon the decision process and there is no decision at all, see, e.g., (Kilgour et al. 2010).

There is no universal way of applying MCDA to group decision and negotiation processes, but the underlying use of a causal map and scenarios generated leads to that the features of interest are preference elicitation and preference aggregation. With respect to integrating multi-criteria decision analysis with causal maps and simulation of scenarios, Comes et al. (2011) presents the concept of “decision maps”, integrating problem structuring and scenario planning using causal maps with multi-criteria decision analysis, but keeps the focus on operational decision making and to a lesser extent on policy making. However, the structure and content of the map informs the building of the value tree, in an ad hoc translation, in which, the causal map can be used to elicit preferences, cf. (Comes et al. 2011). For a simple example of this, see Fig. 5 showing a value tree model based upon the causal map of Fig. 4. Montibeller and Belton (2006) investigate various ways to use causal maps as the underlying problem structuring tool and extending it with decision evaluation features and/or using the map to effectively inform the decision analysis model in the form of a multi attribute value tree.

For simulation, in the SEMPAIFootnote 1 framework reported in Hansson et al. (2011), simulation results of flooding models are combined with multi-criteria decision analysis where multiple stakeholders are present. Elicitation or anticipation of stakeholder preferences is done or assessed in the form of utility statements for each stakeholder or stakeholder group regarding each specific policy option and criteria weights. Stakeholders give the different outcomes a ranking order if they are unsure of their preferred choice and methods for computational decision analysis with imprecise information is promoted in order to support such statements, see (Larsson et al. 2005, Danielson et al. 2007). The result is a preference assessment for each decision maker and/or stakeholder and the value tree is constructed with stakeholders as the lowest level of the tree, see Fig. 4 for the value tree corresponding to the causal map of Fig. 3 where each decision maker has a trade-off expressed as criteria weights between cost and their benefits of concern. With respect to the model in Fig. 4, we can consider the following simple preference scheme using value functions V(S) of the scenarios for each criterion delimiting the decision evaluation to the status-quo scenario called S 0 and the generated scenario shown in Fig. 3 called S 1 (also in Table 2).

Fig. 3.
figure 3

Example of a generated scenario reaching both the emission and the town commerce target. The scenario consists of increasing the subsidy with 15 % at the initial timestep, increasing bus frequencies at the sixth, and increasing the proportion of gas driven buses at timestep 12.

Full size image
Fig. 4.
figure 4

Value tree counterpart of causal map in Fig. 3 in accordance to the SEMPAI framework. In this example, DM Subsidy’s weight for CO2 emissions is 40 % so for simplicity precise numerical weights are used here.

Full size image
Table 2. Preference schema.
Full size table

Decision evaluation is then done stakeholder-wise. Each decision maker or stakeholder will receive a utility value for each scenario and the choice of selecting or discarding one scenario will be based upon its total utility through aggregating the stakeholders’ utility values (see Fig. 5) together with group decision admissibility concepts such as maximum disagreement thresholds or minimum consensus thresholds (Fasth et al. 2013).

Fig. 5.
figure 5

Comparison of scenarios generated by adding part-worth utilities for each stakeholder. There is disagreement since DM Subsidy and DM Gas driven buses consider S 1 to be inferior to S 0 while the other actors have differing preferences.

Full size image

4 Concluding Remarks

In this paper we have outlined the linkages between the policy formulation stage of the policy making process model and contemporary decision analytic support methodologies and based upon that discussed an integrated modeling framework for computer assisted policy formulation. The approach suggested combines causal mapping for problem structuring and simulation of consequences together with identifying scenarios deemed feasible for further investigation in a multi-criteria decision evaluation. This sequence follows a logical work process conforming to the activities of the policy formulation process.

Scenario generation helps policy-makers in identifying feasible options from a possibly vast space of possible ones reaching stipulated targets, while the decision evaluation can supports an in-depth performance evaluation of policy proposals taking the preferences of stakeholders into account. The resulting integrated model for the policy problem situation can improve the process of reaching well-accepted scenarios and a transparent design of policy options while taking into account costs, benefits, resource constraints, different perspectives, and multiple objectives. In the context of policy-making the decision makers are often confronted with large and complex amounts of information, not seldom of a conflicting nature and reflect multiple interests. Problem structuring and decision analysis allows policy-makers to judge the performance of alternative policy options and from stakeholders’ different points of view of what represents a positive or negative policy impact. Hence, policy impact assessment frameworks and tools have much to gain from implementing decision analytic support methodologies.