Keywords

1 Introduction

Many work domains entail complex decision making tasks. In such tasks, users need to assess and relate multiple data sources, each imposing different constraints, to achieve a goal. The context in which these decisions need be made can also vary [1, 2]. For example, decisions may need to be made very quickly, or there may be considerable time available to make them. Here, we tackle the question of how such decisions should be supported by computer interfaces and tools, and how the rich cognitive science of decision making can be applied to ensure that decision support provided users in their computing systems is useful and usable [3].

We focus on supporting geospatial decision making. Geospatial decisions are those whose output is the choice, or assessment, of a location, or locations, under various, often competing, constraints. In naval command and control, for example, planners engage in geospatial decision making to determine the likely locations of pirate activity given merchant shipping, weather and previous episodes of piracy. They do this in order to intelligently position surveillance and interdiction assets against the pirates [4]. Similarly, military navigators perform geospatial decision making to define safe and secure locations to position and route ships and submarines to achieve various mission objectives. These routes must avoid terrain and other navigation hazards, on the one hand, while allowing ships and submarines to remain undetected, on the other. In both civilian and military weather forecasting, forecasters make geospatial decisions when they must predict flash flooding, say, at discrete geospatial locations, from a variety of raw sensor, and derived meteorological model, outputs [5].

2 Trends in Geospatial Decision Support: Data to Decision

The support available to users to perform complex geospatial decision making tasks has improved dramatically over the last twenty years [6]. However, the advances have focused more on improving the technical, underlying computational infrastructure than in addressing the cognitive requirements for what are ultimately complex psychological tasks. A synthesis of the historical trends in geospatial decision support that we are observing in our capacity as scientific design consultants on various fielded systems is offered in Fig. 1. This figure reveals subtle limits in the understanding of the cognitive requirements of decision making by system engineers and interface developers.

Fig. 1.
figure 1

The shrinking gulf of evaluation in geospatial decision making over time, and the march towards a geospatial objective function.

Full size image

Figure 1 is an illustrative plot that shows how scientific and technical advances in the means of geospatial decision support, in time (on the abscissa), are attempting to provide decision support to reduce decision maker’s gulf of evaluation (on the ordinate). The gulf of evaluation is the famous psychological construct in human computer interaction (HCI), that refers to the gap between a user’s internal goals and what an external computer system delivers to achieve them [7, 8]. Mapped to decision support systems, the gulf reflects the mismatch between what a user needs and what the system provides the user to perceive, interpret and evaluate relevant data to make a decision. In Fig. 1, the gulf is broken down into three stages (labelled in white text), (i) getting the data (to perceive it), (ii) relating the data sources together (to interpret it), and (iii) mapping the data to the decision (to evaluate its impact on the decision). Illustrating these stages in an actual application domain both illustrates the gulf, in context, and how research and development trends in geospatial decision support are attempting to bridge, or reduce, it.

Before the advent of networked, digital information systems populated with geo-referenced data, users were challenged to simply obtain and relate decision-relevant information. For example, civilian and military navigators had access to a mix of digital and paper maps, received tasking in writing and tasking updates verbally, obtained printouts and notes of weather forecasts, and heard verbally relayed facts and constraints. The users faced a significant gulf of evaluation in that they had to try and scan, read and recall all the disparate sources of information to perceive it all. If the data was sparse, or missing, they also had to mentally interpolate it. Then they had to try and geo-reference it to begin to relate it together, and try to determine how it constrained and impacted their ship routing. All the while they had to try not to forget any of the data, or their emerging interpretation of it. The task was burdensome and prone to errors. It required skill and expertise to know how to prioritize information and when to judiciously deploy rules of thumb and heuristics to make up for missing, incomplete or forgotten data.

Faced with these significant challenges, unsurprisingly, users resorted to the creation of artifacts and associated processes to relate the data to the map and to externalize their memory of it. Such workarounds are often observed in operational settings as users take it on themselves to try and make up for perceived deficiencies in their computer-based, or other support tools, and business processes [9, 10]. What is interesting and informative for decision support design, is that these workarounds may become metaphors that are then pursued by interface designers, and then become unhelpfully entrenched [11]. For example, faced with paper maps and data printouts, military and civilian navigators used grease pencil markup on transparent acetate overlays superimposed on maps to geo-reference, remember and relate data sources. Modern digital information systems have copied and maintained the acetate metaphor by showing geo-referenced data sources as graphical overlays on geoplots. This is true for client-based, commercial Geographical Information Systems (GIS) [12], dedicated commercial maritime navigation Electronic Chart Display and Information System (ECDIS), and for web-based, freely available and broadly used geospatial mapping and visualization applications such as Google Earth [13]. Analogous to overlaying multiple acetates on a physical map, multiple sources can potentially be related simultaneously by toggling on/off available data sources. Further, most systems go further by providing sliders to set the opacity of each overlay to make several overlays visible at once, even when data is occluded as a result. Comparable, configurable overlay systems are available to commercial and military weather forecasters. For example, US Government forecasters in the National Weather Service use a system called the Advanced Weather Interactive Processing System (AWIPS) which allows a mix of raw and derived data and model predictions to be overlaid on geospace [5]. Modern digital GIS and other systems can also perform mathematical interpolation on sparse data to yield continuous, interpolated data representations that can be provided to users as another overlay, as a heatmap or other 2D color plot [12].

Digital geospatial data overlay systems do provide some decision support to their users. By digitizing and co-locating the data in single system, and presenting it overlaid on a map, users can perceive the data, relate it to geospace, and integrate it with other sources. Therefore, such systems reduce the gulf of evaluation. But they do not eradicate it. There still remains the challenge of mapping the data to a decision to determine its impact (or utility [14]), its relative importance, and to evaluate alternatives to make an informed decision. As shown in Fig. 1, one attempt that has been made to finally close the gulf of evaluation is to generate model-based risk surfaces that attempt to model the impact of several data sources on decision outcomes and then present the output as an integrated visual goodness surface, say as a geospatial heatmap [4]. Jim Hansen and colleagues at the Naval Research Laboratory in Monterey, CA, for example, have generated automated piracy attack predictions that compute likelihood of piracy events across a region given the clemency of the weather for small boat (possible pirate predator) actions and the expected merchant shipping density (prey) [4]. Such analysis provides decision support to military planners because it relieves them of the need to mentally perform the complex mathematical derivation of the impact and relative weighting of these data sources. The risk predictions can be made available during military planning as another overlay on their digital geospatial data overlay systems.

Model-based risk surfaces exist mainly in the form of laboratory prototypes, or are in various stages of advanced development in a few target application domains. Their scope is usually no more than two to three relevant data sources. However, as Fig. 1 shows, as they grow in scope and sophistication, they point to an implicit future of decision support reduced to a choice on a geospatial objective function, where all sources have been mapped and optimally weighed and integrated into a single view. Such a system would finally bridge the decision maker’s gulf of evaluation as the decision maker would be able to reason over all mapped and appropriately weighed data to pick good locations.

3 User Challenges

There are a number of perceptual and cognitive issues raised by the use of (1) digital geospatial data overlay systems to perform geospatial decision making, potentially augmented with (2) model-based risk surfaces. Here, we enumerate the issues and use them to motivate the design of HCI features and functions that address these issues in the next section.

3.1 Digital Geospatial Data Overlay Systems

Relating multiple data sources.

Geospatial overlay systems provide no good way for users to relate multiple data sources together. Data overlays run into either perceptual or cognitive limitations, putting users in an awkward dilemma. If users attempt to look at data simultaneously, for example by turning on multiple overlays, then the clutter, occlusion and perceptual masking of data will result in slow and inaccurate identification [15]. This is true even if the system is augmented with the ability for users to control the opacity of each source, say with sliders. Such systems help mitigate data occlusions by employing transparency to blend layers together, affording users the chance to understand and relate data across layers and to the map background. However, with a lower opacity, each layer will be reduced in contrast, exacerbating clutter and masking issues that will quickly prevent interpretation of more than a couple of layers. Alternatively, if users view overlays sequentially, one at a time, to attempt to overcome the clutter and masking effects, then they discover another limitation of their cognitive architecture - their imperfect memory systems [16]. They will likely suffer slow and inaccurate identification of data from flawed recall and mental comparison of the different states of the display over time. These problems will intensify as more and more data sources inevitably come online.

Metarepresentational Competence.

Inherent in the skill-based problem solving required to perform complex geospatial decision making is the need for flexible representations that allow for creativity [17]. For this reason, and because of the variety and fluidity of the geospatial tasks in modern work domains, designers have opted to provide their expert users maps with configurable overlays because they would seem to provide that necessary flexibility [5]. That users need to configure their own decision support representations raises another subtle and often neglected issue. The approach is premised on the assumption that users are actually capable of the configuration task – that is, that they possess “metarepresentational competence” with the configurable, geospatial overlay tools and displays they are provided [18]. Users are expected to select overlays from the ever-expanding array of sources that the networked systems make available, and then combine and blend them with opacity sliders, to relate them interactively to meet their specific decision making needs. Users aren’t just expected to “finish the design” [19], they are implicitly expected to meet all their decision making requirements. Recent studies have highlighted the downsides of such flexibility by throwing into question the meta-representational competence of users with such systems. Expert users and novices alike underestimate the deleterious effects that clutter has on their visual performance from bringing up task-irrelevant overlays, and inadvertently slow themselves down when performing meteorological forecasting tasks [20, 21].

Determining impact of data.

As discussed in Sect. 2, above, there are several steps involved in making a geospatial decision. The impact of data on a decision, or its utility [14] needs to be determined and weighed. It is left up to the user to mentally determine impact, using their expertise, and then hold it in memory as they continue to relate data sources and their impacts together. Recently, we documented the extent of this problem in a controlled human performance experiment [22]. We measured the quality and time to make geospatial decisions with different support tools in a virtual fishing task (akin to the one relayed in Sect. 4, below, although much simpler). Participants caught 44% fewer fish and were 37% slower making decisions when they had to perform the task with three data overlays than with risk surfaces that mapped the data to decision impact (i.e., how sea temperature affects likelihood of fish vs. raw sea temperature). The results expose the time and mental effort required to interpret and relate data to decision quality from (only three) data overlays.

Contextual control.

The cognitive science of decision making has made dramatic strides in the last two decades. There are now sophisticated conceptualizations of decision making processes and strategy, and how they are deployed flexibly in response to contextual factors, such as time available to decide [14, 23, 24]. An overarching framework that captures the gross influences of shifting context on cognition is Hollnagel’s notion of contextual control modes [1, 2]. Data overlay systems may function acceptably when there is time to relate each data source to each other and to decision quality. But when data changes and users must determine whether and how to re-plan under time pressure, the limits of simple data overlay systems will manifest. Data overlay systems may not support user needs for certain contextual control modes.

3.2 Model-Based Risk Surfaces

Opaqueness.

Second, model-based risk surfaces are opaque to users in that they don’t show the basis of their predictions, and are thus subject to mistrust [25]. By presenting an integrated risk prediction without its pedigree or basis evident, or accessible, expert users may question its validity and then begin to mistrust and disuse it. In this regard, the integrated predictions don’t meet the requirements of military and civilian decision makers, who are taught and used to constantly validating assessments in underlying data (e.g., [26]).

Brittleness.

Model-based risk surfaces are attractive in that they create automatic impact predictions for several data sources. The results can be integrated into a geospatial heatmap and then provided to users as an overlay [4]. Such algorithmic approaches suffer from a key drawback, however, in that they are brittle [27]. That is, they are inevitably limited in what they take into account. Thus, if the context of a situation changes with additional data that it is not, or cannot be, included in the model, then the risk surface becomes less useful to decision makers who may not even realize anything is amiss. For example, recent local law enforcement actions or military exercises might affect the likelihood of piracy, but without access to these changes, they will not be factored into the model’s computations.

Color scales.

The outputs of risk models are often conveyed with continuous 2D color surfaces or heatmaps [4]. Although not inherent to risk models themselves, these outputs often use color scales that are not perceptually linear. That is, they possess misleading perceptual discontinuities at hue boundaries (that are not reflected in changes in the underlying risk predictions) and other artifacts [28].

4 ADVICE Geospatial Decision Support System

We have taken a different approach to enable users to reason about the decision impacts of multiple data sources. We have created a decision support prototype called ADVICE (Active Decision Visualization of Impact Critical Elements) and explored its efficacy for various geospatial tasks. ADVICE augments geospatial overlay systems with a user-generated, flexible integrated risk surface, and tools to explore it. In this section, we introduce and review ADVICE within a fictional scenario, where one must decide where to send vessels to catch fish. The data sources depicted are derived from actual geospatial data, but the specifics of how they relate to fishing is intended only as an illustration of the capabilities of ADVICE for other task domains.

4.1 Overview: Process and Principles

ADVICE is a support system for geospatial decisions, that is, any decision where locations, areas, or paths need to be compared. Here, we illustrate our current prototype supporting the search for promising locations to fish. ADVICE focuses on decisions for which there are multiple sources of data that inform and constrain possible actions (e.g. water depth, traffic, hazards). The data sources must generally be geospatial in nature, present in a geospatial overlay system, and mapped to a common coordinate space.

Figure 2 provides an overview of how ADVICE decision surfaces are generated. ADVICE essentially implements a multi-attribute utility model [14] for geospatial decisions. For a given decision, multiple data sources (Fig. 2, left) will constrain and change the desirability of locations. In ADVICE, users specify requirements and desires for each data source, either directly or imported from templates. Once these desires and requirements are specified, ADVICE automatically processes multiple data sources into the common space of decision impacts (Fig. 2, middle), and weighs impacts into a unified decision surface (Fig. 2, right). Impact and goodness are rendered in a perceptually linear color scale running from bright green (good) to dark red (bad) [29].

Fig. 2.
figure 2

ADVICE is a decision support module that augments data overlay systems (left) with impact mapping (center) to create an integrated decision surface visualization (right). (Color figure online)

Full size image

4.2 Impact Mapper

The foundation of ADVICE is mapping data sources (Fig. 3., left) to decision impacts with an interface. Impact mapping takes data, which may take a number of forms, and transforms it into decision impact (utility) over space (Fig. 3., right). This transformation is based on what data values are desirable for a decision. For example, data sources, such as water depth (bathymetry) or surface temperature, exist as or can be interpolated into data surfaces over space. Different values of this data have different desirability for the decision, that are mapped between -1 and 1 for soft constraints, or to -∞ for values that constitute violation of a hard requirement. If the fish we are seeking prefer surface temperatures at or above 65°F, say, and will not be found for temperatures below 60°F, locations with temperatures of 65°F or higher would be mapped to an impact of 1, interpolating down to impacts of -1 just above 60°F, and -∞ for locations with a measurement of 60°F or below. Similarly, to locate fish we would need to stay outside of outsize of hazardous dumping zones and the exclusive economic zones of other countries. The data in this case are areas defined by polygons of multiple points of latitude and longitude. Given that staying outside of these areas is a hard constraint, any point in these polygons would receive a utility of -∞, eliminating those locations from consideration, no matter how good other aspects of data may be. Data that exist as points (hazards or positions of noise-generating ships that are to be avoided), can be re-represented as a data surface consisting of the minimum distance to any of those points from each location in space. Decision impact are then calculated (must stay 500 yards away from hazards, after 1 mile the benefits level off). Even secondary data such as uncertainty about current measurements can be mapped in this way to impacts on a decision (lower uncertainty is better).

Fig. 3.
figure 3

ADVICE maps raw data values (left) onto their impacts for a decision (right).

Full size image

The final consideration in impact mapping is the relative weighting of the different data sources (surface temperature may be considered more important than depth, say). These weightings are normalized and used to weight relative impacts into the decision surface. This weighing impacts the combination of soft constraints only. Impacts and weightings that are themselves dynamic and change over time (e.g. forecasts weighted less heavily as they age) could easily be incorporated into the general ADVICE framework. More complex, multivariate relationships either between data sources, or external factors can also be captured with additional effort in the impact mapping process. By bringing decision impacts into the system, all the products and tools of ADVICE update with any updates to dynamic data, even alerting based on changes to impacts rather than raw data, something that would be very difficult for a human user to stay abreast of.

While this process may seem arduous, in practice we imagine users rarely needing to set or even adjust these impacts. Most impacts will be constant across different decisions. The preferred water depth might vary for different types of fish, which could be captured in decision templates for each fish species, but one will always want to stay out of fixed hazardous areas. Decision templates could specify most if not all the relevant impacts, and be modified and resaved as needed. While complicated functions mapping data onto impacts could be used, to date, we have found that the impacts for most data sources can be easily summarized.

4.3 Decision Surface, Location Comparer and Constraint Resolver

Once impact mappings and weightings are defined for each data source, ADVICE combines data sources into an overall decision surface showing ‘goodness’ of locations over space for the decision of interest. This combination is done though a weighted average of the impacts. The overall decision surface for our fishing scenario is shown on the right of Fig. 4., using the continuous, perceptually linear color scale discussed in Sect. 4.1. In the decision surface, and areas violating hard constraints are not colored, and the background map shows through. Critically, the decision surface can be probed to explore what is good or bad about locations and compare the raw data and impacts across multiple locations. The left of Fig. 4 shows a comparison interface for the locations labeled A and B on the decision surface. For each location, a data summary statistic over the area is provided, along with an indicator categorically color coding the utility of each source. Circles mark the best utility for each data constraint (both locations are marked in the case of ties). In this example, neither the depth at area A or area B is ideal (lighter, yellow dot), but depth is better at B (circle). Looking at the data summary statistics, Area B is farther from any navigation hazards, but they are both far enough there is no difference for the decision of where to search for fish. Additionally, attributes of the decision not related to data, such as the size of the proposed area, can be provided for comparison. By allowing users to explore the decision surface, we hope to both support more informed decisions and allow remediation of inherent risks (e.g. if the areas are a little deeper than one would like, one can adapt procedures to compensate). Further, we hope to support the creativity and skill-based problem solving of expert users [17] and foster overall trust in the system.

Fig. 4.
figure 4

Overall decision surface (top) and location comparer (bottom) showing the pros and cons of two decision locations. The yellow and green dots categorically show the goodness of each data source. The dots corresponding to the better decision based on a source are circled. (Color figure online)

Full size image

Considering many constraints can quickly limit possibilities. To support better understanding the relative impact of constraints, and intelligent problem solving in such situations, we developed the Constraint Resolver interface. The Constraint Resolver consists of two features. The first is an overlay view that shows only the number of hard constraints violated, giving a quick picture into what areas might be possible to open up by relaxing a small number of constraints. Second, we developed and implemented a simple “restrictiveness” metric characterizing how much a data source limits decisions. This metric is a measure of how much area is uniquely eliminated by a constraint. For each constraint, restrictiveness is calculated as the proportion of area meeting all other hard constraints, that a constraint eliminates. Once calculated, ranked constraints are presented to the user for analysis and follow up. Similar metrics can be defined to capture the influence of soft constraints (e.g. what proportion of an otherwise “good” area is “bad” after considering this constraint?).

4.4 User Challenges Addressed

With ADVICE, we have attempted to address many of the issues with geospatial data overlays and automated risk surfaces enumerated in Sect. 3. With ADVICE’s integrated decision surface, users do not need to hold multiple constraints in memory, or struggle to discriminate different data sources, as they do with data overlays. Nor do users need to mentally interpret data, or re-interpret changed data. These processes are offloaded to ADVICE. They are handled both by upfront mapping of the impact of data sources and by the integrated decision surface updating with the underlying data that feed it. As such, plan degradation is likely far easier with ADVICE, as users can stay abreast of how changed data impacts their decisions and plans in real-time. This also better supports the contextual control mode requirements of users by providing support to complex mental operations when users are the most pressed for time. Similarly, the work of setting up ADVICE can be performed ahead of time, when more time is available and users are in more optimizing context control modes [1, 2].

In comparison to risk surfaces, ADVICE gives users access to probe and modify the mapping of data source impacts on a decision, making it easier for users to understand and trust the resulting risk surface, which they have created themselves. Additionally, by providing additional exploration tools, ADVICE supports collaborative plan critiquing by allowing intelligent constraint relaxation to see the impact on decision outcome. As the users map the impact of data sources, they are aware of the scope of the surfaces they have created, and the resulting brittleness/resilience of what they have created. Finally, by rendering individual and weighted impacts in a perceptually linear color code, ADVICE’s visualization are free of misleading perceptual discontinuities.

5 Conclusions

We have reviewed trends in geospatial decision support toward more integrated geospatial information and identified several human factors issues and gaps with current geospatial decision support. The goal of HCI is to provide useful and effective task support for users through computer-based tools and interfaces. Too often, we have seen users of legacy systems, (figuratively) break into a sweat (Fig. 1, left), as their tools require of them mental gymnastics to make decisions. To further close the gulf of evaluation faced by geospatial decision makers, we developed and presented a prototype decision support system call ADVICE. Through explicit, user-determined mapping of data to decision impacts, ADVICE creates a decision surface and provided tools for the exploration of that surface. Note that the human decision maker is still an active part of the decision process, and that ADVICE does not take the decision from the decision maker. Rather it helps to externalize some of the interpretation of data. In Fig. 1, we see ADVICE as a redirection from the march towards a single unifying, model-based decision surface to something with more user interactivity. In this way human decision makers are still integrated into the decision process, and can still supplying additional contextualizing information that is not or cannot be explicitly represented in the system. Future users of a fielded ADVICE system should be able to focus on insights into complex problems and making quality decisions (Fig. 1, right).

Rather than visualize impact, per se, an alternative approach is to provide complex filtering and searching functionality to assist users determine specific locations that meet restrictive criteria (e.g., see [30]). This may be appropriate in tightly bound work domains where there are exclusively black and white hard constraints, and few opportunities to explore or relax constraints as with selection of nuclear waste disposal sites. In contrast, our end users in military command and control are faced with task domains that are not always bounded. Here, ADVICE provides interactive exploration and visualization of both hard and soft constraints.

While we have focused on a limited set of applications, we see the potential of ADVICE as a decision support framework to be much greater. The principles of helping users externalize utility functions to integrate multiple decision constraints could be readily extended beyond geospatial, to other forms of decision making. ADVICE could easily extend to consumer geospatial decisions such as searching for a house or rental (desiring it to be close to work, restaurants, and parks, with good schools). More generally, the same principles could apply to other situations where user-definable soft-filtering could provide better relevance weightings. For example, in web commerce, rather than a hard filter on 4 star or above ratings, shoppers could define explicit preferences (least 3 but more are better, lower price is better). Similarly, those shopping for a new car could select results not only on hard constraints, but on soft constraints relative to their desires (more MPG, lower price).