Keywords

Introduction

Visual Analytics tools in the recent years have made an impact in the criminal intelligence and analysis communities. Histories of user interactions known as Analytic Provenance have been used to advance our understanding of tool usage and user goals in a variety of areas. User interaction histories contain information about the sequence of choices that analysts make when exploring data or performing a task. To understand how the analyses are being made it requires support of correlating lower-level events during analysis process with upper level sub-tasks, tasks and goals of decision making process as proposed by Gotz and Zhou (2008).

Until recently, most of the research has focused on the techniques and methods for refining visual analytic tools, with the emphasis on empowering analysts to make discoveries faster and more accurately. Although this emphasis is relevant and necessary, we argue that the process through which an analyst arrives at the conclusion is just as important as the discoveries themselves. Understanding how an analyst performs a successful criminal investigation will finally let us start bridging the gap between the art of analysis and the science of analytics. We found out from the detection approach of behavioural marker from analytical data that they can bridge such gap alongside of performance measurement. The overarching aims of this research are based on following research questions to find out-.

RQ1: :

What are the constructs of behavioural markers for criminal intelligence

analysis?

RQ2: :

How to externalize analyst’s thinking processes from constructs of behavioural markers in criminal intelligence analysis?

This contribution is part of a research work aimed to find out appropriate methods or techniques to evaluate a visual analytic tool named as Analyst’s User Interface (AUI) of the project VALCRIFootnote 1 (Visual Analytics for Sensemaking in Criminal Intelligence Analysis). In section “Related Works” numbers of existing related work, in section “Development Approach of Behavioural Marker System” methodology overview to find out Behavioural Markers (BMs), their constructs and detection approaches have been presented. Section “Conclusion” includes conclusion and future work.

Related Works

Behavioural Marker systems are now being developed for performance measurement in a range of organizational settings, especially in high reliability industries such as air aviation, nuclear power, maritime transport, and medicine. They are usually structured into a set of categories (e.g. co-operation, decision making, and situational awareness). Normally, these categories are then sub-divided into more specific nontechnical skills or elements. The seminal research on behavioural markers comes from studies of civilian pilots carried out by Helmreich and colleagues at the University of Texas. In the late 1980s they developed a data collection form called the LINE/LOS Checklist (LLC) to gather information on flight crews’ crew resource management performance (Helmreich etal. 1990). This checklist has been used as the basis of many airlines’ behavioural marker systems (Flin and Martin 2001). Behavioural Markers (BMs) concept is not only used to measure team performance in aviation or medical sectors but also their uses for evaluating visualization are noticeable. North (2006) claims that the purpose of visualization is insight and to determine to what degree visualizations achieve this purpose. He listed some of the characteristics of insight such as – complex, deep, qualitative, unexpected and relevant. Saraiya etal. (2005) defined insight as an individual observation about the data, a unit of discovery. They presented several characteristics of insight while running a pilot study on biological and microarray data such as – observation, time, domain value, hypotheses, directed versus unexpected, breadth and depth, category. In a case study with the popular visual analytics application Jigsaw, Kang etal. (2009) found that analysts’ interaction histories showed evidence of the high-level sensemaking processes (Pirolli and Card 2005). Reda etal. (2014) approached interaction and sensemaking by combining interaction logs and user-reported mental processes into an extended log and modeling the log using transition diagrams to better understand the transition between mental and interaction states.

Development Approach of Behavioural Marker System

The typical method for the initial development of behavioural marker systems is to carry out a literature review of previous domain specific research concerned with nontechnical skills, followed by interviews with subject matter experts designed to extract the nontechnical skills required to do their job effectively (e.g. Fletcher etal. 2004; Mitchell and Flin 2009; Yule etal. 2006). We also carried out a systematic literature review by using several electronic databases (PsychINFO, ScienceDirect, Web of Science, Google Scholar, and the Defence Technical Information Center) to identify research articles with search terms: criminal intelligence, behavioural markers, human factors, situation awareness, decision making, intelligence analyst, cognitive skills etc. We considered cognitive attributes to present our phase-1 (Flin etal. 2008) behavioural markers found from literature review. We also arranged a workshop to discuss different concepts and extract related cognitive behaviours. There were about 30 criminal intelligence domain experts present in the workshop including ex-police, ex-intelligence analysts, researchers and other developers. The whole team was divided into several groups and then each concept was gone through one by one. Each person in the group said some words that they associated with the concept. We put them all on post-its and organized them thematically (i.e, an affinity diagram) at the end. Thus we formed an exhaustive list of behavioural markers for criminal intelligence analyst as shown in Table 10.1. Our aim was to identify a set of mostly relevant behavioural markers by considering human factors and cognitive engineering principles that underlie the design of user interface, visualization and interaction on criminal intelligence analysis system. The goal is to determine the extent to which imagination, insight, transparency and fluidity & rigour are enhanced on the assumption that improving these, will likely improve analysts’ ability to solve crime or be better at performing criminal intelligence analysis by using Analyst’s User Interface (AUI) of the project VALCRI (Wong etal. 2014).

Table 10.1 Constructs of observable behaviours in criminal intelligence analysis
Full size table

Detection Method

From a quantitative behavioural developmental theory perspective (Commons etal. 1998), behavioural constructs are events that have the potential to be directly observed. We have defined a set of behavioural markers into Table 10.1, and mainly look for their occurrence in the recorded analytic process data by considering the context of the situations that these behaviours were observed (i.e. before and after actions and conditions). Within such task environment in criminal intelligence, process data from the task interface allows for the collection of information that may be indicative of observable behaviours. So, the challenges underlies of converting such analytic process related data into behavioural markers. Within the intelligence analysis environment, process data from the task interface allows for the collection of information that may be indicative of behavioural markers. Such as – Fluency, specifically during the data finding process, can be defined as the ability to generate many different pieces of data. Fluency in data finding is the indicative of a behavioural marker known as “creativity”. Imagination can be considered in terms of creativity, and creativity in the literature can be approximated as ‘divergent thinking’, and researchers have attempted to measure divergent thinking through concepts such as ‘fluency in data finding’ or ‘flexibility unshifting between approach’ (Fontenot 1992). This concept of reducing complex construct into simpler, easier to measure constituent cognitive components can be conceivably applied to complex problem solving tasks. Such reductionist approach gives an overview of behavioural markers and their role for the scientists to recognize them when certain behaviours have occurred into analytic process data stream. Data reductions are accomplished through coding and manual interpretation during qualitative research approach, which is extremely labour intensive. Direct observation through video, physical observation, participant interview, audio recording are needed for this purpose.

Action Sequences Computation

The streams of actions during analytic process can be meaningful markers for complex behaviours. Current approaches such as – finite state systems for fixed manipulable elements, a priori establishment of fixed sequences for clearly defined tasks, exhausting all possible sequences for tasks with unpredictable human elements, are available for information computation about behavioural and cognitive processes and their implications for large scale complex analysis. The use of network graph visualization in this context can be a useful exploratory process, rather than exhaustive, to observe and gain understanding which empirical action combinations may provide meaningful sequence for targeted behavioural marker. The sequences need to be converted into a structure that is more suitable for network analysis and visualization. Some sequences might be observed more often while others are only observed in very rare occasions. Low Level Action sequence Seq . #001 A → B → D → E → G as shown in Fig. 10.1, comprises of analytic states A, B, D, E, G are different analytic states after low level actions have been applied on. As we aim to follow a compositionally reductive framework for the contextual information of complex analytic states, we can denote each of them as semantic state composition function P(S) where S is an analytical state.

Fig. 10.1
figure 1

An analytic path showing annotations set by analysts with captured states & their relationships based on interactions with colour coded users (analysts) information. States can be selected from States Panel & RRP list of Analyst’s User Interface (AUI) to load analytic path for understanding intersections of analytical states captured by different analysts during their analysis process (Islam etal. 2016)

Full size image
$$ \mathrm{So},P(S)=S. $$

For Seq . #001, it can expressed as –

$$ P\left({S}_A\right)={S}_A\vspace*{-12pt} $$
$$ P\left({S}_B\right)={S}_B\vspace*{-12pt} $$
$$ P\left({S}_D\right)={S}_D\vspace*{-12pt} $$
$$ \dots \kern0.36em \dots \kern0.36em \dots \kern0.36em \dots\vspace*{-12pt} $$
$$ P\left({S}_n\right)={S}_n,\mathrm{where}\ n\ \mathrm{is}\ \mathrm{the}\ \mathrm{number}\ \mathrm{of}\ \mathrm{nodes}. $$

Thus we computed n th state S n as P : S A, B, D, …, n − 1 → S n. Composition function of different analytic states can be expressed as –

$$ P\left({S}_A\right)\;\mathrm{o}\;P\left({S}_B\right)=P\ \mathrm{o}\ P\left({S}_A,{S}_B\right)=\left\{{S}_A,{S}_B\right\}={S}_{A,B}\ P:{S}_A\to {S}_B \vspace*{-17pt} $$
$$ P\left({S}_B\right)\ \mathrm{o}\;P\left({S}_D\right)=P\ \mathrm{o}\ P\left({S}_B,{S}_D\right)=\left\{{S}_B,{S}_D\right\}={S}_{B,D}\ P:{S}_B\to {S}_D \vspace*{-17pt}$$
$$ \dots \kern0.36em \dots \kern0.36em \dots \kern0.36em \dots \kern0.36em \dots \kern0.36em \dots \kern0.36em \dots\vspace*{-17pt} $$
$$ P\;\mathrm{o}\;P\left({S}_{A,B,D,\dots, n-1},{S}_n\right)=\left\{{S}_A,{S}_B,\dots, {S}_n\right\} $$

P : S A, B, D, …, n − 1 → S n = S ST , where S ST is a Sub-Task State (Gotz and Zhou 2008) through low level actions or events.

This is how other low level action sequences Seq . #002, Seq . #003,…, …, …, Seq . # N can be computed.

To determine which sequences are more valid measures of ‘Behavioural Markers’, we consider attributes of Table 10.1 and this would entail some form of network analysis; so each low level actions (representing an analytic state) can be defined as a ‘node’ and the links that make up a sequence across the nodes can be defined as ‘edges’. Eigenvector centrality is one method of computing the “centrality”, or approximate importance, of each node in a graph network. The assumption is that each node’s centrality is the sum of the centrality values of the nodes that it is connected to. The adjacency and centrality matrices for the action sequence graph as shown in Fig. 10.1 have been computed. The centrality matrix is an eigenvector of the adjacency matrix such that all of its elements are positive. While nodes with higher importance and associated edges indicate that they are taken more often, and therefore may imply that the analysts are finding more sensible choices for shifting from one approach to another (Flexibility) or generating more alternative approaches (Fluency). Creativity is manifested through the flexibility, fluency and originality of responses to a task (Torrance 1988) which can be approximated as ‘divergent thinking’ or alternately “Imagination”.

Conclusion

This research aims to explain how human cognition leads to interactions and vice versa to achieve certain goal. The identified behavioural markers (Table 10.1) are aimed to use as attributes for performance measurement of an Analyst’s User Interface (AUI) for the project VALCRI.1 One of the requirements from a focus group during our previous study (Islam etal. 2016) with the end-users (Police Analysts) was to capture analyst’s thinking processes during their analysis. It is difficult to recover such thinking processes by using extended analytical provenance log or only by observing. For example, knowing when one reasoning process ends and another begins may be unclear from a sequence of interaction alone. In our previous research we proposed a captured logical state composition approach and their grouping arrangement (Fig. 10.1) as the solution to cognitive steps sequencing problem along with analytic data. In this research work we have aimed to couple these cognitive steps with analytic data. Endert etal. (2015) contend that a new methodology to couple the cognitive and computational components of visual analytic system is necessary. We have proposed markers of behaviours as attributes for coupling human cognition and analytic computation through interactions. Our eigenvector centrality computation approach by using adjacency matrix of different captured analytic states through low level interactions provides a simple solution of overcoming tedious effort of qualitative approach to detect behavioural markers from sequential actions into analytic provenance dataset.

As for our future work we also aim to conduct an in-depth evaluation study with our end-users to investigate how transitions among behavioural markers can be detected as well as their influences on analytical activities. Analysis of combinations of such behavioural markers that occur during large complex task also introduces research challenges of predictive analytic goal oriented recommendation for action sequences. The inverse compositional reductionist approach can unfold the process of analysis being carried out to reach a goal. But how can such approach be applied on actual working environment, still requires further research.