Advertisement

Development of a behavioural marker system for incident command in the UK fire and rescue service: THINCS

  • Philip C. Butler
  • Robert C. HoneyEmail author
  • Sabrina R. Cohen-Hatton
Open Access
Original Paper
  • 81 Downloads

Abstract

Non-technical skills in high-reliability industries include decision making, interpersonal communication and personal resilience. It has been argued that these skills are key to effective performance, safety and welfare. However, there is no agreed upon set of such skills or formal means to identify and assess them for incident commanders in the UK fire and rescue service (FRS). The aim of this research was to identify the non-technical skills that underpin effective incident command in the UK FRS, and to develop a behavioural marker system that captures these skills. Our research assessed the current range of training offered in non-technical skills across the UK FRS, and identified the non-technical skills of incident command within the UK FRS through interviews with incident commanders, and workshops with subject-matter experts. The six non-technical skills were: assertive, effective and safe leadership; effective decision making and planning; interpersonal communication; personal resilience; situational awareness; and teamwork and interoperability. A bespoke behavioural marker system, called THINCS, was developed with exemplary behaviours for each skill.

Keywords

Non-technical skills Behavioural marker system Incident command 

1 Introduction

High-reliability industries distinguish between a person’s technical and non-technical skills: Technical skills include knowledge of procedures, hazards and apparatus, whereas non-technical skills include decision making, interpersonal communication and personal resilience. Historically, the UK fire and rescue service (FRS) has emphasised the importance of technical skills. However, incident commanders within the FRS have a multi-faceted role, which requires them to have both technical and non-technical skills (National Operational Guidance Programme 2015). The non-technical skills impact on their ability to work in challenging environments, which are often dangerous and dynamic as well as being emotionally charged and stressful (Health and Safety Executive 2010). The role combines decision making with a variety of other non-technical skills (Klein 2008). As in other high-reliability industries (e.g., aviation, marine, oil and gas exploration, and healthcare), investigations of high-profile incidents involving the deaths of members of the public and/or firefighters have implicated deficiencies in the non-technical skills of incident commanders in the UK FRS (see Torrie 2012; GMFRS 2016, pp. 4–5; Watterson 2015). The National Fire Chiefs Council1 now emphasises the importance of training non-technical skills because ‘they enhance the ability of a Commander to effectively lead and control an incident.’ (CFOA 2015, p. 9); and the UK FRS National Operational Guidance Programme (2016) stressed how incident commanders should utilise non-technical skills to ‘…underpin their judgements, decisions and behaviours’ (p. 7). However, there has been no systematic assessment of the non-technical skills that underpin effective incident command, with the result that the UK FRS has had no consistent means to assess them.

1.1 Non-technical skills and accident causation

Human error is a major causal factor in accidents at work (see Flin et al. 2008; Health and Safety Executive 1999; Reason 1997; Shappell and Weigmann 1997). Reason (1990) defined human error as ‘…planned actions that fail to achieve their desired consequences without the intervention of some chance or unforeseeable agency’ (p. 17). Human errors include slips (of attention) and lapses (of memory) that affect skilled performance, and mistakes that are associated with a lack of knowledge or applying the wrong rule to a situation (see Rasmussen 1982). One leading model of accident causation proposes that a series of defensive barriers (e.g., effective training) lie between a hazard and an accident, and that an accident occurs when there are defects (holes) in these barriers that align and allow the hazard to result in an accident (Reason 1997). Reason (2008) argued that non-technical skills are one defensive barrier. One source of evidence that is consistent with the view that high levels of non-technical skills are related to low levels of error comes from a study that investigated indices of surgical excellence and behavioural markers of non-technical skills. The performance of 16 surgeons was rated, and the 4 who made the least errors and exhibited high levels of safety awareness received high ratings for their non-technical skills (Carthey et al. 2003). While this evidence involves a relatively small sample, and the causal nature of the observed relationship cannot be determined, the results suggest that non-technical skills might reduce error and increase safety.

1.2 Non-technical skills in high-reliability industries

Identifying non-technical skills within high-reliability industries, and developing behavioural markers systems to rate them is recognised as an important component of safety management. As already noted, deficiencies in non-technical skills have been implicated, by accident investigations and/or public inquiries, in accidents in many high-reliability industries (see Weick 1990; Air Accident Investigation Bureau 1990; Marine Casualties Investigative Body 2012; Crichton et al. 2005; Health Commission 2009). Retrospective analyses of aircraft accidents, in particular, have suggested that there was a need to provide training that would support the development of non-technical skills (e.g., Flin et al. 2008). This became known as crew resource management (CRM) training (Helmreich et al. 1999). The efficacy of this training can be assessed using a behavioural marker system that consists of a set of observable domain-specific behaviours that relate to the set of non-technical skills (Klampfer et al. 2001) together with a rating system for those skills (Flin and Martin 2001). This approach has also been adopted for surgeons, anaesthetists, ships officers and nuclear power reactor engineers (see Devitt and Holford 2010; Fletcher et al. 2004; O’Connor et al. 2008; Yule et al. 2006). There is also some evidence that training in non-technical skills reduces error in a healthcare setting. Using a before and after design, in a single group of participants, non-technical skills training improved these skills and this improvement was accompanied by a reduction in the number of errors (McCulloch et al. 2009; see, for a review; Kodate et al. 2012).

1.3 The UK FRS context

The performance of incident commanders in the UK FRS is influenced by many factors, which are summarized in Fig. 1. These factors include incident characteristics, individual characteristics and the nature of the command team. The role of incident commander clearly involves a broad range of non-technical skills, which have been highlighted by national bodies. For example, The Department for Communities and Local Government (2013, p. 32) published a health and safety framework specifically for the operational environment of the UK FRS which stated that: “Fire and Rescue Authorities should consider the impact of ‘human factors’ on the safe, effective and timely resolution of an incident. This includes…human and individual characteristics that influence the behaviour of teams and individuals. Understanding these ‘human factors’ is critical to effective health, safety and welfare management.” Guidance published by the National Operational Guidance Programme (2016) identifies non-technical (command) skills, which overlap with those from other safety–critical settings, and have been linked to the high-profile incidents involving the UK FRS noted above. However, there has been no systematic assessment of the non-technical skills that underpin effective incident command and, consequently, there is no behavioural marker system for incident command in the UK FRS. The development of such a system would enable the National Fire Chiefs Council (who are the professional voice of the fire and rescue service) to meet some of the challenges that are faced in relation to incident command, including improving the selection, assessment and development of incident commanders. We first conducted a survey of the UK FRS that assessed the current range of training in non-technical skills. The results of this survey provided a context for the main aim of our research, which was to identify the non-technical skills of incident command within the UK FRS. This was achieved through interviews with incident commanders, and workshops involving subject-matter experts. A behavioural marker system was then developed to enable these skills to be evaluated. Finally, we conducted a preliminary evaluation of the system.

Fig. 1

Factors that influence the performance of incident commanders

(adapted from Youngson 2016)

2 Materials and methods

2.1 Study design

We conducted an online survey of UK FRS incident command training centre managers to get a snapshot of non-technical skills training across the UK FRS. The survey used the generic descriptors for non-technical skills identified for other high-reliability industries (i.e., decision making, communication, leadership, situational awareness, stress and fatigue management, and teamwork; Flin et al. 2008). The results of the survey provide a clear impetus and context for the development of a bespoke behavioural marker system for the UK FRS, and identifies gaps and inconsistencies in training provision, which the new system has the potential to address.

The development of the behavioural marker system (THINCS: THe INcident Command Skills of the UK FRS) was based primarily on (1) semi-structured interviews with incident commanders, and (2) a series of subject-matter expert workshops. In particular, the transcribed content of the interviews was analysed using thematic analysis (Braun and Clarke 2006; see also; Roberts et al. 2015). This is a structured method for identifying, analysing and reporting themes; and the process followed a theory-driven approach to examine the presence and nature of the generic non-technical skills and sub skills identified by Flin et al. (2008). This approach identified a broad range of non-technical skills associated with incident command, while the workshops refined these skills and agreed a set of behavioural markers for them. This is an example of the use of the development rationale described by Greene, Caracelli, and Graham (1989) in which the results gained through the use of one method (semi-structured interviews) provided the basis for and continuity with the next method (subject-matter expert workshops; see also, Bryman 2006). The use of subject-matter experts allowed us to draw on their domain knowledge, and to characterize patterns of behaviour that are indicative of each non-technical skill (for a review, see Klein 2008). More specifically, we recruited a subject-matter expert from each level of command to increase the applicability of the system across different levels of command. Our design mirrors the approach used in the context of other high-reliability industries (see Crichton and Flin 2004; Fletcher et al. 2004; Mitchell et al. 2011; O’Connor and Long 2011; Roberts et al. 2015). We also conducted a preliminary evaluation of THINCS to assess its utility and to inform the development of an associated app.

2.2 Participants

2.2.1 Online survey of non-technical skills training

The survey was sent to 48 of the 53 local authority UK FRS incident command training centre managers; there were no contact details for the remaining 5 UK FRSs. Although no demographic data were sought about individual training centre managers, the typical role of an incident command training centre manager within the UK FRS is that of a middle manager (i.e., station-to-area manager). They are responsible for organising incident command training for continuous professional development as well as incident command assessment centres. 27 (56%) of the training centre managers responded in full to the survey. Every nation of the UK was represented (except Northern Ireland), as were all 11 National Fire Chief Council regions, including 6 of the 7 larger, metropolitan FRSs in England. The majority (74%) of the FRSs who responded were county-based. Of those who did not complete the survey in full, 10 (21%) did not respond, 7 (15%) partially completed it, 3 (6%) did not consent to participate, and 1 (2%) was unable to complete the survey due to IT difficulties.

2.2.2 Semi-structured interviews with incident commanders

20 incident commanders (17 Level 1 and 3 Level 2 commanders) participated. Level 1 incident commanders are usually the first Fire and Rescue staff on scene at an incident, and Level 2 commanders take charge of medium and large incidents requiring greater command and control (National Operational Guidance Programme 2015). At the time of their interviews, all the participants were serving incident commanders, either as a Station, Watch or Crew Manager.

2.2.3 Subject-matter expert workshops

The selection of the 5 subject-matter experts who contributed to 7 workshops was based on the following criteria: one representative from each level of command; all operationally active incident commanders; and either a member of the National Operational Guidance Programme project to revise and develop UK FRS incident command policy, or, an incident command training manager, or, a member of the National Fire Chiefs Council’s National Command and Control Working Group. The subject-matter experts were: an Assistant Chief Fire Officer (Level 4 commander), a Deputy Assistant Commissioner (Level 3 and 4 commander), a Group Manager (Level 2 commander), a Station Manager and a Watch Manager. Level 3 incident commanders take charge of very large and complex incidents and Level 4 incident commanders represent their organisations at Strategic Co-ordinating Groups or, if in Scotland, Regional Resilience Partnerships (National Operational Guidance Programme 2015). One of these experts was unavailable to participate in the development of the behavioural marker system and was replaced with two further subject-matter experts (a Group Manager and Watch Manager) for the final 2 workshops. This research was approved by the School of Psychology Ethics Committee, and participants gave informed consent for their participation.

2.3 Procedure

2.3.1 Online survey of non-technical skills training

The survey of command training centre managers used Qualtrics software (Qualtrics 2016) and was designed to reveal the extent to which non-technical skills training was evident across the UK FRS. The survey requested information about generic non-technical skills training based on other high-reliability industries (i.e., decision making, communication, leadership, situational awareness, stress and fatigue management, and teamwork; Flin et al. 2008). They were asked to indicate whether they provided training for each of the non-technical skills, whether the training was theoretical and/or practical, and which level(s) of incident commanders received the training. The survey contained six sets of identical questions that focused on each non-technical skill (e.g., ‘Do you deliver practical training for practicing…[non-technical skill]…?’). The participants were given 3 weeks to complete the survey, during which time they received regular reminders. The survey was designed to allow respondents to complete it over a number of sessions. However, if more than 1 week passed without the survey being accessed it was terminated and the partial response collected. The full list of questions is available from the authors on request.

2.3.2 Semi-structured interviews with incident commanders

The interviews were conducted as part of an earlier study in collaboration with the National Operational Guidance Programme that reviewed incident command. The study investigated UK FRS incident commander decision making, which involved placing helmet cameras on incident commanders at incidents and interviewing them using their footage afterwards (Cohen-Hatton et al. 2015). The incident commanders who volunteered to participate were also interviewed about non-technical skills. Each interview lasted approximately 1.5 h and was conducted in a private office at the fire station of the serving officer. Participants were asked open-ended questions about organisational culture and the basic set of non-technical skills that were identified in the summary provided by Flin et al. (2008) as being shared across a range of high-reliability industries. Probe questions were used to explore the personal, social and cognitive skills referenced by the incident commanders to gain an insight into their significance and influence upon incident command (e.g., ‘What makes a good team player?’). The probing question technique was adopted from the Critical Decision Method (Klein et al. 1989). The full list of probe questions is presented in the “Appendix”. The interview transcripts were uploaded onto NVivo 10 software (QSR International 2014), which was used to analyse them line by line to identify references to non-technical skills and collate related references into themes.

2.3.3 Subject-matter expert workshops

The themes drawn from the semi-structured interviews were used to inform the activities undertaken in a series of subject-matter expert workshops. The results of the online survey were also available to the experts. The seven workshops took place in classrooms or conference rooms at FRS fire stations/training venues. To identify a set of non-technical skills, the subject-matter experts were presented with the findings from the literature review, online survey, and the non-technical skills themes derived from the semi-structured interview transcripts. The research findings remained available for review throughout the workshops as points of reference if required. To effectively and efficiently review and refine these types of data, grid and sorting exercises were used. The subject-matter experts were tasked with categorising the non-technical themes into either personal, cognitive or social skills by placing them onto a grid subdivided into those areas. For example, pattern matching and decision making were categorised as cognitive skills; caring and leadership as personal skills; and negotiation and trust as social skills. A sorting task was used to refine the themes based on relationships between them (cf. Fletcher et al. 2004; see also; Rugg and McGeorge 1997).

The subject-matter experts also reviewed the six emergent non-technical skills and their main and component themes, and refined them based on design criteria adapted from those used to develop non-technical skills for pilots (Flin et al. 2003). These criteria were that the skills must be: observable, discrete, and hierarchical; defined using industry terminology; and follow the principle of parsimony to retain the minimum number of non-technical skills necessary. This process involved three stages. First, the experts discussed all of the themes grouped around an emergent non-technical skill and considered if they were observable and how frequently they would be demonstrated by an incident commander. This discussion isolated or reconfigured some of the main and component themes. Second, they defined the component themes in terms of a skill and subsumed them into the definitions of the main themes. Finally, the emergent non-technical skill was defined and labelled a ‘command skill’ with its main themes labelled ‘sub skills’ in accordance with commonly understood UK FRS terminology. This process was repeated for each of the emergent non-technical skills and resulted in a prototype set of UK FRS command skills.

There were four further subject-matter expert workshops with four experts from the first workshops and two additional experts, which ensured all levels of command were represented. The first of these workshops involved the subject-matter experts discussing and identifying specific, observable behaviours indicative of good and poor performance for the sub skills of each command skill. The identified behaviours were reviewed and refined through further discussion into a set of exemplary behaviours for each sub skill. These behaviours were described as action statements that either directly or indirectly denoted the presence of a skill (e.g., an order indicating a type of decision making; Cohen-Hatton et al. 2015; Cohen-Hatton and Honey 2015; Fletcher et al. 2004). The resulting behavioural maker system was further refined by cross-checking its content with UK FRS national occupational standards (Skills for Justice 2013a, b, 2014) and guidance for incident command to ensure that it was comprehensive. The workshops also developed a rating scale. A five point, quality-based Likert scale was selected, where a score of 4 (good) represented consistently high performance of the skill and 0 (unobserved) indicated that a skill should have been demonstrated but was omitted and so represented a significant risk to others. An additional rating of ‘not observed’ was incorporated to record when a skill was not relevant to the situation under observation. The subject-matter experts agreed that an individual’s command skill rating would be the mean of its sub skill ratings (excluding ‘not observed’ ratings).

3 Results

3.1 Online survey of non-technical skills training

The online survey provided a snapshot of the nature and distribution of training in generic non-technical skills: decision making, situational awareness, leadership, teamwork, communication, and personal resilience (cf. Flin et al. 2008). Figure 2 depicts the number of UK fire and rescue services from the online survey that provided training in these non-technical skills. Inspection of the figure shows that some form of decision making training (theoretical or practical) was provided by all of the 27 UK FRSs. However, there was less consistency in the provision of training in the remaining non-technical skills: with 25 providing training in situational awareness, 19 in leadership, 19 in communication, 20 in teamwork, 14 in personal resilience (i.e., stress and fatigue management; and only 9 providing practical training).

Fig. 2

The number of UK FRSs that provided training for each non-technical skill, and of those, the numbers who used theoretical and/or practical training methods

The survey provided further details about the delivery of theoretical and practical training in non-technical skills. For example, 26 of the 27 UK FRSs (i.e., 96%) who provided theoretical training in decision making used the ‘Foundation for Incident Command’ (National Operational Guidance Programme 2015). This document also formed the basis of theoretical training for each of the remaining skills: 20 of the 22 UK FRSs (91%) who provided theoretical training for situational awareness; 15 of the 17 for leadership (88%); 15 of the 16 for teamwork (94%); 16 of the 18 for communication (89%); and 14 of the 14 for resilience (100%). In terms of practical training, live exercises were the most frequently endorsed: 25 of 25 for decision making (100%); 20 of 22 for situational awareness (91%); 14 of 14 for leadership (100%); 17 of 18 for teamwork (94%); 14 of 15 for communication (93%); and 8 of 9 for resilience (89%). An additional noteworthy finding from the survey was that while a large number of those surveyed reported providing training in non-technical skills for Level 1 Incident Commanders this was not the case for Level 4 Incident Commanders. For example, 25 of the 27 UK FRSs (i.e., 93%) provided theoretical training in decision making for Level 1 Incident Commanders based on the ‘Foundation for Incident Command’ (National Operational Guidance Programme 2015), only 7 of the 27 (26%) provided this training for Level 4 Incident Commanders. Similarly, while all 25 UK FRSs (i.e., 100%) who provided practical training for decision making using live exercises did so for Level 1 Incident Commanders, only 6 (24%) did so for Level 4 Incident Commanders. This difference between Level 1 and Level 4 commanders was equally marked across (1) all of the non-technical skills that were surveyed, (2) theoretical and practical training, and (3) all sources of potential training. The amount of training provided to Level 2 and Level 3 commanders fell between that given to Level 1 and Level 4 commanders (further details available on request).

3.2 Semi-structured interviews with incident commanders

The transcribed content of the interviews was analysed using thematic analysis (Braun and Clarke 2006; see also; Roberts et al. 2015), which is a structured method for identifying, analysing and reporting themes. A theory-driven approach was employed to examine the presence and nature of the non-technical skills and sub skills (cf. Flin et al. 2008). Thematic coding produced 59 different non-technical skills-related themes from ‘adaptable’ to ‘working relationships’ (see Table 1). The themes consisted of a set of references to cognitive skills, attributes, and knowledge. The five themes that were referenced most often by incident commanders across all of the interview transcripts were: leadership (211), pressure (95), stress management (89), organisational culture (87) and decision making (69).

Table 1

Themes derived from the interview transcripts

Adaptable

Dissonance

Listening

Red tape

Analytical

Empathy

Memory

Resource control

Assertive

Empowering

Mental check list

Respect

Attention

Experience

Motivation

Responsibility

Briefing

Fairness

Negotiation

Risk management

Caring

Fatigue

Organisational culture

Safety

Communication

Fatigue management

Pattern matching

Situational awareness

Competent

Identify

People-oriented

Stress management

Confidence

Information

Perception

Teamwork

Consistency

Inspire

Planning

Technical support

Consultation

Interpersonal skills

Presence

Thinking time

Cooperation

Judgement

Pressure

Trust

Debrief

Knowledge

Priorities

Understanding

Decision making

Leadership

Problem solving

Working relationships

Delegation

Learning

Questioning

 

3.3 Subject-matter expert workshops

The panel of experts were free to accept or reject the 59 themes identified from the analysis of the interviews. This resulted in the rejection of one theme ‘red tape’ as they did not regard dealing with it as a non-technical skill. They refined the remaining 58 themes based on relationships between them into 20 main themes such as ‘planning’, which included a number of component themes such as risk management, priorities and resource control. The subject-matter experts could add a non-technical related item they considered necessary based on their knowledge and experience. This added 12 items to seven of the 20 main themes: decision making method and decision evaluation were added to the ‘decision making’ theme; communication style and method were added to the ‘communication’ theme; setting standards and expectations were included in the ‘consistency’ theme; staff development was added to the ‘motivation’ theme; anticipation and expectation were included in the situational awareness theme; confidence and self-awareness were added to the ‘resilience’ theme; and skills matching to the ‘teamwork’ theme. The subject-matter experts grouped the 20 main themes around six emergent non-technical skills: communication, decision making, leadership, personal resilience, situational awareness and teamwork. The emergent ‘leadership’ non-technical skill comprised the main themes of consistency, leadership, motivation, safety and values; ‘decision making’ comprised the main themes of adaptable, decision making and planning; and ‘communication’ comprised communication, briefing and interpersonal skills; ‘personal resilience’ comprised the main themes fatigue, pressure and stress; ‘situational awareness’ comprised attention, experience, information gathering and situational awareness; and ‘teamwork’ comprised team dynamics and teamwork. Further refinement by the subject-matter experts determined a set of command skills and their associated sub skills (Table 2), and identified the exemplary behaviours for each one, which formed the THINCS system. For example, Table 3 shows the good and poor practice behavioural markers for the sub skill ‘thinking time’ of the ‘personal resilience’ command skill.

Table 2

Prototype UK FRS command skills

Command skill

Sub skills

Assertive, effective and safe leadership

Setting and maintaining standards of performance

Values and supports others

Leadership style

Competence

Safety leadership

Effective decision making and planning

Intuitive decision making

Analytical decision making

Planning

Interpersonal communication

Listening

Communication style

Briefing

Personal resilience

Thinking time

Stress and fatigue management

Confidence

Situational awareness

Information gathering

Understanding information

Anticipating incident developments

Teamwork and interoperability

Cooperation

Team formation

People-oriented

Table 3

Exemplary behavioural markers for the sub skill ‘thinking time’ from the ‘personal resilience’ command skill of the THINCS system

Behavioural markers

Good practice

Poor practice

Requests ‘quiet’ at the command location

Fails to control the distractions at the command location

Creates a barrier, when necessary, to distractions, or physically removes self away from them

Fails to control others wishing to communicate, jeopardising the successful transfer of critical information

Allocates appropriate command tasks to others to create time to think

Becomes overloaded as fails to delegate roles and work appropriately

Maintains appropriate spans of control to effectively manage workload

Becomes distracted and fails to respond to critical information and act in a timely manner

4 Preliminary evaluation

To provide a preliminary assessment of the useability and the capacity of THINCS to discriminate between incident commanders, four raters independently scored the non-technical command skills in six incident commanders. The recordings used were of incident commanders responding to real incidents (IC 1, IC 2 and IC 6), live exercises (IC 3 and IC 4), and a computer-simulated exercise (IC 5). Two of the raters had been involved in the development of the system and 2 were naïve to the system. Each rater completed a questionnaire at the conclusion of the evaluation. The questionnaire focused on how comprehensive the command skills and sub skills were, the behavioural markers, and the overall utility of the system. The raters were required to either rate an experience on a scale of 1–5 (1 = very difficult to 5 = very easy), or respond to series of questions with yes/no answers to represent their views.

Figure 3 shows the mean ratings for each command skill for the six incident commanders; with each of score for a command skill being based on a mean of scores for between three and five sub skills. Inspection of the figure indicates that the use of THINCS by the four raters identified consistent performance differences between the 6 commanders. Notably, one incident commander (IC 2) consistently received higher ratings across the command skills than the remaining incident commanders; and two incident commanders (IC 1 and IC 3) received consistently lower ratings than the remaining incident commanders. ANOVA conducted on the scores from the four raters, with the within-subjects variables of incident commander (IC 1–IC 6) and command skill as factors, revealed an effect of incident commander (F(5, 15) = 6.78, p < 0.005, np2 = 0.69), no effect of command skill (F < 1), and no interaction between these factors (F(25, 75) = 1.12, p = 0.34, np2 = 0.27); and post hoc tests with a Bonferroni correction revealed a significant difference between IC 1 and IC 2 (p < 0.05). The fact that IC 1 and IC 2 differed significantly represents evidence that the raters were scoring them as consistently different from one another. We also conducted an analysis of inter-rater reliability on the mean scores pooled across all command skills, which were rounded to the closest whole numbers. This analysis revealed a Cronbach’s α = 0.87.

Fig. 3

Mean ratings (+ SEM) for each non-technical command skill for six incident commanders (IC 1–IC 6)

The questionnaire given to the raters confirmed that they thought that the system was comprehensive. On a scale of 1–5 (1 = very difficult and 5 = very easy), the raters reported that it was straightforward to link the skills to observed behaviours (their ratings were: 4, 4, 4 and 3); and they indicated that the documentation used to capture observations of, and feedback to, incident commanders was useful (their ratings were: 4, 3, 3 and 3).

5 Discussion

Deficiencies in non-technical skills have been linked to accidents in several high-reliability industries (e.g., Crichton et al. 2005; Flin et al. 2008; Helmreich et al. 1999; Weick 1990). The UK FRS has acknowledged the importance of incident commanders possessing an appropriate set of non-technical skills. However, there is no agreed upon set of non-technical skills for the UK FRS, and there is no behavioural marker system to assess them. A national survey of all UK FRSs established that the provision of training in (generic) non-technical skills is inconsistent: For example, while 25/27 of the respondents from the UK FRSs provided theoretical training in decision making for Level 1 Incident Commanders using the Foundation for Incident Command (National Operational Guidance Programme 2015), only 14/27 provided training in personal resilience. There was clearly an unmet need to establish an agreed upon set of context-appropriate non-technical skills for incident command. Thematic analysis of semi-structured interviews with incident commanders resulted in the identification of 59 themes, and workshops involving subject-matter experts refined these into six non-technical command skills (with associated sub skills). The six non-technical skills were: assertive, effective and safe leadership; effective decision making and planning; interpersonal communication; personal resilience; situational awareness; and teamwork and interoperability. On the basis of these skills, a behavioural marker system, called THINCS, was developed with exemplary behaviours for each skill and sub skill.

We attempted to ensure that the system was credible by basing it on converging evidence from incident command training managers, interviews with incident commanders, and subject-matter experts (Bryman 2012). The use of subject-matter experts at different levels of command was especially important in identifying behavioural markers for good and poor practice associated with the command skills used by incident commanders. The system was also informed by current UK FRS incident command guidance and national occupational standards. This overarching approach helped to ensure that the research represented different perspectives from the stakeholders and enabled the research to act as a potential catalyst for change (Guba and Lincoln 1994). In fact, the observation that training provision in generic non-technical skills was very uneven across different fire and rescue services (see Fig. 2) and levels of incident command, already provides one such catalyst.

A comparison of THINCS with other behavioural marker systems that have been developed for surgeons (NOTSS: non-technical skills for surgeons; Yule et al. 2006) and European pilots (NOTECHS: non-technical skills for pilots; Flin et al. 2003) reveals that there are similarities and differences in the non-technical skills across these settings. The THINCS system consists of six non-technical skills and 20 sub skills, which is a higher number than other systems. For example, the NOTSS system developed for surgeons and the NOTECHS system for European pilots both consist of four non-technical skills with 12 and 14 sub skills, respectively. THINCS shares non-technical skills with the other systems, such as leadership, decision making, situational awareness, and teamwork. For example, some of the sub skills involving situational awareness are similar across different domains (for a review, see Flin et al. 2008); acknowledging that there are differing views about the nature and utility of situational awareness as a theoretical construct (Carsten and Vanderhaegen 2015). Yet the working environment of incident commanders is very different from that of pilots and surgeons, and the sub skills that underpin the remaining non-technical skills confirms the domain specificity of the three systems. Table 4 provides a direct comparison of the sub skills for the shared non-technical skills from the THINCS, NOTECHS and NOTSS systems. Inspection of Table 4 confirms that while there is overlap between the sub skills, there are significant differences in the terminology used, emphasis and their nature.

Table 4

Comparison between the shared non-technical skills of the behavioural maker systems for FRS Incident Commanders (THINCS), Pilots (NOTECHS) and Surgeons (NOTSS)

 

THINCS

NOTECHS

NOTSS

Decision making

Effective decision making and planning

Decision making

Decision making

Intuitive decision making

Problem definition and diagnosis

Considering options

Analytical decision making

Option generation

Selecting and communicating option

Planning

Risk assessment and option selection

Implementing and reviewing decisions

Outcome review

Leadership

Assertive, effective and safe leadership

Leadership

Leadership

Setting and maintaining standards of performance

Use of authority and assertiveness

Setting and maintaining standards

Values and supports others

Providing and maintaining standards

Supporting others

Leadership style

Workload management

Coping with pressure

Competence

Safety leadership

Situational awareness

Situational awareness

Situation awareness

Situation awareness

Information gathering

Awareness of aircraft systems

Gathering information

Understanding information

Awareness of external environment

Understanding information

Anticipating incident developments

Awareness of time

Projecting and anticipating future state

Teamwork

Teamwork and Interoperability

Cooperation

Communication and Teamwork

Cooperation

Team-building and maintaining

Exchanging information

Team formation

Considering others

Establishing a shared understanding

People oriented

Supporting others

Conflict solving

Coordinating team activities

The useability of a new system or tool plays an important role in whether it is taken forward and used in practice (Fletcher et al. 2004). After feedback from the raters, the paper-based system was improved and a mobile tablet-based app was developed, which affords greater ease of use in a range of operational, training and research settings. Prompted by feedback from the preliminary evaluation, the app integrates the observation and feedback systems. It also calculates the command skill rating and collates time-stamped observations about each sub skill, which can be scored and used to provide immediate feedback to an incident commander. This feedback could be communicated alongside video footage of the commander’s performance.2

6 Conclusion

The THINCS system incorporates the generic non-technical skills described by Flin et al. (2008) and echoes the command skills within the UK FRS national guidance for incident command and its foundation document (National Operational Guidance Programme 2015, 2016). The system is domain specific: It was framed in UK FRS terminology, and developed with the assistance of subject-matter experts from within the service. The development of the THINCS system will assist the UK FRS to prepare its incident commanders more effectively, and to monitor and determine the influence that command skills have on incident outcomes. In so doing, human error might be reduced and the safety of UK FRS operations will be enhanced.

Footnotes

  1. 1.

    For the purposes of this paper the National Fire Chiefs Council is synonymous with the Chief Fire Officers Association, whose work it took over in 2017.

  2. 2.

    The THINCS paper-based system, app, and guidance are available on request from the authors. It is freely available to the UK FRS under license from Cardiff University; and can be accessed by other organisations through independent licensing agreements with Cardiff University.

Notes

Acknowledgements

The research was supported by an Economic and Social Research Council (ESRC) studentship awarded to Philip C. Butler, and the School of Psychology. We should like to acknowledge the advice and support provided by the late Dr. Margaret Crichton. The authors thank the National Fire Chiefs Council for their support, and members of the UK Fire and Rescue Services. They also thank Paul Allen for his assistance in generating high fidelity audio–video recordings of the 6 incident commanders. Development of the THINCS app was supported by an ESRC Impact Acceleration Account, administered by Cardiff University (ES/M500422/1), and awarded to P.C.B. and R.C.H.. MyOxygen built the app, and we thank them for their ongoing support and encouragement.

Compliance with ethical standards

Conflict of interest

The authors confirm that they have no conflict of interests with respect to the publication of this paper, and that the research contained within it was approved by the Ethics Committee of School of Psychology, Cardiff University.

References

  1. Air Accident Investigation Bureau (1990) Report on the accident to Boeing 737-400G-OBME near Kegworth, Leicestershire on 8th January 1989. Aircraft Accident Report 4/90. HMSO, LondonGoogle Scholar
  2. Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res in Psychol 3:77–101CrossRefGoogle Scholar
  3. Bryman A (2006) Integrating quantitative and qualitative research: how is it done? Qual Res 6:97–113CrossRefGoogle Scholar
  4. Bryman A (2012) Social research methods, 4th edn. Oxford University Press, OxfordGoogle Scholar
  5. Carsten O, Vanderhaegen F (2015) Situation awareness: valid or fallacious? Cog Tech Work 17:157–158CrossRefGoogle Scholar
  6. Carthey J, de Leval M, Wright D, Farewell V, Reason J, all UK paediatric cardiac centres (2003) Behavioural markers of surgical excellence. Saf Sci 41: 409–425CrossRefGoogle Scholar
  7. Chief Fire Officers Association (2015) The future of incident command. Chief Fire Officers Association, TamworthGoogle Scholar
  8. Cohen-Hatton SR, Honey RC (2015) Goal-oriented training affects decision-making processes in virtual and simulated fire and rescue environments. J Exp Psychol Appl 21:395–406CrossRefGoogle Scholar
  9. Cohen-Hatton SR, Butler PC, Honey RC (2015) An investigation of operational decision making in situ: Incident command in the UK fire and rescue service. Hum Factors 57:793–804CrossRefGoogle Scholar
  10. Crichton MT, Flin R (2004) Identifying and training non-technical skills of nuclear emergency response teams. Ann Nucl Energy 31:1317–1330CrossRefGoogle Scholar
  11. Crichton MT, Lauche K, Flin R (2005) Incident command skills in the management of an oil industry drilling incident: a case study. J Cont Crisis Manag 13:116–128CrossRefGoogle Scholar
  12. Department for Communities and Local Government (2013) Fire and rescue authorities health, safety and welfare framework for the operational environment. GOV.UK: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/209362/HSFrameworkJunecombined.pdf. Accessed 27 May 2017
  13. Devitt K, Holford S (2010) The development of resource management and leadership behavioural markers for the Merchant Navy. Warsash Maritime Academy: http://www.warsashacademy.co.uk/news-events/resources/behavioural-markers-research-katherine-devitt-simon-holford-270710.pdf. Accessed 31 May 2017
  14. Fletcher G, Flin R, McGeorge P, Glavin R, Maran N, Patey R (2004) Rating non-technical skills: Developing a behavioural maker system for use in anaesthesia. Cog Technol Work 6:165–171Google Scholar
  15. Flin R, Martin L (2001) Behavioural markers for crew resource management: A review of current practice. Int J Aviation Psychol 11:95–118CrossRefGoogle Scholar
  16. Flin R, Martin L, Goeters K-M, Hörmann H-J, Amalberti R, Valot C, Nijhuis H (2003) Development of the NOTECHS (non-technical skills) system for assessing pilots’ CRM skills. Hum Factors Aero Saf 3:95–117Google Scholar
  17. Flin R, O’Connor P, Crichton M (2008) Safety at the sharp end: a guide to non-technical skills. Ashgate, AldershotGoogle Scholar
  18. Greater Manchester Fire and Rescue Service (GMFRS) (2016) Fatal accident investigation report. Paul’s hair and beauty world, Oldham Street, Manchester. July 13 2013. Greater Manchester Fire and Rescue Authority website: http://authority.manchesterfire.gov.uk/documents/s50006158/Oldham%20St.%20Report%20FINAL%20low%20res%20web.pdf. Accessed 23 Aug 2017
  19. Greene JC, Caracelli VJ, Graham WF (1989) Toward a conceptual framework for mixed-method evaluation designs. Educ Eval Policy Anal 11:255–274CrossRefGoogle Scholar
  20. Guba EG, Lincoln YS (1994) Competing paradigms in qualitative research. In: Denzin NK, Lincoln YS (eds) Handbook of qualitative research. Sage, Thousand Oaks, pp 105–117Google Scholar
  21. Health and Safety Executive (1999) HSG48 Reducing error and influencing behaviour, 2nd edn. HSE Books, NorwichGoogle Scholar
  22. Health and Safety Executive (2010) Striking the balance between operational and health and safety duties in the Fire and Rescue Service. Health and Safety Executive website: http://www.hse.gov.uk/services/fire/duties.pdf. Accessed 2 Sept 2017
  23. Health Commission (2009) Investigation into Mid Staffordshire NHS foundation trust. nhshistory.net website: http://www.nhshistory.net/midstaffs.pdf. Accessed 16 Aug 2017
  24. Helmreich RL, Merrit AC, Wilhelm JA (1999) The evolution of crew resource management training in commercial aviation. Int J Aviation Psychol 9:19–32CrossRefGoogle Scholar
  25. Klampfer B, Flin R, Helmreich RL, Häuser R, Sexton B, Fletcher G, Field P, Staender S, Lauche K, Dieckmann A, Amacger A (2001) Enhancing performance in high risk environments recommendations for the use of behavioural markers. The European Aviation Safety Agency information hub website: https://www.casa.gov.au/sites/g/files/net351/f/_assets/main/download/caaps/ops/sms-3-1_appb.pdf. Accessed 4 Feb 2019
  26. Klein GA (2008) Naturalistic decision making. Hum Factors 50:456–460CrossRefGoogle Scholar
  27. Klein G, Calderwood R, Macgregor D (1989) Critical decision method for eliciting knowledge. IEEE Trans Syst Man Cybern 19:462–472CrossRefGoogle Scholar
  28. Kodate N, Ross AJ, Anderson JE, Flin R (2012) Non-technical skills (NTS) for enhancing patient safety: Achievements and future directions. Jpn J Quality Saf Healthcare 7:360–370Google Scholar
  29. Marine Casualty Investigative Body (2012) Cruise ship Costa Concordia marine casualty on January 13, 2012 report on the safety technical investigation. gCaptian website: http://3kbo302xo3lg2i1rj8450xje.wpengine.netdna-cdn.com/wp-content/uploads/2013/05/Costa_Concordia_-_Full_Investigation_Report.pdf. Accessed 16 Aug 2017
  30. McCulloch P, Mishra A, Handa A, Dale T, Hirst G, Catchpole K (2009) The effects of aviation-style non-technical skills training on technical performance and outcome in the operating theatre. Qual Saf Health Care 18:109–115CrossRefGoogle Scholar
  31. Mitchell L, Flin R, Yule S, Mitchell J, Coutts K, Youngson G (2011) Thinking ahead of the surgeon. An interview study to identify scrub nurses’ non-technical skills. Int J Nurs Studies 48:818–828CrossRefGoogle Scholar
  32. National Operational Guidance Programme (2015) The foundation for incident command (2nd ed.). National Operational Guidance Programme website: https://www.ukfrs.com/foundation-knowledge/foundation-incident-command. Accessed 7 Oct 2017
  33. National Operational Guidance Programme (2016) Incident Command Guidance. National Operational Guidance Programme website: https://www.ukfrs.com/guidance/incident-command. Accessed 7 Oct 2017
  34. O’Connor P, Long WM (2011) The development of a prototype behavioral marker system for US Navy officers of the deck. Saf Sci 49:1381–1387CrossRefGoogle Scholar
  35. O’Connor P, O’Dea A, Flin R, Belton S (2008) Identifying the team skills required by nuclear power plant operations personnel. Int J Industr Ergo 38:1028–1037CrossRefGoogle Scholar
  36. QSR International (2014) NVivo 10 for Windows 10 [Computer software]. http://www.qsrinternational.com/products_nvivo.aspx. Accessed 4 Feb 2019
  37. Qualtrics (2016) Qualtrics [Computer software]. https://www.qualtrics.com. Accessed 4 Feb 2019
  38. Rasmussen J (1982) Human error: A taxonomy for describing human malfunction in industrial installations. J Occ Accidents 4:311–333CrossRefGoogle Scholar
  39. Reason J (1990) Human error. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  40. Reason J (1997) Managing the risks of organizational accidents. Ashgate, AldershotGoogle Scholar
  41. Reason J (2008) The human contribution: Unsafe acts, accidents and heroic recoveries. Routledge, LondonGoogle Scholar
  42. Roberts R, Flin R, Cleland J (2015) Staying in the zone: offshore drillers’ situation awareness. Hum Factors 57:573–590CrossRefGoogle Scholar
  43. Rugg G, McGeorge P (1997) The sorting techniques: a tutorial paper on card sorts, picture sorts and item sorts. Expert Syst 14:80–93CrossRefGoogle Scholar
  44. Shappell SA, Weigmann DA (1997) A human error approach to accident investigation: The taxonomy of unsafe operations. Int J Aviation Psychol 7:269–291CrossRefGoogle Scholar
  45. Skills for Justice (2013a) SFJ EFSM1 Provide strategic advice and support to resolve operational incidents. The National Occupational Standards website: https://www.ukstandards.org.uk/PublishedNos/SFJEFSM1.pdf. Accessed 9 Jun 2018
  46. Skills for Justice (2013b) SFJ EFSM2 Lead, monitor and support people to resolve operational incidents. The National Occupational Standards website: https://www.ukstandards.org.uk/PublishedNos/SFJEFSM2.pdf. Accessed 9 Jun 2018
  47. Skills for Justice (2014) SFJ FRS WM7 Lead and support people to resolve operational incidents. The National Occupational Standards website: http://nos.ukces.org.uk/PublishedNos/SFJWM7.pdf#search=WM7. Accessed 9 Jun 2018
  48. Torrie S (2012) A report to Scottish Ministers the 2008 Galston mine incident. APS Group Scotland, EdinburghGoogle Scholar
  49. Watterson A (2015) Fire-fighter fatalities at fires in the UK: 2004–2013: voices from the fireground. The Fire Brigades Union website: https://www.fbu.org.uk/publication/briefing-firefighter-fatalities-fires-uk-2004-2013-voices-fireground. Accessed 24 Aug 2017
  50. Weick K (1990) The vulnerable system: an analysis of the Tenerife air disaster. J Management 16:571–593Google Scholar
  51. Youngson GG (2016) Intraoperative performance, non-technical skills and surgical safety. In: Flin R, Youngson GG, Yule S (eds) Enhancing surgical performance: a primer in non-technical skills. CRC Press, London, pp 3–16Google Scholar
  52. Yule S, Flin R, Paterson-Brown S, Maran N, Rowley D (2006) Development of a rating system for surgeons’ non-technical skills. Med Educ 40:1098–1104CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • Philip C. Butler
    • 1
  • Robert C. Honey
    • 1
    Email author
  • Sabrina R. Cohen-Hatton
    • 1
    • 2
  1. 1.School of PsychologyCardiff UniversityCardiffUK
  2. 2.London Fire BrigadeLondonUK

Personalised recommendations