Keywords

1 Introduction

Unmanned Aerial Vehicles (UAVs) encompass a wide range of vehicles from hand held consumer toys like the AR. Drone by Parrot [1] to large military vehicles like the Global Hawk which has a 35 m wing span [2]. A military Unmanned Aircraft System (UAS) includes the UAV; the Ground Control Station (GCS); dedicated crews for maintenance, launch and recovery; and a crew to operate the UAV during mission execution. Military UAS are typically outfitted with a variety of sensors and used for intelligence, surveillance and reconnaissance (ISR) missions, with some UAV having targeted strike capabilities [3]. Military UAS are designed to allow for ISR missions in hostile environments for extended periods of time without putting flight crews at risk. Due to the nature of UAS having its crew in a separate location from the aircraft, it is necessary that automation be employed to ensure safe control of the aircraft and its systems. It is critical that the human-automation interaction between a UAS and its crew be effective, this is especially true for military systems that are used on the battlefield where lives are at stake [4]. To promote mission effectiveness, it is important that the human crew have trust in their autonomous systems [4]. This trust comes from machine reliability and an effective Human-Machine Interface (HMI) that can explain and justify the automation’s recommendations and actions [4].

The Canadian Armed Force’s Joint Unmanned Surveillance and Target Acquisition System (JUSTAS) program has the goal to procure a long endurance UAS for domestic and international operations [5]. As a stakeholder, JUSTAS has mandated DRDC to identify critical UAS GCS HMI information requirements and provide recommendations on GCS crew resource management. DRDC has additional stakeholder requirements to identify UAS crew responsibilities and training requirements, develop preliminary training technologies and material, and provide guidance on GCS airworthiness certification. The GCS of the JUSTAS UAS will follow the Royal Canadian Air Force (RCAF) squadron level UAS control concept, where each UAS mission crew includes: a pilot or Air Vehicle Operator (AVO); airborne sensor operators referred to as Payload Operators (PO); and intelligence analysts for Imagery (IMA) and Electronic Warfare (EW).

Defence Research and Development Canada (DRDC) has performed a three phase research project to meet these stakeholder requirements [6]. Phase 1, called stakeholder analysis, included the development of a composite mission based on the JUSTAS Concept of Operations (CONOPS) [7], UAS operator characteristics analysis and a preliminary Human Factors (HF) engineering system analysis to identify equipment and function allocation following NATO STANAG 3994 [8].

Phase 2 had multiple tasks to analyze UAS GCS functional requirements and training needs. The first task was to develop a functional requirements document that lists all UAS operator functions to accomplish the missions identified in the JUSTAS CONOPS. Another main task for this phase was a Cognitive Task Analysis (CTA) [9] to identify crew information requirements and decisions. The results of the CTA were used as the basis of a task flow analysis to develop task network models for operator performance modelling and data analysis for UAS GCS design and training requirements. Additionally HMI requirements and initial GCS user interface concepts were also derived from the CTA results. A training needs analysis was also performed to identify the knowledge, skills and abilities required for each of the UAS crew members.

DRDC is currently in Phase 3, which is focused on the development and evaluation of a simulation-based UAS mission experimentation testbed called Testbed for Integrated GCS Experimentation and Rehearsal (TIGER). Experimentation on TIGER will address stakeholder requirements on crew configuration, GCS functional requirements, workspace layouts, airworthiness certification and training requirements. TIGER consists of six crew workstations, three simulation control workstations and two experimenter workstations [6]. The six crew workstations include positions for an AVO, PO, two IMA, and two EW.

To date, preliminary trials have been conducted to assess TIGER’s baseline capabilities and crew performance with six crew members [10]. Results of the trial indicated that multiple crew members often lost situational awareness (SA) due to unfamiliarity with the system and did not consider following Rules of Engagement (ROE) and Laws of Armed Conflict (LOAC) while engaging targets. Participant feedback after the trials also indicated that an automated decision aid could be helpful for improving crew adherence to ROE and LOAC. This led to the presentation of the “Authority Pathway” concept [10], an Intelligent Adaptive Interface (IAI) that assists UAS crews in following ROE and LOAC when engaging targets. An IAI is an operator interface that dynamically changes the display and/or control characteristics of human machine systems to adaptively react to the operator states and external events in real time [11]. The Authority Pathway, being developed on TIGER to address JUSTAS requirements for GCS HMI recommendations, is an IAI because it dynamically changes its HMI on TIGER in real-time in response to changes in the mission and environment as well as when it receives inputs from the UAS crew or Tasking Authority (TA).

2 Authority Pathway

The Authority Pathway was developed to support UAS crew HF studies investigating the use of Intelligent Adaptive Automation (IAA) technology [12] to assist the crew in maintaining awareness of the steps and permissions needed to conduct a lawful and successful target engagement [10]. An IAA technology is necessary for functional integration, rather than functional separation [12]. IAA technologies are adaptive because system control is shared dynamically by both the operator and automation based on their availabilities, capabilities, and limitations. IAA technologies are intelligent because they seek to restore the operator to the role of decision maker and provide safeguards for situations where time constraints or problem complexity restrict the operators problem-solving ability [12]. IAA systems proactively collect information, are goal driven, are capable of reasoning at multiple levels, and can learn from experience [12]. The goal of IAA is to enhance operator judgement, decision making, and responsibility while mitigating operator limitations [12]. The Authority Pathway uses IAA technology to assist the UAS crew in making critical target engagement decisions, while maintaining mission SA, and following proper ROE and LOAC, especially in time critical and complex situations. The Authority Pathway is agile to dynamically changing mission goals, has a variety of HMIs for different users, and allows for varying levels of automation to be consistent with the procedures and time-critical operational context.

The Authority Pathway concept is based on the steps comprising the targeting process depicted in Fig. 1. The process essentially commences once a Person of Interest (POI) has been detected, localized and retained as a potential target. The subsequent steps involve: completing a positive identification (PID) checklist; notifying the Tasking Authority (TA) and requesting authorization for the use of a kinetic response; performing a Collateral Damage Estimate (CDE) while simultaneously completing the weapon engagement planning; notifying the TA of the UAS readiness to engage and requesting weapon release authorization; lazing the target (as required); and releasing the weapon. Maintaining continuous eyes on target is a requisite parallel activity that must occur throughout the target engagement process. Battle Damage Assessment (BDA), not shown in the figure, is performed subsequent to the weapon release.

Fig. 1.
figure 1

Target engagement process overview.

Full size image

During a sortie, the UAS may take on different roles. For instance, in the case of a call for direct fire, the UAS may be responsible for the targeting and weapon release whereas in the case of a call for indirect fire, the UAS crew may only be providing a target lazing capability for another manned or unmanned asset.

The Authority Pathway seeks to address two critical functional areas of the target engagement process: (1) maintaining a shared mission SA among all UAS crew members; and (2) ensuring that all applicable, relevant ROE, LOAC are considered in a timely manner and that the standard operating procedures are followed. One of the underlying premises of the Authority Pathway concept is that the judicious use of IAA applied to the GCS HMI will facilitate the individual and collaborative decision-making during the target engagement process.

Since the work reported on in [10], recent analysis and design efforts for the Authority Pathway have highlighted a number of challenges:

  • the system must adapt to the mission type and UAS role, both of which may vary over the course of the sortie;

  • the system must provide user-specific views (e.g. AVO, PO, IMA, role player, experiment observer); and

  • the system must allow for the evolution of the level of automation consistent with future UAS requirements for additional semi-automated crew functions related to target engagement.

The following subsections address each of these challenges and describe these challenges in terms of requirements. The last Subsection 2.4, presents the advantages and disadvantages of the Authority Pathway application.

2.1 Authority Pathway Mission Agility

“C2 Agility is the capability of [a system] to successfully effect, cope with, and/or exploit changes in circumstances. […] Agility enables entities to effectively and efficiently employ the resources they have in a timely manner.” [13].

The Authority Pathway will operate in an environment in which operational and emulated command and control (C2) systems have been integrated. It will support a variety of missions that will result in a specific instantiation based on mission templates. The fog of war, the unpredictability of military operations and the dynamic nature of UAS missions are all reasons why these missions are subject to change, sometimes referred to as dynamic re-tasking, i.e. a change in mission during a sortie. During dynamic re-tasking, the system is required to reconfigure itself to meet the evolving mission requirements.

The so-called agility enablers are: responsiveness, versatility, flexibility, resilience, adaptiveness and innovativeness [13]. The Authority Pathway has implicit C2 agility requirements since the target engagement requires many of these enablers. In particular, the responsiveness and adaptiveness of the system will impact the efficiency with which requests to the TA can be communicated and processed. Limitations of current systems relying too heavily on radio communications and chat for UAV operations have been documented [14]. For example, these enablers are important in the case when a Troops-In-Contact (TIC) mission preempts an ongoing Pattern-Of-Life (POL) mission. The system must be adaptive and react to this change by presenting a modified instance of the Authority Pathway that reflects the relevant set of applicable ROE and is consistent with the procedures and time-critical operational context. The Authority pathway could automatically detect this change in mission by interpreting UAS system data (e.g., C2 systems, sensor) or the Authority Pathway ROE could be changed manually by the UAS crew. These changes in ROE would result in a new or modified Authority Pathway display to inform the crew of the changes.

The need for C2 agility of the Authority Pathway is further compounded by the requirement for UAS collaboration scenarios wherein one UAS asset, for instance may provide a target lazing capability for another manned or unmanned asset that is providing a weapons fire. Collaborations among assets, whose crews are not co-located, present specific coordination and communication challenges related to maintaining mission SA while executing critical decision-making. For example, timely communication and coordination are vital to ensuring the quick response times required in the case of time-sensitive targeting (TST), in both deliberate and dynamic targeting scenarios while considering CDE, ROE, airspace and other restrictions during the targeting process [15].

2.2 Authority Pathway User Views

The Authority Pathway will be accessed by crew members, the TA, and experimenters. Different users require different functionality from their Authority Pathway views. Each view is a separate client application or configuration.

2.2.1 Shared SA View

This view accepts no user input. It displays the current status of the target engagement activity in the form of a graphic that indicates completed Authority Pathway steps in green and uncompleted steps in orange (see Fig. 2). The Shared SA View can be integrated into other applications, such as an overall mission SA display, as shown in Fig. 3.

Fig. 2.
figure 2

Example Authority Pathway shared SA view (Color figure online)

Full size image
Fig. 3.
figure 3

Authority Pathway shared SA view integrated into an example overall mission SA display which also includes UAS sensor feed, maps, and other important mission information.

Full size image

2.2.2 UAS Crew View

This view is intended for use by the UAS crew members not performing the AVO or mission commander roles (e.g. PO/IMA/EW). In addition to the Shared SA View, this view provides information panes with a summary of the various steps (see Fig. 4). The information panes also allow a crew member to request actions from other users or set reminder alarms. When clicking on a specific information item, this view offers a drill-down capability to access increasing levels of detail. The information panes are related to another important feature of the Authority Pathway; checklists that crew members must complete and submit as part of the authorization request process.

Fig. 4.
figure 4

Authority Pathway information pane

Full size image

2.2.3 UAS Crew Commander View

This view is intended for the AVO, generally acting as crew commander, i.e. the person responsible for the mission and weapon engagements. In addition to the functionality of the UAS crew view, this view allows the crew commander to validate checklists and submit kinetic response and weapon release authorization requests to the TA.

2.2.4 Tasking Authority/White Cell View

During experiments, this view is for use by a role player acting as the TA. In addition to the read-only access to the Authority Pathway status and information panes, this view allows the user to grant or deny authorizations from the AVO. This view allows the user to review a checklist and to flag items that are incomplete, missing, or otherwise inadequate. The user can reject a request and provide instructions for rectifying it for subsequent resubmission.

2.2.5 Experimenter View

The Authority Pathway will have a record and playback capability for after-action review (AAR). Since the current focus of the Authority Pathway application is to support experiments, AAR will include observations from experimenters. The experimenter view allows the user to generate events and comments for subsequent review as part of the AAR. For example, the user can identify specific instances during which Eyes on Target were lost and then regained. Future experimenter view functions may include the capability to record observations concerning the operator’s state. Future plans for the TIGER platform include psycho-physiological operator state monitoring capabilities that will provide data to the Authority Pathway during the experiment for the purposes of AAR and other post-experiment analyses.

2.3 Target Engagement Crew Function Automation

In the context of the Authority Pathway concept development, one of the underlying goals of integrating IAA aides within the GCS is to automate certain aspects of crew functions and thereby reduce the operator workload so that the crew can better focus on tasks that cannot or should not be automated.

The introduction of automation technology into complex environments such as UAS operations requires consideration of the risks and implications associated with a transfer of control from a human operator to a machine. The Pilot Authority and Control of Tasks (PACT) framework was initially developed to investigate the role of automation in future manned military aircraft [16]. Although there are many taxonomies to describe levels of automation when discussing human-automation interactions [12], PACT is used in this case because it gives a simple and clean description of the level of automation for Authority Pathway at a conceptual level. Applying these levels of automation (LOAs) is not always practical for systems design however it provides a conceptual understanding of the human-automation interaction goals of the system. PACT suggests that there are three basic levels of automation: commanded (full pilot authority); assisted (shared pilot/computer authority); and automatic (full computer authority). Although fully automated control of UAV platforms currently is not a viable option, the PACT framework descriptions of assisted levels of authority are useful for the design of semi-automated UAS crew functions.

Figure 5 represents the six PACT levels of human/machine control in terms of human-machine responsibility sharing. The third column indicates a level of authority that includes the notion of automation management strategies [16]. Levels 0 and 5 are reference levels where the human and the pilot have complete control, respectively. Levels 1 through 4 represent shared human machine control with increasing machine control. With respect to the automation of crew functions, some functions are obvious candidates for automation while the automation of other functions may be contrary to doctrine and/or legal considerations. For example, certain aspects of the CDE activity can be greatly facilitated through the use of agent-based and other artificial intelligence technologies. Automatically signaling potential fratricide or civilian casualties and other relevant information to the crew early on in the target engagement process could facilitate crew decision-making, leading to remedial measures and alternate courses of action. To illustrate this point Fig. 6 depicts the cognitive process for determining if a target location is free of neutrals or friendly forces in the form of a decision ladder identified in the CTA analysis. The left side of the decision ladder involves primarily an analysis of the situation by the decision maker. The top of the ladder requires an interpretation or value judgment leading to a task, while the right side of the ladder is related to the planning and execution of the task. The CTA decision ladders were a key tool for the UAS target engagement task analyses and were used to identify areas where IAS technologies could be applied.

Fig. 5.
figure 5

Adapted from [16]

PACT levels of autonomy

Full size image
Fig. 6.
figure 6

Example UAS target engagement decision ladder

Full size image

The decision ladders identified as pertinent to the Authority Pathway included two possible types of automation requirements: (1) IAA requirements; and (2) IAI requirements. IAA requirements are related to behind the scenes calculations and process performed on behalf of (or instead of) a crew member. IAI requirements involve visual and aural cues, checklists and indications that consolidate or accentuate information directed to the crew member. The two main areas addressed by the Authority Pathway, i.e. sharing SA and ensuring that procedures are followed, fit clearly in left and right sides of the decision ladder, respectively. For the ten steps comprising the target engagement process shown in Fig. 1, nineteen decision ladders were identified from the CTA. Table 1 presents a set of automation requirements associated with the decision ladder shown in Fig. 6. For each of the five tasks shown, two requirements have been identified. Features A, B, C and D involve the production of cues directed at the operator geared at increasing operator situation awareness concerning the existence of neutrals of friendlies in the vicinity of a target location. Feature C is of the type IAA and involves communication with other systems and automated processing in order to support feature D, the map display. These features are consistent with PACT autonomy level 2: the system is configured to inform the operator when specific conditions are met or triggers occur. The human operator is still responsible for determining if the location is free of neutrals or friendlies. Features E and F support the principal human decision-making task for this activity and also consistent with PACT autonomy level 2.

Table 1. Automation requirements from decision ladder: Location free from friendlies/neutrals
Full size table

Feature G involves the system calculating potential strategies for subsequent actions and therefore is consistent with level 3 control. Features H and I collectively call for displaying a checklist for engaging the target and an automatic execution of compliant engagement strategy which also is consistent with level 3 – the system proposes an action that must be authorized by the operator. In the case of any changes in settings, feature J calls for confirmation by the operator, also consistent with level 3 control.

The set of automation requirements associated with the nineteen decision ladders is one of the key inputs into the Authority Pathway analysis and design. The following sections describe some of the primary design principles that enable the automation of crew functions: (1) procedures and ROE checklists; (2) authorization requests; and (3) status displays.

2.3.1 Procedure and ROE Checklists

Target engagement procedures can vary as a function of the operational context, the type of mission and the force structure. The steps involved in a TST or TIC situation are not the same as those required for engaging a target during a POL mission. Also, it can be time-consuming for personnel to identify all of the applicable and relevant ROE for a given situation. The Authority Pathway address these concerns by being adaptive to the specific situation and generating appropriate checklists to guide the target engagement process, including facilitating the assessment of relevant ROE and other items related to the LOAC.

The Authority Pathway includes mechanisms for: (1) completing a checklist among the UAS crew; (2) having the crew commander confirm that the checklist is completed; and (3) having the crew commander communicate that a checklist is incomplete with indications as to how to complete it.

2.3.2 Semi-automated Authorization Requests

Authorization requests are made by the UAS crew commander to the TA at two steps of the target engagement process: (1) after the PID checklist has been completed and confirmed, at which point the UAS crew requests an Authorization for Kinetic Response; and (2) once the Weapon Engagement Planning (WEP) and CDE steps have been completed and confirmed at which time the UAS crew requests an Authorization for Weapon Release.

The Authority Pathway includes mechanisms for communicating requests for authorization to the TA along with access to the completed checklists. The TA has the capability to approve (grant) a request for authorization or to refuse (deny) a request based on their knowledge of the ROEs and LOAC for the current mission situation. In the case of a denied request, the checklist items that were incomplete, missing or otherwise inadequate will be flagged and returned as part of the Response for Authorization Request. In the case of an approved request, additional information or guidance may be provided as part of the response.

2.3.3 Common/Shared Status Display

At the highest level, the Authority Pathway status display (see Fig. 2) provides an unambiguous, instantaneous view of the target engagement process status. This view serves two major purposes: (1) it contributes to the overall mission SA so that the UAS crew is aware of the current step in the process; (2) it facilitates any necessary remedial actions by clearly indicating why a given step is stalled or why a Request for Authorization has been denied.

The crew status displays are equipped with a drill-down feature that allows the operators to access increasing levels of detail concerning information items of a given checklist or response to a Request for Authorization.

2.4 Advantages and Disadvantages of Authority Pathway

The automation of alerts, notifications and warnings contribute to increasing the responsiveness and therefore the C2 agility of a UAS crew during a target engagement. At the same time, caution must be exercised to avoid an excessive number of visual and aural cues to which the crew is subjected to which case the crew is unable to respond. Automation bias is another potential disadvantage wherein the crew becomes accustomed to the system providing cues for a given event or in a given situation and therefore hesitated to identify an event in the absence of a cue from the system.

The use of automated checklists has several advantages, including: ensuring that procedures are followed; detecting/preventing human error; facilitating the communication/sharing of task status; facilitating the request for authorization process; and the possibility of recording task preparation for AAR. The adaptiveness of the system to varying operational contexts and mission types contributes to a greater C2 agility. As with automated alerts, one disadvantage of automating procedure checklists is that operators may become over-reliant on the system for knowledge of procedures and thus become vulnerable in case of system failures or when operating in a context not covered by the system or if the system is not available (e.g. a multi-national task force or exercise). As with other fields where operators rely increasingly on automation, the risks associated with automation bias must be mitigated [12].

The use of automation to formulate, communicate and track requests for authorization between the UAS crew and the TA for a variety of situations presents several advantages over traditional methods involving radio communications and chat. In particular, for POL missions, the ability to prepare, inspect, review and remediate requests for authorization to the tasking authority using semi-structured data contributes to the system flexibility (different situations) and versatility (i.e. robustness) and thus contributes to increasing the C2 agility of the UAS. However, in the case of TST situations during which a near-instantaneous response is required, in some cases traditional methods may be better suited. Also, augmenting introducing IAI capabilities into existing chat communications has been suggested as a means to bridge existing means of communication during operations with IAS technologies [14].

In the context of the Authority Pathway, the lack of a common mission SA was identified as one of the key issues during analysis of HF experiments conducted on TIGER with military personnel [6]. Therefore, providing a common/shared target engagement SA display should alleviate this issue, and will be investigated in a future trial.

The example decision ladder discussed above considers the task of CDE, which primarily involves analysis. For this reason, it is has a high potential for automation. Other crew functions and related decision-making processes, such as weapon release, likely should remain the responsibility of the human for the foreseeable future.

3 Conclusions

Through the HF analysis and experimentation of a Canadian simulation-based UAS mission experimentation testbed, it was identified that a novel UAS GCS IAI, called the Authority Pathway, could improve UAS crew SA and adherence to ROE and LOAC. The Authority Pathway must have the agility to adapt to different mission types, provide specific interfaces for specific users, and allow a level of automation that meets future requirements. The Authority Pathway will increase crew responsiveness, ensure procedures are followed, facilitate communication, and allow for comprehensive AAR. However care is needed to ensure crews do not become over-reliant and certain scenarios may require more traditional methods. These capabilities enable the Authority Pathway as a research and development tool to support stakeholder requirements: specifically identifying UAS GCS HMI recommendations.