Keywords

1 Introduction

Eye tracking provides scientific evidence on the underlying causal relations between independent variables and dependent variables. In other words, eye-trackers offer not only what causes the results, but also how the results are caused (Mayer 2010). The application of eye-tracking in the study of flight simulation is promising as it provides direct feedback, which could diagnose potential factors that impact upon pilot attention and situation awareness on the flight deck (Robinski and Stein 2013). Military aviation studies suggest analysing eye movements is beneficial for fighter pilots to increase their tactical performance (Wetzel et al. 1998; Yu et al. 2014). Eye tracking methodology is based on two assumptions: eye-mind and immediacy assumptions. The immediacy assumption proposed the location of a fixation coincides with the cognitive processing of concurrent visual stimuli (e.g., words) at that location. The eye-mind assumption indicated eye movement is correlated to concurrent perceptual and cognitive processes which coincides with, and is bounded by, the position fixated at the point in time, and that this processing starts at the point of fixation and continues until all possible analyses were completed (Just and Carpenter 1980). Furthermore, eye tracking studies focus on two aspects: “When” and “What” (van Gompel 2007). The temporal aspect of eye movement control (when) primarily concerns the question as to when a given saccade is executed or, more precisely, the time course of cognitive processing events and control decisions occurring during a fixation. In contrast, “What” concerns what information is extracted concurrently to guide the eyes. With the development of technologies, more research has adopted eye-tracking in various contexts, such as cognitive processes in reading (Rayner 1998), learning (van Gog and Scheiter 2010), problem solving (Hegarty et al. 1995; Lin and Lin 2014), information processing (Lu et al. 2011), and flight deck design (Li et al. 2015). It provides researchers a promising way to study what people think when they see something, such as text or graphics (Renshaw et al. 2004).

The formal authority of the automation status is communicated within the cockpit via the FMAs situated at the top of the cockpit PFD. Monitoring these FMAs, and calling out mode transitions seen via the FMAs is considered important for obtaining and maintaining mode awareness on the flight deck. The ‘call-out’ is when one member of the flight deck team aurally announces a mode change to highlight the change to the other crew member; intending to ensure effective crew communication and SA (Airbus 2006). However, an experiment using eye tracking techniques investigating mode awareness by Björklund et al. (2006) found that flight crews used a variety of strategies to keep track of the autopilot status, and relied little on the PFD FMAs. Lack of SA is a primary reason for pilot error, even among experienced pilots, and pilot SA can be assessed by monitoring visual behavior (van Dijk et al. 2011). Eye movement patterns can be used as an objective measure of cognitive workload and thus the efficiency of a HMI design; where inefficient designs lead to an increase in relative cognitive workload. An eye tracking device can be used to measure various metrics related to a pilots’ attention (Zelinsky 2008).

Breakdowns in human-machine coordination have been a repetitive problem in automated aircraft (Dekker 2000; Woods and Sarter 2000), and recent reports captured via the Aviation Safety Reporting System (ASRS) administered by NASA (National Aeronautics and Space Administration) show that this continues to be the case (NASA 2015). To avoid human-machine coordination breakdown in the cockpit, pilots have to maintain situation awareness (SA) of the automatic system’s status. ‘Mode awareness’ is a critical ingredient for avoiding automation-related problems (Funk et al. 1997). The introduction of autopilot and auto-thrust functions on aircraft was designed to reduce flight crew workload and therefore reduce the number of accidents and incidents that occur due to high workload conditions as a contributing factor, among other reasons such as more efficient trajectory flying. Particularly on long flights, use of an autopilot can reduce pilot fatigue by maintaining a set course and steady, level flight for long periods of time without needing the human pilot to concentrate on this task (Harris 2011). However, while the original aim of this was to reduce crew workload in terms of manually flying the aircraft, it shifted the pilot’s role from hands-on flying to a systems managing role while the autopilot is in operation. Rather than reducing workload, this changed the workload; relieving the pilot of perceptual motor load (‘doing’) with an increase in cognitive workload (‘thinking’). Humans are not ideally suited to monitoring roles. Combined with inadequate feedback from automation systems, this creates a recipe for mode awareness to be reduced (Endsley 1996). The aims of current research are to investigate the design aspect of FMAs with a potential for improvement to Human-Computer Interactions (HCI) on the flight desk to improve pilot SA performance.

2 Method

2.1 Participants

The study involved twenty-five participants consisting of three qualified commercial pilots with flight experience between 1,242 and 2,400 h (M = 1722.3, SD = 603.7); eight private pilot license holders with flight experience between 50 and 185 h (M = 108.1, SD = 41.67) defined as experienced participants; and 14 avionics engineers with limited flight experience consisting of between 0 and 10 h (M = 3.64, SD = 5.84) defined as non-experienced participants. As data were gathered from human participants a research proposal was created and submitted to the Cranfield University Research Ethics System (CURES) for ethical approval of the research and experiment. Ethical approval was granted for the research prior to starting the experiment by the CURES team, and informed consent secured by all participants prior to commencement of the experiment. All signed forms are available upon request.

2.2 Apparatus

B747-400 Flight Simulator.

The experiment was run on Cranfield University’s high-fidelity B747-400 Flight Simulator. This simulator comprises a realistic mock-up of a cockpit of Boeing commercial aircraft with functioning flight controls, stick-shaker stall warning, and overspeed alerts (Fig. 1a).

Fig. 1a.
figure 1

Cranfield B747-400 flight simulator.

Full size image

Eye Tracking Device.

To capture objective eye metrics a Pupil Labs “Pupil Pro” eye tracking device was applied. The device carries the following specifications, Eye Camera Maximum Resolution – 640 × 480 at 30fps; World Camera Maximum Resolution – 1920 × 1080 at 30fps; Headset Weight 44 g plus Cable Weight – 60 g. The Pupil Pro eye tracking device is worn like a pair of glasses and connected via cable to a data recorder. This device has a ‘World Camera’ mounted in the centre of the glasses showing the orientation and view of the wearer’s head. A second camera, the Eye Camera, is mounted offset right and low. This part of the device tracks a participant’s pupil on the right eye (Fig. 1b).

Fig. 1b.
figure 2

Pupil pro eye tracker

Full size image

2.3 Research Design

To test ability of participants in noticing, monitoring, and responding to mode changes during flight a scenario was prescribed that would induce some workload to keep the participants on a primary task of flying the aircraft. The participants were set a workload-inducing scenario consisting of flying the B747 down a 3° ILS beam while on approach to land; starting at 3,200ft, 8 miles from the runway. Due to some participants having little or no experience of the cockpit, aircraft configuration settings were set at gear down, flaps 20°, and a power setting to achieve 170–190 knots for all flights. To accommodate limited access to the flight simulator and increase the number of participants, a short flight was devised of 2.5 min, stopping approximately 1 mile before final touchdown on the runway. The vertical and horizontal deviation from the ILS beam was displayed to the researchers during the experiment. The flight simulator had the lights and switches in the correct places on the MCP for a realistic setting. A bespoke FMAs panel was created which was convenient to relocate around the cockpit instruments, connected to a switch held by the researchers to turn the mode annunciations on and off. Automatic-unexpected mode changes would be simulated and require the participants to callout the mode change when noticed; introduced as another task for the participants to be aware of, and to test data-driven monitoring performance from the FMAs design.

Two different positions of FMAs were evaluated for the efficiency of increasing pilot’s SA on the changing modes of automation. The FMAs on the position A is situated above the attitude indicator on the PFD, where FMAs are traditionally placed (Fig. 2a); and the FMAs on the position B is situated on the far left of the MCP (Fig. 2b). Position B was so designated as to keep within the pilot’s primary field of view, and in a position that could accommodate the FMAs panel without significant disruption or redesign of current MCPs. The participants were split into two groups: The control group was assigned to fly the scenario with the FMAs panel on the PFD and the experimental group was assigned to fly the mission with the FMAs panel on the MCP. There are four defined AOIs for the current experiment design, namely the airspeed, attitude, altitude indicators, and the FMAs panel.

Fig. 2a.
figure 3

FMAs on the top of PFD.

Full size image
Fig. 2b.
figure 4

FMAs on the left of MCP

Full size image

2.4 Data Collection Process

Once the participant’s consent form was signed, participant were given a briefing sheet for the experiment, followed by calibration process to the left seat of the B747 simulator and fitted with the eye-tracking device. The laptop displays the Pupil Labs ‘Pupil Player’ showing the World View camera and the Eye Viewer feed. Adjustments are made on the Eye Viewer camera to find the optimal position with the participant looking towards the PFD screens. Optimal position was deemed to be found when the Confidence bar was consistently above 70 %.

The calibration process consisted of moving a black and white circle marker around the viewing area with the participant tracking the centre of the marker with their eye, as depicted in Fig. 3. The marker is held still at a location on the viewing area while the eye-tracker fixes the position. Within a few seconds an aural beep is heard once the marker position is fixed by eye-tracker. The marker is then moved onto another location and so on, until the approximate area of the viewing area has been covered. This process is repeated for a minimum of nine ‘marks’. Once the calibration process is finished the participant is instructed to look at the centre of the PFD, ND, and FMAs Panel. At each fixation, the tracking of the eye-tracker is verified by visually confirming the feedback of the tracking via red dots, representing fixations, shown on the World View camera. The calibration process ensured accuracy for the specific viewing areas calibrated, but accuracy was found to drop off when participants looked at areas beyond the calibration marks. If participants looked at an area beyond the calibrated areas the data was recorded as ‘Undefined’.

Fig. 3.
figure 5

The calibration process of Pupil Pro is searching for the marker held adjacent to Navigation Display

Full size image

Recording began to capture at least 5–10 s of participant ‘at rest’ data for baseline capture, then a signal to the flight simulator technician is given that the experiment is ready to start. The technician commences the flight with a 3 s countdown. 50 s into each flight an FMAs is switched on by the researcher and the participant response, if any, is noted. At 1 min 30 s into the flight, the FMAs mode is changed via a switch by the researcher and again response is recorded. Mode changes are made at the same time for each participant. At 2 min 30 s the simulation is stopped and the participant told to relax for a few moments. The mode display is switched off, and calibration is checked by having the participant focus on the PFD, ND and mode panel. If calibration is still considered acceptable, by the red dot tracking fixation point within one inch on the World View, the second flight commences with the same format as the first. If calibration been found inaccurate, usually resulting from excessive head movement, the calibration process is repeated and the participant advised not to make large head movements.

3 Result

Six out of 25 participants were discounted due to incomprehensible data expressed as excessive amounts of undefined fixations and low data confidence (below 70 % confidence). Reasons for the incomprehensible data included: excessive head movement by the participant upsetting calibration (3); participant wearing mascara upsetting the eye tracking algorithm and data confidence (1); participant requiring eye correction and unable to wear glasses while wearing the eye-tracker, leading to difficulties reading the flight displays accurately (1); improper fit of tracker due to physical size of participant with very large head, leading to low data confidence (x1). This left 19 validated participants in total for the position design of FMAs on the flight deck, 9 for the PFD, 10 for the MCP position (Table 1).

Table 1. Mean (SD) of eye movement measures for FMAs between PFD and FCP in 4 AOIs
Full size table

4 Discussion

Previous research found mode changes are often missed on the flight deck, and low salience of FMAs – small alphanumeric displays against a dynamic background on the PFD produced in the form of cryptic abbreviations – may be a significant contributor to this. Pilots were familiar with using the MCP to track automation, despite the MCP not being designed for this purpose. A design solution was proposed that moved the FMAs to a glareshield position, taking into account design guidance from EASA and the FAA, as well as several design principles including positioning to increase salience and the PCP. Therefore, current research planned to evaluate this design using objective eye tracking and subjective feedback methods. The results of the experiment found that there were no significance in the fixation counts of FMAs situated on PFD (M = 16.44, SD = 9.62) compared with the MCP (M = 17.30, SD = 10.34). However, a significant difference was found in the mean fixation duration on the FMAs with pilots spending a longer duration on the PFD (M = 337.44, SD = 158.85) than on the MCP (M = 278.80, SD = 133.27). The greater attention allocated to the PFD by the experienced participants through Mean Fixation Duration (MFD) is mirrored in the percentage of total fixations allocated to the PFD (M = 48.28, SD = 32.32), which was also found to be significant larger than MCP (M = 43.08, SD = 27.99).

Based on the observation, the non-experienced participants did not value the information on the PFD as much as the experienced participants when completing the ILS-following task. During the briefing for the experiment participants were instructed to follow the ND lateral and vertical guidance bugs to guide them down the ILS beam, and the PFD was introduced as the display to which they can find their flight parameters such as attitude, airspeed, and altitude. It is likely that the non-experienced participants did not appreciate the importance of monitoring basic flight parameters, and even if they did, they may not have had the background knowledge to understand appropriate airspeeds, descent attitude etc. without specific briefing on these elements. This may have been exacerbated by having the power settings fixed. The lower MFD on the PFD from the non-experienced participants was not correlated with improved ILS beam tracking (deviations noted informally during the experiment), and can be explained by these participants extracting less meaning from the parameters presented. The experienced participants, with a greater resource of knowledge, will have spent many hours maintaining awareness of their flight parameters using the PFD, and so could be expected to give more importance to these parameters, with ‘flying the beam’ as a secondary priority; quite rightly. This reflects previous research where experienced pilots were found to visit more important instruments more often (Bellenkes et al. 1997).

There are two aspects of the cockpit instrument panel to illustrate proximity compatibility principle. The first aspect relates to the layout of the pilots’ most important cockpit instruments. The second aspect of the instrument panel demonstrates how display proximity can be achieved through the actual integration of related information rather than spatial proximity. While the proximity compatibility principle dictates display closeness for information that needs to be integrated, it also dictates that for information channels that do not need to be integrated but should be the sole focus of attention, close proximity (to other information) should be avoided, since such proximity produces unwanted clutter (Wickens and Carswell 1995). Based on current research, the FMAs position had no significant difference on PFD and MCP, however a trend showing an increase of approximately 70 ms duration can be seen for the PFD FMAs position. This is supported by percentage of fixation that showed a trend for increased attention allocation to the PFD (48.28 %) for the than MCP position 38.40 %). Greater time allocated to the PFD may be due to the proximity of relevant instruments, including the distinguishing the modes of FMAs task for the current flight status. The proximity compatibility principle can therefore be used to help reduce attentional demands when comparing information. However, closely-spaced irrelevant information might distract the focus of attention. The Attitude (AOI-2) indicator shown no significant differences of total fixation duration, but pilots demonstrated a trend towards shorter mean fixation duration on the MCP (M = 379, SD = 151.59) position than on the PFD (M = 477, SD = 311.91) position.

Interestingly some participants who had hardly recorded visits to the FMAs panel still correctly called out mode changes. These individuals may have relied on peripheral vision and memory of the three possible modes that could present. There also appears to be a trend towards reduced MFD on the FMAs panel for the MCP position. It was thought MFD would be lower for the MCP FMAs position, on the assumption that a more salient FMAs position would require less time to interpret. Reduced salience of the PFD position may be drawing attention away faster. Participants viewing MCP FMAs are slightly fast on the FMAs due to lower background dynamic activity and the increased salience, this may increase their mode awareness as they afford more time for information extraction. It has been found in a previous study that this position did not have a more detrimental effect on the performance of other concurrent visual tasks (Nikolic and Sarter 2001).

5 Conclusion

With the increase of automation complexity on the flight deck, there has been a corresponding rise in ‘automation surprise’ related incidents and accidents. Lack of mode awareness is a contributing factor to automation surprise. The aviation industry has responded to this through improved training and procedures for operators in order to solve this problem. However, mode related incidents and accidents are still prevalent. Long and costly certification procedures inherent to the safety-critical nature of aviation, dictating an evolutionary rather than revolutionary design process, may explain why an effective, targeted design solution has not yet been introduced. The current design method of FMAs with a green box to highlight mode changes is well recognised in the literature as an imperfect design. Reinforcement of the status quo by certification guidance enables manufacturers to design for airworthiness approval at minimal cost, but given the prevalence of mode-related incidents and accidents, the design guidance may require a critical review. Simply acknowledging in the guidance that the problem is known, while specifically encouraging the known flawed-design solution of current FMAs has not led to more effective designs. Current research has found mode changes are often missed on the flight deck, and pilots prefer using the MCP to track automation, despite the MCP not being designed for this purpose. A design solution was proposed that moved the FMAs to a glareshield position, taking into account design guidance from EASA and the FAA, as well as several design principles including positioning to increase salience and the proximity compatibility principle. The results of the experiment found that FMAs on the MCP did not adversely affect pilot performance and could increase pilot’s SA and reduced the mean fixation duration compared on the PFD position.