Keywords

1 Introduction

Human operators generally face a complex, dynamic and uncertain environment under time pressure. The occurrence of unexpected events (e.g., critical failure) requires flexibility and cognitive regulation policies to meet task demand (Sperandio 1978). Various strategies may be employed by humans to achieve adaptation. In a study of the physiological effects of a kinetically adaptive environment, Jager et al. (2017) describe the reciprocal relationship between adaptive humans and such environments. Schwarz and Fuchs (2017) point out that “humans are adaptive systems themselves”, that is, they are able to mitigate critical user states by applying self-regulation strategies. They cite as examples “investing more effort if task demands increase or drinking coffee to combat fatigue”. Additionally, the system itself can be made to adapt to the human.

Some examples of the integration of simultaneous human and system adaptation are aimed at psychophysiological goal achievement, not necessarily at the immediate achievement of optimal performance. Biocybernetic adaptation has been employed as a self-regulation training method for application in clinical and sports settings (Pope et al. 2014). In these technologies, the adaptation approach involves physiological signals modulating some aspects of the training tasks in such a way as to reward trainees for approaching a target signal. These physiological self-regulation training technologies are designed to improve adherence to a training regimen by delivering the training through engaging, motivating, and entertaining experiences. The processing employed in these technologies is minimal to enable real-time feedback. Likewise, the decision rules are usually simple, e.g., modulating a single task element based upon signal level. For instance, some consequences in a digital game or simulation reward the user for achieving a psychophysiological goal by diminishing an undesirable effect in a game (analogous to negative reinforcement). Other consequences reward the user for achieving a psychophysiological goal by producing a desirable effect (analogous to positive reinforcement) such as additional scoring opportunities. That is, some modulation effects enable superimposed disadvantages in a digital game or simulation to be reduced by progression toward a psychophysiological goal, whereas others enable advantages to be effected by progression toward a psychophysiological goal.

Schmorrow proposes a system that adapts to a trainee’s level in the context of flight training: “Imagine an aviation recruit experiencing a simulator that is tailored to the trainee at the most fundamental neurophysiological level. Imagine that this simulator’s integrated helmet and sensor suite are hooked up to a ‘black box’ that modifies the simulated flight exercise based on a real-time assessment of the student pilot’s cognitive state, using information collected by the sensor suite.” (Schmorrow 2005). Similarly, the task modulation concept embodied in the self-regulation training technology based on biocybernetic adaptation may be adapted for use in task simulators. The simulator embodiment of the closed-loop modulation concept, Stress Counter-response Training (Palsson and Pope 1999), integrates physiological self-regulation training into the practice of mission-relevant tasks. Stress Counter-response Training is based upon the concept of instrument functionality feedback which ties the functionality of a simulator to the requirement to maintain the physiological equanimity suited for optimal cognitive and motor performance under emergency events in an airplane cockpit.

In these technologies, the physiological modulation method is tailored to the overall game or simulation task, but without regard for changes in the task context or other situational factors. Fuchs and Schwarz (2017) identify this as a “hard-coded” adaptation strategy, where the system triggers a predetermined adaptation strategy. As will be shown, even more complex adaptation strategies have considerations in common with the simple self-regulation training strategy.

2 Biocybernetic Loop Implementation for Adaptive Systems

A first and important step for biocybernetic adaptation is to determine what temporal and magnitude changes in physiological signals reflect operator or trainee state changes that warrant mitigation (Fairclough and Gilleade 2013). Indeed, one important concern with the implementation of such assisting systems is to succeed in providing assistance in a timely and appropriate manner (Parasuraman et al. 1999). Spurious triggering of the assistance system may have negative consequences on human operators (Parasuraman et al. 1997). Therefore, an approach is to target mental states that are (1) relevant predictors of human performance and (2) that can be robustly identified via behavioral and neurophysiological measures. Mental states of interest are discussed, followed by a description of the biocybernetic adaptation pipeline.

Traditionally, most of the research has focused on mental workload-based biocybernetic adaptation. However, the usability of the mental workload construct remains limited. Although theoretically and practically interesting, it remains ill-defined (Mandrick et al. 2016), providing a non-specific and generic index rather like a thermometer. Moreover, mental workload should not be viewed as the result of an external demand applied on an individual passively adapting to it, but rather as an active process that depends on the human operator’s level of engagement. For instance, a highly demanding situation will not necessarily induce high workload if an individual does not engage to achieve. Several reasons may account for this lack of engagement such as excessive task difficulty (Durantin et al. 2014), repetitive and boring tasks (Durantin et al. 2015) and cognitive fatigue (Hopstaken et al. 2015). Conversely, over-engagement in a non-priority and non-demanding task could induce high workload (e.g., interacting with the entertainment system or texting while driving) and jeopardize safety (Lee 2014; Dehais et al. 2012). Thus, human cognitive performance has to be considered the byproduct of the level of task demand by the level of task engagement. Interestingly, the concept of engagement is related to a triad of attentional states: attentional disengagement, attentional over-engagement, and attentional in-engagement. Also, the study of engagement is richer than the concept of workload: this concept accounts for neurophysiological and behavioral phenomena and it can be characterized with portable measurement tools (Verdiere et al. 2018). For example, a biocybernetic system was designed to mitigate task disengagement due to automation by triggering changes in task mode based on the fluctuations of an engagement index constructed as a ratio of EEG band powers (Scerbo et al. 2000). Derivation of the engagement index was based on the proposition that the closed-loop paradigm that represents the adaptive configuration in which physiological indices are to have a steering role can also serve as a prior validation test bed for the indices themselves (Pope et al. 1995).

Firstly, attentional disengagement occurs when task demand is too low leading to episodes of mind wandering (Durantin et al. 2015) or when task demand exceeds mental capacity. In these two extreme situations, human operators generally drop the primary task to focus on automatic secondary tasks. These two states are characterized by the disengagement of the executive network, underpinned by the deactivation of the dorsolateral prefrontal cortex (Durantin et al. 2014; Harrivel et al. 2013). Secondly, attentional over-engagement, also referred to attentional tunneling (Wickens 2005) and “channelized attention” (Harrivel et al. 2016), is defined as “the allocation of attention to a particular channel of information, diagnostic hypothesis or task goal, for a duration that is longer than optimal, given the expected cost of neglecting events on other channels, failing to consider other hypotheses, or failing to perform other tasks”. Some authors postulate that this impaired attentional state results from a disengagement deficit of the orientation network underpinned by the thalamus (LaBerge et al. 1992). Whereas the assessment of such brain structure remains difficult to be perform in operational context - it requires the use of fMRI – some studies have disclosed that attentional over-engagement is associated with an attentional shrinking and long fixation time (Dehais et al. 2011). Recently, the EEG engagement index proposed by Pope et al. (1995) was shown to be sensitive to episodes of over-engagement leading to inattentional deafness to auditory alarm under real-flight settings (Dehais et al. 2014).

Lastly, recent work has shown the existence of an attentional in-engagement state whereby human operators are unable to engage their attention to process relevant information when facing critical situations. One could describe this state as “panic mode” in a vernacular fashion. This state, that is the exact opposite of attentional tunneling, is explained in terms of impaired thalamus tonic mode to maintain focused attention. This state of “attentional confusion” or “attentional entropy” is associated with high saccadic activity and absence of long fixations (Dehais et al. 2015).

Another interesting approach could be to identify the dynamic model of such features. Tools derived from the linear algebra and control communities can be applied to perform an approximation of the neurophysiological features model that could be explored to monitor the engagement of an operator. The method provides a smooth interpolation of all the data points enabling the extraction of frequency features that reveal fluctuations in engagement with growing time-on-task (Poussot-Vassal et al. 2017). Alternatively, the use of large-scale EEG connectivity is a relevant approach not only to detect but also to predict future performance and fluctuation of engagement (Senoussi et al. 2017).

The implementation of the biocybernetic adaptation pipeline mostly consists of the classical steps of a Brain-Computer Interface, that is to say a signal acquisition step (e.g., EEG), a preprocessing step that generally deals with artifacts (e.g., eye blinks) and better conditions the signal, a feature extraction step (e.g., extraction of the average power in specific frequency bands), a machine learning step (e.g., a classification step), and lastly an adaptation step (Roy and Frey 2016). This last step can consist of providing the estimated mental state to the system’s decisional unit. The decisional unit system allows the loop to be closed. This is done by implementing a decisional unit driven by a policy resulting from the resolution of a (Partially Observable) Markov Decision Process ((PO)MDP) that takes into consideration uncertainties on actions, partial observable states (i.e., mental states) or potentially non-deterministic behavior of the human operator (Gateau et al. 2016; Drougard et al. 2017). Eventually, a last step is to design a catalogue of adaptive solutions to mitigate decline in performance and improve human performance. These solutions are presented in the next section.

3 Successful Implementation of Adaptive Solutions

Self-regulation training can be deliberate as described earlier or could occur inadvertently as a result of an operator’s exposure to an adaptive system. Technology in the field of self-regulation training has commonly taken into account the fact that the physiological self-regulation behavior and skill of the trainee changes as training progresses. These systems have incorporated algorithms that respond to momentary, transient changes in physiological signals in real time, as well as longer time course changes that reflect a trainee’s emerging ability to voluntarily control physiological parameters. The momentary changes are displayed as information and reward feedback for learning of self-regulation skill, while the longer time course measurements are assessed to guide the setting of higher and higher self-regulation performance goals.

An early example is an electromyographic biofeedback training system that implemented a shaping procedure by adjusting the gain of the feedback loop after each interval of training based on a trainee’s success at lowering EMG levels (Pope and Gersten 1977). This system employed a fixed strategy by which task characteristics are adapted to the individual. A training strategy implies a set of assertions relating strategy characteristics and their effects on training progress. In a more advanced implementation, a data base of these assertions could be updated on-line and the training system would be self-improving. In effect, the system would evaluate the results of mini-experiments with various strategy versions within a session and modify the strategy accordingly. O’Shea and Sleeman (1973) developed this hierarchical framework in the context of adaptive teaching systems.

Similarly, physiologically adaptive systems will need to be designed to respond appropriately not only to transient changes and spontaneous drifts in operator state due to developing conditions such as fatigue, but also to conditioning of physiological changes as a result of an operator’s extended exposure to information feedback about their physiological state. Accordingly, an adaptive implementation that took into consideration the operator “training” effect of its information feedback employed a continually updated model of the operator analogous to the “template of average performance” in the “symbionic cockpit” (Reising and Moss 1985). Techniques developed for adapting a brain-computer interface classifier to adjust for possible features drift could be applied to address this type of consequence (Vidaurre et al. 2011). Configuration of an adaptive system that takes into consideration these long-term and short-term processes is depicted in Fig. 1.

Fig. 1.
figure 1

Configuration of an adaptive system for managing human user engagement. The Desired Short-Term Engagement Criteria and Long Term Engagement Trends vary according to context (e.g., phase of flight). The Adjustable System has adjustable parameters, such as automation level. The Adjustable System is adjusted by the currently invoked Short-Term Adaptation Strategy and is driven by input from the human user. The Selectable Short-Term Adaptation Strategy has a catalog of selectable strategies, each one of which is in effect for a selectable duration. The currently invoked Strategy is selected by the Long-Term Adaptation Scheme and is driven by the discrepancy between the desired engagement criteria and actual engagement status. The Long-Term Adaptation Scheme is driven by the discrepancy between the desired and actual engagement trend. The Achieved Engagement Output is the level of an engagement index derived from physiological and behavioral measures and is driven by the effects of the Adjustable System on the Human User.

Full size image

An additional strategy is the deliberate exercise of self-regulation skill acquired as a result of self-regulation training. Prinzel et al. (2002) demonstrated that participants given feedback of the accuracy of their estimates of engagement levels, across a partitioning of the range of an EEG-based engagement index into six subranges, were able to achieve “a 70% level of correct identifications.” Further, while interacting with an adaptive automation system, “Participants in the self-regulation condition were better able to maintain their task engagement level within a narrower range of task modes, thereby reducing the need for task mode changes. The effect of this was an increase in task performance as well as a decrease in reported workload.” Prinzel et al. (2002) further comment, “The neurofeedback provided during training may have allowed these participants to better manage their cognitive resources and thereby regulate their engagement state, allowing them to better respond to a change in automation mode. The results of this study support other research that has shown that physiological self-regulation could enhance the cognitive resource management skills of pilots and complement the benefits of adaptive automation.” An outcome of self-regulation training accomplished with an adaptive system is improved cognitive state management skill which is effectively meta-awareness on the part of the trainee. This effect demonstrates an observing system as defined in second order cybernetics (von Foerester 1995).

In addition to clinical and sports applications, biocybernetic adaptation as self-regulation training is applied in a third area, the aircrew training context (Stephens et al. 2017). In this application, the instructor-trainee interaction is influenced, closing the loop on a broader time scale. Here the adaptation involves an attention management training approach to complement the usual observations of airline training instructor pilots by informing them, in the training context, of the occurrence of attention-related human performance limiting states (AHPLS) experienced by their trainees. Classifier models are trained to recognize trainee state during simulated flight scenarios based on patterns of the physiological signals measured during benchmark tasks (Harrivel et al. 2016). Machine learning models’ real time determinations of the cognitive states induced by the scenario tasks are displayed as gauges embedded in a mosaic of windows that also displays real time images of the scenario tasks that the trainee is performing (e.g., scene camera, simulator displays, animation of simulator controls), and this mosaicFootnote 1 is video recorded (Harrivel et al. 2017). The loop is closed when their state information is conveyed to the trainee as part of each session debrief. This approach involving trainee-trainer interaction leverages the effective bio-social influences on learning specified by Kamiya (Strehl 2014). Like the adaptive automation application, the adaptation strategy here takes into consideration contextual parameters such as the instructor’s discretion regarding the appropriateness of conveying particular state information to the trainee.

This psychophysiologically-based AHPLS detection and mitigation system is modeled after the Hypoxia Familiarization Training (HFT) employed in aviation. The focus of HFT is on recognizing symptoms of hypoxia and taking steps to recover from the hypoxia being experienced. Similarly, recognition and recovery from AHPLS is intended to improve self-monitoring of and response to one’s own attentional performance, maintaining more effective states and managing attention. Such meta-awareness results from this form of self-regulation training intended to develop attention management skill. If deployed in ground-based commercial aviation training contexts, the intent is to mitigate potential in-flight loss of airplane state awareness (ASA) and thus reduce aviation accidents and incidents.

Biocybernetic adaptation can be applied within autonomous systems to imbue further intelligence into the systems about the humans involved in operations. In a potential adaptive automation application, the cognitive state of the operator of a semi-autonomous vehicle would be tracked by the vehicle system. The system uses the cognitive state information to judge the operator’s ability to take back control of the system in critical or noncritical hand-off instancesFootnote 2.

4 Conclusion

The implementations of specific adaptations described in this paper represent actual systems designed for improving human/machine interaction and furthermore enabling human operators to improve self-regulation skills. The adaptation strategies described herein include combinations of technological advances in the areas of neuroscience and psychophysiology designed for specific contexts including clinical, aviation, and sports. The example implementation systems instantiate concepts and enable practical and empirical testing to evaluate adaptation strategies. Adaptation management issues were discussed including dynamic selection and configuration of adaptations. Development of adaptation strategies can create further questions for consideration such as how to handle possible side effects on the human operator caused by setting up a biocybernetic loop. This and other questions require empirical results to be sufficiently addressed. Ongoing research efforts at NASA and the Institut Supérieur de l’Aéronautique et de l’Espace (ISAE) seek to apply adaptation strategies to answer these questions and reveal further questions with the ultimate goals of improved safety and efficiency in aerospace operations.