Understanding Situation Awareness Development Processes Through Self-confrontation Interviews Based on Eye-Tracking Videos

Abstract

The importance of Situation Awareness (SA) for effective decision making and safety in risky and complex socio-technical systems is closely linked to the methods used to evaluate it. Many of the existing methods are based primarily on measures of SA quality. However, in the context of the design of a training program to improve the SA skills of teams operating nuclear submarines, it became apparent that it was necessary to develop a method to account for the cognitive processes underlying the development of SA. To achieve this, we held self-confrontation interviews with operators, who were asked to explicit their own actions through eye-tracking video recordings. These interviews allowed operators to verbalize their mental representations and reasoning, while the eye-tracking video highlighted the allocation of visual attention. This method has been put in practice with eight submarine team leaders and revealed several points of interest. First, it makes it possible to take into account Endsley’s three levels of SA: the eye-tracking video addresses the capture of visual information, while the operator’s explanation captures information from other sensory channels as well as their understanding and expectations of the situation. Furthermore, the risk of memory lapses is overcome by eye-tracking video anchoring; data reliability is improved by the supplementary explanations provided by the operator; and data are enriched by access to operators’ emotional experience. This method provides some interesting insights, especially in the context of professional training design.