My Brain Is Out of the Loop: A Neuroergonomic Approach of OOTL Phenomenon

  • Bruno BerberianEmail author
  • Jonas Gouraud
  • Bertille Somon
  • Aisha Sahai
  • Kevin Le Goff
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10284)


The world surrounding us has become increasingly technological. Nowadays, the influence of automation is perceived in each aspect of everyday life and not only in the world of industry. Automation certainly makes some aspects of life easier, faster and safer. Nonetheless, empirical data suggests that traditional automation has many negative performance and safety consequences. Particularly, in cases of automatic equipment failure, human supervisors seemed effectively helpless to diagnose the situation, determine the appropriate solution and retake control, a set of difficulties called the “out-of-the-loop” (OOL) performance problem. Because automation is not powerful enough to handle all abnormalities, this difficulty in “takeover” is a central problem in automation design.

The OOL performance problem represents a key challenge for both systems designers and human factor society. After decades of research, this phenomenon remains difficult to grasp and treat and recent tragic accidents remind us the difficulty for human operator to interact with highly automated system. The general objective of our research project is to improve our comprehension of the OOL performance problem. To address this issue, we aim (1) to identify the neuro-functional correlates of the OOL performance problem, (2) to propose design recommendations to optimize human-automation interaction and decrease OOL performance problem occurrence. Behavioral data and brain imaging studies will be used to provide a better understanding of this phenomenon at both physiological and psychological levels.


Human Operator Performance Monitoring Human Machine Interface Automation Technology Vigilance Decrement 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This project is supported by an ANR grant (Young researcher program).


  1. 1.
    Kaber, D.B., Onal, E., Endsley, M.R.: Design of automation for telerobots and the effect on performance, operator situation awareness, and subjective workload. Hum. Factors Ergon. Manuf. 10(4), 409–430 (2000)CrossRefGoogle Scholar
  2. 2.
    Parasuraman, R., Sheridan, T.B., Wickens, C.D.: A model for types and levels of human interaction with automation. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 30(3), 286–297 (2000)CrossRefGoogle Scholar
  3. 3.
    Woods, D.D., Tinapple, D.: W3: watching human factors watch people at work. In: Presidential Address, presented at the 43rd Annual Meeting of the Human Factors and Ergonomics Society, Houston, TX, September 1999Google Scholar
  4. 4.
    Endsley, M.R., Kiris, E.O.: The out-of-the-loop performance problem and level of control in automation. Hum. Factors: J. Hum. Factors Ergon. Soc. 37(2), 381–394 (1995)CrossRefGoogle Scholar
  5. 5.
    Merat, N., Jamson, A.H.: The effect of stimulus modality on signal detection: implications for assessing the safety of in-vehicle technology. Hum. Factors: J. Hum. Factors Ergon. Soc. 50(1), 145–158 (2008)CrossRefGoogle Scholar
  6. 6.
    Horberry, T., Stevens, A., Regan, M.A. (eds.): Driver Acceptance of New Technology: Theory, Measurement and Optimisation. Ashgate Publishing Ltd., Farnham (2014)Google Scholar
  7. 7.
    Shneiderman, B., Plaisant, C.: Designing the User Interface: Strategies for Effective Human-Computer Interaction, 4th edn. Pearson Education, London (2004)Google Scholar
  8. 8.
    Baron, S.: Pilot control. In: Wiener, E.L., Nagel, D.C. (eds.) Human Factors in Aviation, pp. 347–386. Academic Press, San Diego, CA (1988)Google Scholar
  9. 9.
    Wiener, E.L.: Cockpit automation. In: Wiener, E.L., Nagel, D.C. (eds.) Human Factors in Aviation, pp. 433–459. Academic Press, San Diego, CA (1988)Google Scholar
  10. 10.
    Wiley, D.: The Coming Collision Between the Automated Instruction and Learning Communities Camps of Online Learning Research. Working Draft. Accessed 22 June 2003Google Scholar
  11. 11.
    Bandura, A., Barbaranelli, C., Caprara, G.V., Pastorelli, C.: Mechanisms of moral disengagement in the exercise of moral agency. J. Pers. Soc. Psychol. 71(2), 364 (1996)CrossRefGoogle Scholar
  12. 12.
    Bratman, M.: Structures of Agency: Essays. Oxford University Press, Oxford (2007)CrossRefGoogle Scholar
  13. 13.
    Borg, J.S., Hynes, C., Van Horn, J., Grafton, S., Sinnott-Armstrong, W.: Consequences, action, and intention as factors in moral judgments: an fMRI investigation. J. Cogn. Neurosci. 18(5), 803–817 (2006)CrossRefGoogle Scholar
  14. 14.
    Bainbridge, L.: Ironies of automation. Automatica 19(6), 775–779 (1983)CrossRefGoogle Scholar
  15. 15.
    Baxter, G., Rooksby, J., Wang, Y., Khajeh-Hosseini, A.: The ironies of automation: still going strong at 30? In: Proceedings of the 30th European Conference on Cognitive Ergonomics, New York, USA, pp. 65–71. ACM (2012)Google Scholar
  16. 16.
    Kaber, D.B., Endsley, M.R.: Out-of-the-loop performance problems and the use of intermediate levels of automation for improved control system functioning and safety. Process Saf. Prog. 16(3), 126–131 (1997)CrossRefGoogle Scholar
  17. 17.
    Moray, N., Inagaki, T.: Attention and complacency. Theoret. Issues Ergon. Sci. 1(4), 354–365 (2000)CrossRefGoogle Scholar
  18. 18.
    Sheridan, T.B., Parasuraman, R.: Human–automation interaction. In: Nickerson, R.S. (ed.) Reviews of Human Factors and Ergonomics, vol. 1, pp. 89–129. Human Factors and Ergonomics Society, Santa Monica (2006)Google Scholar
  19. 19.
    Parasuraman, R., Molloy, R., Singh, I.L.: Performance consequences of automation induced ‘complacency’. Int. J. Aviat. Psychol. 3(1), 1–23 (1993)CrossRefGoogle Scholar
  20. 20.
    Parasuraman, R., Riley, V.: Humans and automation: use, misuse, disuse, abuse. Hum. Factors: J. Hum. Factors Ergon. Soc. 39(2), 230–253 (1997)CrossRefGoogle Scholar
  21. 21.
    Singh, I.L., Molloy, R., Parasuraman, R.: Automation-induced “complacency”: development of the complacency-potential rating scale. Int. J. Aviat. Psychol. 3(2), 111–122 (1993)CrossRefGoogle Scholar
  22. 22.
    Bonini, F., Burle, B., Liégeois-Chauvel, C., Régis, J., Chauvel, P., Vidal, F.: Action monitoring and medial frontal cortex: leading role of supplementary motor area. Science 343(6173), 888–891 (2014)CrossRefGoogle Scholar
  23. 23.
    Ullsperger, M., Fischer, A.G., Nigbur, R., Endrass, T.: Neural mechanisms and temporal dynamics of performance monitoring. Trends Cogn. Sci. 18(5), 259–267 (2014)CrossRefGoogle Scholar
  24. 24.
    Falkenstein, M., Hohnsbein, J., Hoormann, J., Blanke, L.: Effects of cross-modal divided attention on late ERP components. II. Error processing in choice reaction tasks. Electroencephalogr. Clin. Neurophysiol. 78(6), 447–455 (1991)CrossRefGoogle Scholar
  25. 25.
    Berberian, B., Sarrazin, J.C., Le Blaye, P., Haggard, P.: Automation technology and sense of control: a window on human agency. PLoS ONE 7(3), e34075 (2012)CrossRefGoogle Scholar
  26. 26.
    Allain, S., Carbonnell, L., Falkenstein, M., Burle, B., Vidal, F.: The modulation of the Ne-like wave on correct responses foreshadows errors. Neurosci. Lett. 372(1), 161–166 (2004)CrossRefGoogle Scholar
  27. 27.
    Davies, D.R., Parasuraman, R.: The psychology of vigilance. Academic Press, Cambridge (1982)Google Scholar
  28. 28.
    Matthews, G., Davies, D.R.: Individual differences in energetic arousal and sustained attention: a dual-task study. Pers. Individ. Differ. 31(4), 575–589 (2001)CrossRefGoogle Scholar
  29. 29.
    Methot, L.L., Huitema, B.E.: Effects of signal probability on individual differences in vigilance. Hum. Factors: J. Hum. Factors Ergon. Soc. 40(1), 102–110 (1998)CrossRefGoogle Scholar
  30. 30.
    O’Hanlon, J.F.: Boredom: practical consequences and a theory. Acta Psychol. 49(1), 53–82 (1981)CrossRefGoogle Scholar
  31. 31.
    Strauch, B.: Investigating Human Error: Incidents, Accidents, and Complex Systems. Ashgate, Burlington (2002)Google Scholar
  32. 32.
    Broadbent, D.E.: Decision and Stress. Academic Press, New York (1971)Google Scholar
  33. 33.
    Kahneman, D.: Attention and Effort, p. 246. Prentice-Hall, Englewood Cliffs (1973)Google Scholar
  34. 34.
    Coull, J.T.: Neural correlates of attention and arousal: insights from electrophysiology, functional neuroimaging and psychopharmacology. Progress Neurobiol. 55(4), 343–361 (1998)CrossRefGoogle Scholar
  35. 35.
    Oken, B.S., Salinsky, M.C., Elsas, S.M.: Vigilance, alertness, or sustained attention: physiological basis and measurement. Clin. Neurophysiol. 117(9), 1885–1901 (2006)CrossRefGoogle Scholar
  36. 36.
    Mason, M.F., Norton, M.I., Van Horn, J.D., Wegner, D.M., Grafton, S.T., Macrae, C.N.: Wandering minds: the default network and stimulus-independent thought. Science 315(5810), 393–395 (2007)CrossRefGoogle Scholar
  37. 37.
    Smallwood, J., Schooler, J.W.: The restless mind. Psychol. Bull. 132(6), 946 (2006)CrossRefGoogle Scholar
  38. 38.
    Giambra, L.M.: A laboratory method for investigating influences on switching attention to task-unrelated imagery and thought. Conscious. Cogn. 4(1), 1–21 (1995)CrossRefGoogle Scholar
  39. 39.
    Smallwood, J., Beach, E., Schooler, J.W., Handy, T.C.: Going AWOL in the brain: mind wandering reduces cortical analysis of external events. J. Cogn. Neurosci. 20(3), 458–469 (2008)CrossRefGoogle Scholar
  40. 40.
    Braboszcz, C., Delorme, A.: Lost in thoughts: neural markers of low alertness during mind wandering. Neuroimage 54(4), 3040–3047 (2011)CrossRefGoogle Scholar
  41. 41.
    Daniel, R.S.: Alpha and Theta EEC in vigilance. Percept. Mot. Skills 25(3), 697–703 (1967)CrossRefGoogle Scholar
  42. 42.
    Duffy, F.H., Albert, M.S., McAnulty, G., Garvey, A.J.: Age-related differences in brain electrical activity of healthy subjects. Ann. Neurol. 16(4), 430–438 (1984)CrossRefGoogle Scholar
  43. 43.
    Berka, C., Levendowski, D.J., Lumicao, M.N., Yau, A., Davis, G., Zivkovic, V.T., Olmstead, R.E., Tremoulet, P.D., Craven, P.L.: EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat. Space Environ. Med. 78(1), B231–B244 (2007)Google Scholar
  44. 44.
    Jung, T.P., Makeig, S., Stensmo, M., Sejnowski, T.J.: Estimating alertness from the EEG power spectrum. IEEE Trans. Biomed. Eng. 44(1), 60–69 (1997)CrossRefGoogle Scholar
  45. 45.
    Goldinger, S.D., Papesh, M.H.: Pupil dilation reflects the creation and retrieval of memories. Curr. Dir. Psychol. Sci. 21(2), 90–95 (2012)CrossRefGoogle Scholar
  46. 46.
    Verney, S.P., Granholm, E., Marshall, S.P.: Pupillary responses on the visual backward masking task reflect general cognitive ability. Int. J. Psychophysiol. 52(1), 23–36 (2004)CrossRefGoogle Scholar
  47. 47.
    Wierda, S.M., van Rijn, H., Taatgen, N.A., Martens, S.: Pupil dilation deconvolution reveals the dynamics of attention at high temporal resolution. Proc. Nat. Acad. Sci. 109(22), 8456–8460 (2012)CrossRefGoogle Scholar
  48. 48.
    Endsley, M.R.: Automation and situation awareness. In: Parasuraman, R., Mouloua, M. (eds.) Automation and Human Performance: Theory and Applications, pp. 163–181. Lawrence Erlbaum, Mahwah (1996)Google Scholar
  49. 49.
    Bahner, J.E., Hüper, A.D., Manzey, D.: Misuse of automated decision aids: complacency, automation bias and the impact of training experience. Int. J. Hum.-Comput. Stud. 66(9), 688–699 (2008)CrossRefGoogle Scholar
  50. 50.
    Plat, M., Amalberti, R.: Experimental crew training to deal with automation surprises. In: Cognitive Engineering in the Aviation Domain, pp. 287–308. Lawrence Erlbaun Associates, Hillsdale (NJ) (2000)Google Scholar
  51. 51.
    Sheridan, T.B., Verplank, W.L.: Human and Computer Control of Undersea Teleoperators (1978)Google Scholar
  52. 52.
    Endsley, M.R.: The application of human factors to the development of expert systems for advanced cockpits. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 31, no. 12, pp. 1388–1392. SAGE Publications (1987)Google Scholar
  53. 53.
    Endsley, M.R.: Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics 42(3), 462–492 (1999)CrossRefGoogle Scholar
  54. 54.
    Kaber, D.B., Endsley, M.R.: The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task. Theoret. Issues Ergon. Sci. 5(2), 113–153 (2004)CrossRefGoogle Scholar
  55. 55.
    Dekker, S.W., Woods, D.D.: MABA-MABA or abracadabra? Progress on human–automation coordination. Cogn. Technol. Work 4(4), 240–244 (2002)CrossRefGoogle Scholar
  56. 56.
    Sarter, N.B., Woods, D.D., Billings, C.E.: Automation surprises. In: Handbook of Human Factors and Ergonomics, vol. 2, pp. 1926-1943 (1997)Google Scholar
  57. 57.
    Norman, D.A.: The ‘problem’ with automation: inappropriate feedback and interaction, not ‘over automation’. Philos. Trans. R. Soc. B: Biol. Sci. 327(1241), 585–593 (1990)CrossRefGoogle Scholar
  58. 58.
    Hollnagel, E.: From function allocation to function congruence. In: Dekker, S.W.A., Hollnagel, E. (eds.) Coping with computers in the cockpit, pp. 29– 53 (1999)Google Scholar
  59. 59.
    Christoffersen, K., Woods, D.D.: How to make automated systems team players. Adv. Hum. Perform. Cogn. Eng. Res. 2, 1–12 (2002)CrossRefGoogle Scholar
  60. 60.
    Klein, G., Woods, D.D., Bradshaw, J.M., Hoffman, R.R., Feltovich, P.J.: Ten challenges for making automation a “team player” in joint human-agent activity. IEEE Intell. Syst. 19(6), 91–95 (2004)CrossRefGoogle Scholar
  61. 61.
    Hoc, J.M.: Human and automation: a matter of cooperation. In: HUMAN 2007, pp. 277–285. Université de Metz (2007)Google Scholar
  62. 62.
    Hoc, J.M., Carlier, X.: Role of a common frame of reference in cognitive cooperation: sharing tasks between agents in air traffic control. Cogn. Technol. Work 4(1), 37–47 (2002)CrossRefGoogle Scholar
  63. 63.
    Dragan, A.D., Lee, K.C., Srinivasa, S.S.: Legibility and predictability of robot motion. In: 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 301–308. IEEE, March 2013Google Scholar
  64. 64.
    Zimmermann, M., Bauer, S., Lutteken, N., Rothkirch, I.M., Bengler, K.J.: Acting together by mutual control: evaluation of a multimodal interaction concept for cooperative driving. In: 2014 International Conference on Collaboration Technologies and Systems (CTS), pp. 227–235. IEEE, May 2014Google Scholar
  65. 65.
    Pacherie, E.: The sense of control and the sense of agency. Psyche 13(1), 1–30 (2007)Google Scholar
  66. 66.
    Gallagher, S.: Philosophical conceptions of the self: implications for cognitive science. Trends Cogn. Sci. 4(1), 14–21 (2000)CrossRefGoogle Scholar
  67. 67.
    Blakemore, S.J., Wolpert, D.M., Frith, C.D.: Abnormalities in the awareness of action. Trends Cogn. Sci. 6(6), 237–242 (2002)CrossRefGoogle Scholar
  68. 68.
    Frith, C.D., Blakemore, S.J., Wolpert, D.M.: Explaining the symptoms of schizophrenia: abnormalities in the awareness of action. Brain Res. Rev. 31(2), 357–363 (2000)CrossRefGoogle Scholar
  69. 69.
    Wegner, D.M.: The Illusion of Conscious Will. MIT press, Cambridge (2002)Google Scholar
  70. 70.
    Moore, J.W., Obhi, S.S.: Intentional binding and the sense of agency: a review. Conscious. Cogn. 21(1), 546–561 (2012)CrossRefGoogle Scholar
  71. 71.
    Taleb, N.N.: Antifragile: Things that Gain from Disorder. Random House Incorporated, New York (2012)Google Scholar
  72. 72.
    Pacherie, E.: Action (2012)Google Scholar
  73. 73.
    Wegner, D.M., Wheatley, T.: Apparent mental causation: sources of the experience of will. Am. Psychol. 54(7), 480 (1999)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Bruno Berberian
    • 1
    Email author
  • Jonas Gouraud
    • 1
  • Bertille Somon
    • 1
  • Aisha Sahai
    • 1
  • Kevin Le Goff
    • 2
  1. 1.French Aerospace Lab (ONERA)SalonFrance
  2. 2.Airbus GroupToulouseFrance

Personalised recommendations