Gaze-Aware Cognitive Assistant for Multiscreen Surveillance

  • Sébastien TremblayEmail author
  • Daniel Lafond
  • Cindy Chamberland
  • Helen M. Hodgetts
  • François Vachon
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 722)


Surveillance operators must scan multiple camera feeds to ensure timely detection of incidents; however, variability in scanning behavior can lead to untimely/failed detection of critical information in feeds that were neglected for a long period. Using an eye tracker to monitor screen fixations we can calculate (in real-time) the time elapsed since the last scan of each particular feed, allowing the setting-up of targeted countermeasures contingent on operator oculomotor behavior. One avenue is to provide operators with timely alerts to modulate the scan pattern to avoid attentional tunneling and inattentional blindness. We test such an adaptive solution within a major event surveillance simulation and preliminary results show that operator scan behavior can be modulated, although further investigation is required to determine warning frequency and modality to optimize the balance between saliency and workload increase. Future work will focus on adding a real-time vigilance detection and countermeasure capability.


Security surveillance Human-computer interaction Cognitive support Eye tracking Adaptive intelligent system 



This work was supported by grants from the National Sciences and Engineering Research Council of Canada (NSERC), the Ministère de l’économie, des sciences et de l’innovation du Québec (MESI), and Prompt Information Technology R&D awarded to Sébastien Tremblay and François Vachon. We are also grateful to the financial and in-kind contributions of Thales Research and Technology Canada.


  1. 1.
    Mack, A., Rock, I.: Inattentional Blindness. MIT Press, Cambridge (1998)Google Scholar
  2. 2.
    Imbert, J.P., Hodgetts, H.M., Parise, P.R., Vachon, F., Dehais, F., Tremblay, S.: Attentional costs and failures in air traffic control notifications. Ergonomics 57, 1817–1832 (2014)CrossRefGoogle Scholar
  3. 3.
    Vallières, B., Hodgetts, H.M., Vachon, F., Tremblay, S.: Supporting dynamic change detection: using the right tool for the task. Cogn. Res. Principles Implications 1, 32 (2016)CrossRefGoogle Scholar
  4. 4.
    Troscianko, T., Holmes, A., Stillman, J., Mirmehdi, M., Wright, D., Wilson, A.: What happens next? The predictability of natural behaviour viewed through CCTV cameras. Perception 33, 87–101 (2004)CrossRefGoogle Scholar
  5. 5.
    Monsell, S.: Task switching. Trends Cogn. Sci. 7, 134–140 (2003)CrossRefGoogle Scholar
  6. 6.
    Valera, M., Velastin, S.A.: Intelligent distributed surveillance systems: a review. IEEE Proc. Vis. Image Sign. Process. 152, 192–204 (2005)CrossRefGoogle Scholar
  7. 7.
    Dadashi, N., Stedmon, A.W., Pridmore, T.P.: Semi-automated CCTV surveillance: the effects of system confidence, system accuracy and task complexity on operator vigilance, reliance and workload. Appl. Erg. 44, 730–738 (2003)CrossRefGoogle Scholar
  8. 8.
    Vachon, F., Vallières, B.R., Suss, J., Thériault, J.-D., Tremblay, S.: The CSSS microworld: a gateway to understanding and improving CCTV security surveillance. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 265–269. SAGE publications (2016)Google Scholar
  9. 9.
    Atrey, P.K., El Saddick, A., Kankanhalli, M.S.: Effective multimedia surveillance using a human-centric approach. Multimedia Tools Appl. 51, 697–721 (2011)CrossRefGoogle Scholar
  10. 10.
    Blignaut, P.: Fixation identification: the optimum threshold for a dispersion algorithm. Attention Percept. Psychol. 71, 881–895 (2009)CrossRefGoogle Scholar
  11. 11.
    Vachon, F., Vallières, B.R., Jones, D.M., Tremblay, S.: Nonexplicit change detection in complex dynamic settings: what eye movements reveal. Hum. Factors 54, 996–1007 (2012)CrossRefGoogle Scholar
  12. 12.
    Steelman, K.S., McCarley, J.S., Wickens, C.D.: Modeling the control of attention in visual workspaces. Hum. Factors 53, 142–153 (2011)CrossRefGoogle Scholar
  13. 13.
    Taylor, P., Bilgrien, N., He, Z., Siegelmann, H.T.: EyeFrame: real-time memory aid improves human multitasking via domain-general eye tracking procedures. Front. ICT 2, 17 (2015)Google Scholar
  14. 14.
    Hodgetts, H.M., Tremblay, S., Vallieres, B., Vachon, F.: Decision support and vulnerability to interruption in a dynamic multitasking environment. Int. J. Hum. Comput. Stud. 79, 106–117 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Sébastien Tremblay
    • 1
    Email author
  • Daniel Lafond
    • 2
  • Cindy Chamberland
    • 1
  • Helen M. Hodgetts
    • 1
    • 3
  • François Vachon
    • 1
  1. 1.Université LavalQuebec CityCanada
  2. 2.Thales Research and TechnologyQuebec CityCanada
  3. 3.Cardiff Metropolitan UniversityCardiffUK

Personalised recommendations