Task (over-)load imposed on surgeons is a main contributing factor to surgical errors. Recent research has shown that gaze metrics represent a valid and objective index to asses operator task load in non-surgical scenarios. Thus, gaze metrics have the potential to improve workplace safety by providing accurate measurements of task load variations. However, the direct relationship between gaze metrics and surgical task load has not been investigated yet. We studied the effects of surgical task complexity on the gaze metrics of surgical trainees.
We recorded the eye movements of 18 surgical residents, using a mobile eye tracker system, during the performance of three high-fidelity virtual simulations of laparoscopic exercises of increasing complexity level: Clip Applying exercise, Cutting Big exercise, and Translocation of Objects exercise. We also measured performance accuracy and subjective rating of complexity.
Gaze entropy and velocity linearly increased with increased task complexity: Visual exploration pattern became less stereotyped (i.e., more random) and faster during the more complex exercises. Residents performed better the Clip Applying exercise and the Cutting Big exercise than the Translocation of Objects exercise and their perceived task complexity differed accordingly.
Our data show that gaze metrics are a valid and reliable surgical task load index. These findings have potential impacts to improve patient safety by providing accurate measurements of surgeon task (over-)load and might provide future indices to assess residents’ learning curves, independently of expensive virtual simulators or time-consuming expert evaluation.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Khuri SF, Henderson WG, Daley J, Jonasson O, Jones RS, Campbell DA Jr, Fink AS, Mentzer RM Jr, Steeger JE (2007) The patient safety in surgery study: background, study design, and patient populations. J Am Coll Surg 204:1089–1102. doi:10.1016/j.jamcollsurg.2007.03.028
Pham JC, Aswani MS, Rosen M, Lee H, Huddle M, Weeks K, Pronovost PJ (2012) Reducing medical errors and adverse events. Annu Rev Med 63:447–463. doi:10.1146/annurev-med-061410-121352
Graafland M, Schraagen JMC, Boermeester MA, Bemelman WA, Schijven MP (2015) Training situational awareness to reduce surgical errors in the operating room. Br J Surg 102:16–23. doi:10.1002/bjs.9643
Yurko YY, Scerbo MW, Prabhu AS, Acker CE, Stefanidis D (2010) Higher mental workload is associated with poorer laparoscopic performance as measured by the NASA-TLX tool. Simul Healthc J Soc Simul Healthc 5:267–271. doi:10.1097/SIH.0b013e3181e3f329
Marks J, Tacchino R, Roberts K, Onders R, Denoto G, Paraskeva P, Rivas H, Soper N, Rosemurgy A, Shah S (2011) Prospective randomized controlled trial of traditional laparoscopic cholecystectomy versus single-incision laparoscopic cholecystectomy: report of preliminary data. Am J Surg 201:369–373. doi:10.1016/j.amjsurg.2010.09.012
Islam A, Castellvi AO, Tesfay ST, Castellvi AD, Wright AS, Scott DJ (2011) Early surgeon impressions and technical difficulty associated with laparoendoscopic single-site surgery: a Society of American Gastrointestinal and Endoscopic Surgeons learning center study. Surg Endosc 25:2597–2603. doi:10.1007/s00464-011-1594-4
Canes D, Desai MM, Aron M, Haber G-P, Goel RK, Stein RJ, Kaouk JH, Gill IS (2008) Transumbilical single-port surgery: evolution and current status. Eur Urol 54:1020–1030. doi:10.1016/j.eururo.2008.07.009
Park A, Lee G, Seagull FJ, Meenaghan N, Dexter D (2010) Patients benefit while surgeons suffer: an impending epidemic. J Am Coll Surg 210:306–313. doi:10.1016/j.jamcollsurg.2009.10.017
Zheng B, Tien G, Atkins SM, Swindells C, Tanin H, Meneghetti A, Qayumi KA, Neely O, Panton M (2011) Surgeon’s vigilance in the operating room. Am J Surg 201:673–677. doi:10.1016/j.amjsurg.2011.01.016
Amalberti R, Vincent C, Auroy Y, de Maurice GS (2006) Violations and migrations in health care: a framework for understanding and management. Q Saf Health Care 15:i66–i71. doi:10.1136/qshc.2005.015982
Ziv A, Wolpe PR, Small SD, Glick S (2006) Simulation-based medical education: an ethical imperative. Simul Healthc J Soc Simul Healthc 1:252–256. doi:10.1097/01.SIH.0000242724.08501.63
Martin JA, Regehr G, Reznick R, Macrae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278. doi:10.1046/j.1365-2168.1997.02502.x
Hart SG (2006) NASA-task load index (NASA-TLX); 20 years later. In: Proceedings of the human factors and ergonomics society annual meeting, pp 904–908
Yiannakopoulou E, Nikiteas N, Perrea D, Tsigris C (2015) Virtual reality simulators and training in laparoscopic surgery. Int J Surg 13:60–64. doi:10.1016/j.ijsu.2014.11.014
Di Stasi LL, Renner R, Catena A, Cañas JJ, Velichkovsky BM, Pannasch S (2012) Towards a driver fatigue test based on the saccadic main sequence: a partial validation by subjective report data. Transp Res C Emerg Technol 21:122–133. doi:10.1016/j.trc.2011.07.002
D’Angelo A-LD, Rutherford DN, Ray RD, Laufer S, Kwan C, Cohen ER, Mason A, Pugh CM (2015) Idle time: an underdeveloped performance metric for assessing surgical skill. Am J Surg 209:645–651. doi:10.1016/j.amjsurg.2014.12.013
Sodergren M, McGregor C, Farne HA, Cao J, Lv Z, Purkayastha S, Athanasiou T, Darzi A, Paraskeva P (2013) A randomised comparative study evaluating learning curves of novices in a basic single-incision laparoscopic surgery task. J Gastrointest Surg Off J Soc Surg Aliment Tract 17:569–575. doi:10.1007/s11605-012-2113-1
Tien T, Pucher PH, Sodergren MH, Sriskandarajah K, Yang G-Z, Darzi A (2014) Eye tracking for skills assessment and training: a systematic review. J Surg Res 191:169–178. doi:10.1016/j.jss.2014.04.032
Di Stasi LL, McCamy MB, Macknik SL, Mankin JA, Hooft N, Catena A, Martinez-Conde S (2014) Saccadic eye movement metrics reflect surgical residentsʼ fatigue. Ann Surg 259:824–829. doi:10.1097/SLA.0000000000000260
Causer J, McCormick SA, Holmes PS (2013) Congruency of gaze metrics in action, imagery and action observation. Front Hum Neurosci. doi:10.3389/fnhum.2013.00604
Tien T, Pucher PH, Sodergren MH, Sriskandarajah K, Yang G-Z, Darzi A (2015) Differences in gaze behaviour of expert and junior surgeons performing open inguinal hernia repair. Surg Endosc Interv Tech 29:405–413. doi:10.1007/s00464-014-3683-7
Di Stasi LL, Catena A, Cañas JJ, Macknik SL, Martinez-Conde S (2013) Saccadic velocity as an arousal index in naturalistic tasks. Neurosci Biobehav Rev 37:968–975. doi:10.1016/j.neubiorev.2013.03.011
Di Stasi LL, Marchitto M, Antolí A, Cañas JJ (2013) Saccadic peak velocity as an alternative index of operator attention: a short review. Rev Eur Psychol Appliquée European Rev Appl Psychol 63:335–343. doi:10.1016/j.erap.2013.09.001
Krupinski EA, Tillack AA, Richter L, Henderson JT, Bhattacharyya AK, Scott KM, Graham AR, Descour MR, Davis JR, Weinstein RS (2006) Eye-movement study and human performance using telepathology virtual slides. Implications for medical education and differences with experience. Hum Pathol 37:1543–1556. doi:10.1016/j.humpath.2006.08.024
Alzubaidi M, Patel A, Panchanathan S, Black J, John A (2010) Reading a radiologist’s mind: monitoring rising and falling interest levels while scanning chest X-rays. In: Proceedings of the SPIE international society for optical engineering. San Diego, CA, USA, pp 0F1–0F10
Richstone L, Schwartz MJ, Seideman C, Cadeddu J, Marshall S, Kavoussi LR (2010) Eye metrics as an objective assessment of surgical skill. Ann Surg 252:177–182. doi:10.1097/SLA.0b013e3181e464fb
Cao Y, Miura S, Kobayashi Y, Kawamura K, Sugano S, Fujie M (2016) Pupil variation applied to the eye tracking control of an endoscopic manipulator. IEEE Robot Autom Lett 1–1. doi:10.1109/LRA.2016.2521894
Tien G, Zheng B, Atkins MS (2011) Quantifying surgeons’ vigilance during laparoscopic operations using eyegaze tracking. Stud Health Technol Inform 163:658–662
Eivazi S, Bednarik R, Tukiainen M, von und zu Fraunberg M, Leinonen V, Jääskeläinen JE (2012) Gaze behaviour of expert and novice microneurosurgeons differs during observations of tumor removal recordings. In: Proceedings of the symposium on eye tracking research and applications. New York, NY, USA, pp 377–380
Atkins MS, Tien G, Khan R, Meneghetti A, Zheng B (2012) What do surgeons see: capturing and synchronizing eye gaze for surgery applications. Surg Innov. doi:10.1177/1553350612449075
Giovinco NA, Sutton SM, Miller JD, Rankin TM, Gonzalez GW, Najafi B, Armstrong DG (2015) A passing glance? Differences in eye tracking and gaze patterns between trainees and experts reading plain film bunion radiographs. J Foot Ankle Surg 54:382–391. doi:10.1053/j.jfas.2014.08.013
Allsop J, Gray R (2014) Flying under pressure: effects of anxiety on attention and gaze behavior in aviation. J Appl Res Mem Cogn 3:63–71. doi:10.1016/j.jarmac.2014.04.010
Tole JR, Stephens AT, Harris RL, Ephrath AR (1982) Visual scanning behavior and mental workload in aircraft pilots. Aviat Space Environ Med 53:54–61
Di Nocera F, Camilli M, Terenzi M (2007) A random glance at the flight deck: Pilots’ scanning strategies and the real-time assessment of mental workload. J Cogn Eng Decis Mak 1:271–285
Camilli M, Nacchia R, Terenzi M, Di Nocera F (2008) ASTEF: a simple tool for examining fixations. Behav Res Methods 40:373–382
Zheng B, Jiang X, Atkins MS (2015) Detection of changes in surgical difficulty: evidence from pupil responses. Surg Innov. doi:10.1177/1553350615573582
Beatty J, Lucero-Wagoner B (2000) The pupillary system. In: Cacioppo JT, Tassinary LG, Berntson GG (eds) Handbook of psychophysiology, 2nd edn. Cambridge University Press, New York, pp 142–162
Bradley MM, Miccoli L, Escrig MA, Lang PJ (2008) The pupil as a measure of emotional arousal and autonomic activation. Psychophysiology 45:602–607. doi:10.1111/j.1469-8986.2008.00654.x
Morales JM, Di Stasi LL, Diaz-Piedra C, Morillas C, Romero S (2015) Real-time monitoring of biomedical signals to improve road safety. Springer, Palma de Mallorca, pp 89–97
Szulewski A, Howes D (2014) Combining first-person video and gaze-tracking in medical simulation: a technical feasibility study. Sci World J 2014:e975752. doi:10.1155/2014/975752
WMA (1964) Human experimentation: code of ethics of WMA. Br Med J 2:177
Kössi J, Luostarinen M (2009) Virtual reality laparoscopic simulator as an aid in surgical resident education: two years’ experience. Scand J Surg 98:48–54. doi:10.1177/145749690909800109
Hoddes E, Zarcone V, Smythe H, Phillips R, Dement WC (1973) Quantification of sleepiness: a new approach. Psychophysiology 10:431–436
Connor J, Norton R, Ameratunga S, Robinson E, Civil I, Dunn R, Bailey J, Jackson R (2002) Driver sleepiness and risk of serious injury to car occupants: population based case control study. BMJ 324:1125
Di Stasi LL, Diaz-Piedra C, Suárez J, McCamy MB, Martinez-Conde S, Roca-Dorda J, Catena A (2015) Task complexity modulates pilot electroencephalographic activity during real flights. Psychophysiology. doi:10.1111/psyp.12419
Wilson MR, McGrath JS, Vine SJ, Brewer J, Defriend D, Masters RSW (2011) Perceptual impairment and psychomotor control in virtual laparoscopic surgery. Surg Endosc 25:2268–2274. doi:10.1007/s00464-010-1546-4
Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Hum Ment Workload 1:139–183
Corker HP, Singh P, Sodergren MH, Balaji S, Kwasnicki RM, Darzi AW, Paraskeva P (2015) A randomized controlled study to establish the effect of articulating instruments on performance in single-incision laparoscopic surgery. J Surg Educ 72:1–7. doi:10.1016/j.jsurg.2014.08.004
Fransen SAF, Mertens LS, Botden SMBI, Stassen LPS, Bouvy ND (2011) Performance curve of basic skills in single-incision laparoscopy versus conventional laparoscopy: Is it really more difficult for the novice? Surg Endosc 26:1231–1237. doi:10.1007/s00464-011-2041-2
Borg G (1998) Borg’s perceived exertion and pain scales. Human Kinetics Publishers, Champaign
Wickens CD, McCarley JS (2007) Applied attention theory. CRC Press, Boca Raton
Schiller PH, Tehovnik EJ (2005) Neural mechanisms underlying target selection with saccadic eye movements. Prog Brain Res 149:157–171. doi:10.1016/S0079-6123(05)49012-3
Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:379–423. doi:10.1002/j.1538-7305.1948.tb01338.x
Leigh RJ, Thurston SE, Tomsak RL, Grossman GE, Lanska DJ (1989) Effect of monocular visual loss upon stability of gaze. Invest Ophthalmol Vis Sci 30:288–292
Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65–70
Watson RA (2013) Quantification of surgical technique using an inertial measurement unit. Simul Healthc J Soc Simul Healthc 8:162–165. doi:10.1097/SIH.0b013e318277803a
Yerkes RM, Dodson JD (1908) The relation of strength of stimulus to rapidity of habit-formation. J Comp Neurol Psychol 18:459–482. doi:10.1002/cne.920180503
Di Stasi LL, McCamy MB, Pannasch S, Renner R, Catena A, Cañas JJ, Velichkovsky BM, Martinez-Conde S (2015) Effects of driving time on microsaccadic dynamics. Exp Brain Res 233:599–605. doi:10.1007/s00221-014-4139-y
Tramper JJ, Lamont A, Flanders M, Gielen S (2013) Gaze is driven by an internal goal trajectory in a visuomotor task. Eur J Neurosci 37:1112–1119. doi:10.1111/ejn.12107
Najemnik J, Geisler WS (2005) Optimal eye movement strategies in visual search. Nature 434:387–391. doi:10.1038/nature03390
McCamy MB, Otero-Millan J, Di Stasi LL, Macknik SL, Martinez-Conde S (2014) Highly informative natural scene regions increase microsaccade production during visual scanning. J Neurosci 34:2956–2966. doi:10.1523/JNEUROSCI.4448-13.2014
Gottlieb J, Oudeyer P-Y, Lopes M, Baranes A (2013) Information-seeking, curiosity, and attention: computational and neural mechanisms. Trends Cogn Sci 17:585–593. doi:10.1016/j.tics.2013.09.001
Jarodzka H, Balslev T, Holmqvist K, Nyström M, Scheiter K, Gerjets P, Eika B (2012) Conveying clinical reasoning based on visual observation via eye-movement modelling examples. Instr Sci 40:813–827. doi:10.1007/s11251-012-9218-5
Papenmeier F, Huff M (2010) DynAOI: a tool for matching eye-movement data with dynamic areas of interest in animations and movies. Behav Res Methods 42:179–187. doi:10.3758/BRM.42.1.179
We thank the Tobii Group for the EyeTrackAwards Stipend awarded to LLDS, and the IAVANTE staff (Andalusian Public Foundation for Progress and Health) for their help during the data collection. This study was funded by the Campus of International Excellence (BioTic Granada) Research Programme (Research Project V7-2015 to CDP). The project was approved by the Talentia Postdoc Program launched by the Andalusian Knowledge Agency, co-funded by the European Commission’s 7th Framework Programme for Research and Technological Development—Marie Skłodowska-Curie actions—and the Andalusian Department of Economy, Innovation, Science and Employment (COFUND—Grant Agreement No. 267226 to LLDS). Research by LLDS is funded by the BBVA Foundation Program for Research, Innovation, and Cultural Creation (Grant No. 2015-2). CDP is supported by a UGR Postdoctoral Fellowship (2013 University of Granada Research Plan). Research by AC is funded by a Spanish Ministry of Economy and Competitiveness grant (PSI2012-39292 to AC). Research by LLDS, CDP, and AC is funded by a Spanish Department of Transportation grant (SPIP2014-1426 to LLDS).
Leandro L. Di Stasi, Carolina Diaz-Piedra, Héctor Rieiro, José M. Sánchez Carrión, Mercedes Martin Berrido, Gonzalo Olivares, and Andrés Catena have no conflicts of interest or financial ties to disclose.
Leandro L. Di Stasi and Carolina Diaz-Piedra have contributed equally to this work.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Di Stasi, L.L., Diaz-Piedra, C., Rieiro, H. et al. Gaze entropy reflects surgical task load. Surg Endosc 30, 5034–5043 (2016). https://doi.org/10.1007/s00464-016-4851-8
- Eye metrics
- Surgical skills assessment
- Patient safety