Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Objective assessment of surgical skill transfer using non-invasive brain imaging

  • 287 Accesses

  • 2 Citations

Abstract

Background

Physical and virtual surgical simulators are increasingly being used in training technical surgical skills. However, metrics such as completion time or subjective performance checklists often show poor correlation to transfer of skills into clinical settings. We hypothesize that non-invasive brain imaging can objectively differentiate and classify surgical skill transfer, with higher accuracy than established metrics, for subjects based on motor skill levels.

Study design

18 medical students at University at Buffalo were randomly assigned into control, physical surgical trainer, or virtual trainer groups. Training groups practiced a surgical technical task on respective simulators for 12 consecutive days. To measure skill transfer post-training, all subjects performed the technical task in an ex-vivo environment. Cortical activation was measured using functional near-infrared spectroscopy (fNIRS) in the prefrontal cortex, primary motor cortex, and supplementary motor area, due to their direct impact on motor skill learning.

Results

Classification between simulator trained and untrained subjects based on traditional metrics is poor, where misclassification errors range from 20 to 41%. Conversely, fNIRS metrics can successfully classify physical or virtual trained subjects from untrained subjects with misclassification errors of 2.2% and 8.9%, respectively. More importantly, untrained subjects are successfully classified from physical or virtual simulator trained subjects with misclassification errors of 2.7% and 9.1%, respectively.

Conclusion

fNIRS metrics are significantly more accurate than current established metrics in classifying different levels of surgical motor skill transfer. Our approach brings robustness, objectivity, and accuracy in validating the effectiveness of future surgical trainers in translating surgical skills to clinically relevant environments.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    Dawe SR, Windsor JA, Broeders JAJL, Cregan PC, Hewett PJ, Maddern GJ (2014) A systematic review of surgical skills transfer after simulation-based training. Ann Surg 259:236–248. https://doi.org/10.1097/SLA.0000000000000245

  2. 2.

    Fraser SA, Klassen DR, Feldman LS, Ghitulescu GA, Stanbridge D, Fried GM (2003) Evaluating laparoscopic skills: Setting the pass/fail score for the MISTELS system. Surg Endosc 17:964–967. https://doi.org/10.1007/s00464-002-8828-4

  3. 3.

    Fraser SA, Feldman LS, Stanbridge D, Fried GM (2005) Characterizing the learning curve for a basic laparoscopic drill. Surg Endosc 19:1572–1578. https://doi.org/10.1007/s00464-005-0150-5

  4. 4.

    Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240:518–525. https://doi.org/10.1097/01.SLA.0000136941.46529.56

  5. 5.

    McCluney AL, Vassiliou MC, Kaneva PA, Cao J, Stanbridge DD, Feldman LS, Fried GM (2007) FLS simulator performance predicts intraoperative laparoscopic skill. Surg Endosc 21:1991–1995. https://doi.org/10.1007/s00464-007-9451-1

  6. 6.

    Scott DJ, Ritter EM, Tesfay ST, Pimentel EA, Nagji A, Fried GM (2008) Certification pass rate of 100% for fundamentals of laparoscopic surgery skills after proficiency-based training. Surg Endosc 22:1887–1893. https://doi.org/10.1007/s00464-008-9745-y

  7. 7.

    Satava RM (2010) Emerging trends that herald the future of surgical simulation. Surg Clin North Am 90:623–633. https://doi.org/10.1016/j.suc.2010.02.002

  8. 8.

    Fried GM (2008) FLS assessment of competency using simulated laparoscopic tasks. J Gastrointest Surg 12:210–212. https://doi.org/10.1007/s11605-007-0355-0

  9. 9.

    Poulose BK, Vassiliou MC, Dunkin BJ, Mellinger JD, Fanelli RD, Martinez JM, Hazey JW, Sillin LF, Delaney CP, Velanovich V, Fried GM, Korndorffer JR, Marks JM (2014) Fundamentals of endoscopic surgery cognitive examination: development and validity evidence. Surg Endosc 28:631–638. https://doi.org/10.1007/s00464-013-3220-0

  10. 10.

    Vassiliou MC, Dunkin BJ, Fried GM, Mellinger JD, Trus T, Kaneva P, Lyons C, Korndorffer JR, Ujiki M, Velanovich V, Kochman ML, Tsuda S, Martinez J, Scott DJ, Korus G, Park A, Marks JM (2014) Fundamentals of endoscopic surgery: creation and validation of the hands-on test. Surg Endosc 28:704–711. https://doi.org/10.1007/s00464-013-3298-4

  11. 11.

    McDougall EM (2007) Validation of surgical simulators. J Endourol 21:244–247. https://doi.org/10.1089/end.2007.9985

  12. 12.

    Scott DJ, Pugh CM, Ritter EM, Jacobs LM, Pellegrini CA, Sachdeva AK (2011) New directions in simulation-based surgical education and training: validation and transfer of surgical skills, use of nonsurgeons as faculty, use of simulation to screen and select surgery residents, and long-term follow-up of learners. Surgery 149:735–744. https://doi.org/10.1016/j.surg.2010.11.010

  13. 13.

    Aggarwal R, Grantcharov T, Moorthy K, Milland T, Papasavas P, Dosis A, Bello F, Darzi A (2007) An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 245:992–999. https://doi.org/10.1097/01.sla.0000262780.17950.e5

  14. 14.

    Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113. https://doi.org/10.1016/j.amjsurg.2005.04.004

  15. 15.

    Doyle JD, Webber EM, Sidhu RS (2007) A universal global rating scale for the evaluation of technical skills in the operating room. Am J Surg 193:551–555. https://doi.org/10.1016/j.amjsurg.2007.02.003

  16. 16.

    Dawe SR, Pena GN, Windsor JA, Broeders JAJL, Cregan PC, Hewett PJ, Maddern GJ (2014) Systematic review of skills transfer after surgical simulation-based training. Br J Surg 101:1063–1076. https://doi.org/10.1002/bjs.9482

  17. 17.

    Moorthy K, Munz Y (2003) Objective assessment of technical skills in surgery. Br Med J 327:1032–1037. https://doi.org/10.1136/bmj.327.7422.1032

  18. 18.

    Hogle NJ, Chang L, Strong VEM, Welcome AOU, Sinaan M, Bailey R, Fowler DL (2009) Validation of laparoscopic surgical skills training outside the operating room: a long road. Surg Endosc 23:1476–1482. https://doi.org/10.1007/s00464-009-0379-5

  19. 19.

    Bosecker C, Dipietro L, Volpe B, Krebs HI (2010) Kinematic robot-based evaluation scales and clinical counterparts to measure upper limb motor performance in patients with chronic stroke. Neurorehabil Neural Repair 24:62–69. https://doi.org/10.1177/1545968309343214

  20. 20.

    Buckley CE, Kavanagh DO, Traynor O, Neary PC (2014) Is the skillset obtained in surgical simulation transferable to the operating theatre? Am J Surg 207:146–157. https://doi.org/10.1016/j.amjsurg.2013.06.017

  21. 21.

    Dai X, Zhang T, Yang H, Tang J, Carney PR, Jiang H (2017) Fast non-invasive functional diffuse optical tomography for brain imaging. J Biophotonics. https://doi.org/10.1002/jbio.201600267

  22. 22.

    Leff DR, Orihuela-Espina F, Elwell CE, Athanasiou T, Delpy DT, Darzi AW, Yang G-Z (2011) Assessment of the cerebral cortex during motor task behaviours in adults: a systematic review of functional near infrared spectroscopy (fNIRS) studies. Neuroimage 54:2922–2936. https://doi.org/10.1016/j.neuroimage.2010.10.058

  23. 23.

    Leff DR, Elwell CE, Orihuela-Espina F, Atallah L, Delpy DT, Darzi AW, Yang GZ (2008) Changes in prefrontal cortical behaviour depend upon familiarity on a bimanual co-ordination task: an fNIRS study. Neuroimage 39:805–813. https://doi.org/10.1016/j.neuroimage.2007.09.032

  24. 24.

    James DRC, Orihuela-Espina F, Leff DR, Sodergren MH, Athanasiou T, Darzi AW, Yang G-Z (2011) The ergonomics of natural orifice translumenal endoscopic surgery (NOTES) navigation in terms of performance, stress, and cognitive behavior. Surgery 149:525–533. https://doi.org/10.1016/j.surg.2010.11.019

  25. 25.

    Ohuchida K, Kenmotsu H, Yamamoto A, Sawada K, Hayami T, Morooka K, Takasugi S, Konishi K, Ieiri S, Tanoue K, Iwamoto Y, Tanaka M, Hashizume M (2009) The frontal cortex is activated during learning of endoscopic procedures. Surg Endosc 23:2296–2301. https://doi.org/10.1007/s00464-008-0316-z

  26. 26.

    Shewokis PA, Shariff FU, Liu Y, Ayaz H, Castellanos A, Lind DS (2017) Acquisition, retention and transfer of simulated laparoscopic tasks using fNIR and a contextual interference paradigm. Am J Surg 213:336–345. https://doi.org/10.1016/j.amjsurg.2016.11.043

  27. 27.

    Andreu-Perez J, Leff DR, Shetty K, Darzi A, Yang G-Z (2016) Disparity in frontal lobe connectivity on a complex bimanual motor task aids in classification of operator skill level. Brain Connect 6:375–388. https://doi.org/10.1089/brain.2015.0350

  28. 28.

    Modi HN, Singh H, Yang G-Z, Darzi A, Leff DR, Mansur A, Yang AE, Darzi G-Z, Leff A DR (2017) A decade of imaging surgeons’ brain function (part I): terminology, techniques, and clinical translation. Surgery 28:2189–2198. https://doi.org/10.1016/j.surg.2017.05.021

  29. 29.

    Nemani A, Kruger U, Intes X, De S (2017) Increased sensitivity in discriminating surgical motor skills using prefrontal cortex activation over established metrics. In: Optics in the life sciences congress. OSA, San Diego, p 11

  30. 30.

    Soper NJ, Fried GM (2008) The fundamentals of laparoscopic surgery: its time has come. Bull Am Coll Surg 93:30–32

  31. 31.

    Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, Schirmer B, Hoffman K, the SAGES FLS Committee the SF (2004) Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery 135:21–27. https://doi.org/10.1016/S0039-6060(03)00156-9

  32. 32.

    Zhang L, Sankaranarayanan G, Arikatla VS, Ahn W, Grosdemouge C, Rideout JM, Epstein SK, De S, Schwaitzberg SD, Jones DB, Cao CGL (2013) Characterizing the learning curve of the VBLaST-PT(©) (Virtual basic laparoscopic skill trainer). Surg Endosc 27:3603–3615. https://doi.org/10.1007/s00464-013-2932-5

  33. 33.

    Maciel A, Liu Y, Ahn W, Singh TP, Dunnican W, De S (2008) Development of the VBLaST: a virtual basic laparoscopic skill trainer. Int J Med Robot Comput Assist Surg 4:131–138. https://doi.org/10.1002/rcs.185

  34. 34.

    Arikatla VS, Sankaranarayanan G, Ahn W, Chellali A, De S, Caroline GL, Hwabejire J, DeMoya M, Schwaitzberg S, Jones DB (2013) Face and construct validation of a virtual peg transfer simulator. Surg Endosc 27:1721–1729. https://doi.org/10.1007/s00464-012-2664-y

  35. 35.

    Sankaranarayanan G, Lin H, Arikatla VS, Mulcare M, Zhang L, Derevianko A, Lim R, Fobert D, Cao C, Schwaitzberg SD, Jones DB, De S (2010) Preliminary face and construct validation study of a virtual basic laparoscopic skill trainer. J Laparoendosc Adv Surg Tech 20:153–157. https://doi.org/10.1089/lap.2009.0030

  36. 36.

    Chellali A, Ahn W, Sankaranarayanan G, Flinn JT, Schwaitzberg SD, Jones DB, De S, Cao CGL (2015) Preliminary evaluation of the pattern cutting and the ligating loop virtual laparoscopic trainers. Surg Endosc 29:815–821. https://doi.org/10.1007/s00464-014-3764-7

  37. 37.

    Nemani A, Ahn W, Cooper C, Schwaitzberg S, De S Convergent validation and transfer of learning studies of a virtual reality-based pattern cutting simulator. Surg Endosc. https://doi.org/10.1007/s00464-017-5802-8

  38. 38.

    Hikosaka O, Nakamura K, Sakai K, Nakahara H (2002) Central mechanisms of motor skill learning. Curr Opin Neurobiol 12:217–222. https://doi.org/10.1016/S0959-4388(02)00307-0

  39. 39.

    Wolpert DM, Diedrichsen J, Flanagan JR (2011) Principles of sensorimotor learning. Nat Rev Neurosci 12:739–751. https://doi.org/10.1038/nrn3112

  40. 40.

    Swinnen SP (2002) Intermanual coordination: from behavioural principles to neural-network interactions. Nat Rev Neurosci 3:348–359. https://doi.org/10.1038/nrn807

  41. 41.

    Swinnen SP, Gooijers J (2015) Bimanual Coordination. In: Brain mapp, 1st edn. Elsevier, San Diego, pp 475–482

  42. 42.

    Swinnen SP, Wenderoth N (2004) Two hands, one brain: cognitive neuroscience of bimanual skill. Trends Cogn Sci 8:18–25

  43. 43.

    Nemani A, Intes X, De S (2014) Monte Carlo based simulation of sensitivity curvature for evaluating optimal probe geometry. In: Biomedical optics 2014. OSA, Washington, D.C., p BM3A.36

  44. 44.

    Cope M, Delpy DT (1988) System for long-term measurement of cerebral blood and tissue oxygenation on newborn infants by near infra-red transillumination. Med Biol Eng Comput 26:289–294. https://doi.org/10.1007/BF02447083

  45. 45.

    Delpy DT, Cope M, Zee P, van der Arridge S, Wray S, Wyatt J (1988) Estimation of optical pathlength through tissue from direct time of flight measurement. Phys Med Biol 33:1433–1442. https://doi.org/10.1088/0031-9155/33/12/008

  46. 46.

    Huppert TJ, Diamond SG, Franceschini MA, Boas DA (2009) Homer: a review of time-series analysis methods for near-infrared spectroscopy of the brain. Appl Opt 48:280–298. https://doi.org/10.1364/AO.48.00D280

  47. 47.

    Franceschini MA, Joseph DK, Huppert TJ, Diamond SG, Boas DA (2006) Diffuse optical imaging of the whole head. J Biomed Opt 11:1–22. https://doi.org/10.1117/1.2363365

  48. 48.

    Zhang Y, Brooks DH, Franceschini MA, Boas DA (2005) Eigenvector-based spatial filtering for reduction of physiological interference in diffuse optical imaging. J Biomed Opt 10:1–11. https://doi.org/10.1117/1.1852552

  49. 49.

    Gagnon L, Yücel MA, Boas DA, Cooper RJ (2014) Further improvement in reducing superficial contamination in NIRS using double short separation measurements. Neuroimage 85:127–135. https://doi.org/10.1016/j.neuroimage.2013.01.073

  50. 50.

    Tabachnick BG, Fidell LS (2007) Using multivariate statistics, 5th edn. Allyn & Bacon, Needham Heights

  51. 51.

    Silverman BW (1986) Density estimation for statistics and data analysis. CRC press, Boca Raton

  52. 52.

    Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193:797–804. https://doi.org/10.1016/j.amjsurg.2006.06.050

  53. 53.

    Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–463. https://doi.org/10.1097/01.SLA.0000028969.51489.B4

  54. 54.

    Ahlberg G, Hultcrantz R, Jaramillo E, Lindblom A, Arvidsson D (2005) Virtual reality colonoscopy simulation: a compulsory practice for the future colonoscopist? Endoscopy 37:1198–1204. https://doi.org/10.1055/s-2005-921049

  55. 55.

    Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91:146–150. https://doi.org/10.1002/bjs.4407

  56. 56.

    Sakai K, Hikosaka O, Miyauchi S, Takino R, Sasaki Y, Pütz B (1998) Transition of brain activation from frontal to parietal areas in visuomotor sequence learning. J Neurosci 18:1827–1840

  57. 57.

    Honda M (1998) Dynamic cortical involvement in implicit and explicit motor sequence learning: a PET study. Brain 121:2159–2173. https://doi.org/10.1093/brain/121.11.2159

  58. 58.

    Nakamura K, Sakai K, Hikosaka O (1998) Neuronal activity in medial frontal cortex during learning of sequential procedures. J Neurophysiol 80:2671–2687

  59. 59.

    Klintsova AY, Greenough WT (1999) Synaptic plasticity in cortical systems. Curr Opin Neurobiol 9:203–208

  60. 60.

    Wolpert DM, Ghahramani Z, Flanagan JR (2001) Perspectives and problems in motor learning. Trends Cogn Sci 5:487–494. https://doi.org/10.1016/S1364-6613(00)01773-3

  61. 61.

    Serrien DJ, Ivry RB, Swinnen SP (2006) Dynamics of hemispheric specialization and integration in the context of motor control. Nat Rev Neurosci 7:160–166. https://doi.org/10.1038/nrn1849

  62. 62.

    Gagnon L, Cooper RJ, Yücel MA, Perdue KL, Greve DN, Boas DA (2012) Short separation channel location impacts the performance of short channel regression in NIRS. Neuroimage 59:2518–2528. https://doi.org/10.1016/j.neuroimage.2011.08.095

Download references

Acknowledgements

The authors would like to thank the medical student subjects and their dedication for this study. The authors would also like to thank the anatomical gift program and the gross anatomy lab at University at Buffalo for their support regarding the ex-vivo cadaveric samples. We would also like to thank Arthur “Buzz” DiMartino and his team at TechEn in graciously providing us support with the CW6 spectrometer.

Funding

This work is supported by NIBIB 1R01EB014305, NHBLI 1R01HL119248, and NCI 1R01CA197491 grants that were awarded to Suvranu De.

Author information

Correspondence to Suvranu De.

Ethics declarations

Disclosures

Drs. Arun Nemani, Uwe Kruger, Meryem Yucel, Clairice Cooper, Steven Schwaitzberg, Xavier Intes, and Suvranu De have no conflicts of interest or financial ties to disclose.

Additional information

Presented at ACS Clinical Congress 2017, San Diego, CA.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nemani, A., Kruger, U., Cooper, C.A. et al. Objective assessment of surgical skill transfer using non-invasive brain imaging. Surg Endosc 33, 2485–2494 (2019). https://doi.org/10.1007/s00464-018-6535-z

Download citation

Keywords

  • Surgical skill assessment
  • Surgical skill transfer
  • Brain imaging
  • Surgical simulators
  • Surgical training
  • Functional near-infrared spectroscopy