Advertisement

Surgical Endoscopy

, Volume 32, Issue 3, pp 1265–1272 | Cite as

Convergent validation and transfer of learning studies of a virtual reality-based pattern cutting simulator

  • Arun Nemani
  • Woojin Ahn
  • Clairice Cooper
  • Steven Schwaitzberg
  • Suvranu De
Article

Abstract

Introduction

Research has clearly shown the benefits of surgical simulators to train laparoscopic motor skills required for positive patient outcomes. We have developed the Virtual Basic Laparoscopic Skill Trainer (VBLaST) that simulates tasks from the Fundamentals of Laparoscopic Surgery (FLS) curriculum. This study aims to show convergent validity of the VBLaST pattern cutting module via the CUSUM method to quantify learning curves along with motor skill transfer from simulation environments to ex vivo tissue samples.

Methods

18 medical students at the University at Buffalo, with no prior laparoscopic surgical skills, were placed into the control, FLS training, or VBLaST training groups. Each training group performed pattern cutting trials for 12 consecutive days on their respective simulation trainers. Following a 2-week break period, the trained students performed three pattern cutting trials on each simulation platform to measure skill retention. All subjects then performed one pattern cutting task on ex vivo cadaveric peritoneal tissue. FLS and VBLaST pattern cutting scores, CUSUM scores, and transfer task completion times were reported.

Results

Results indicate that the FLS and VBLaST trained groups have significantly higher task performance scores than the control group in both the VBLaST and FLS environments (p < 0.05). Learning curve results indicate that three out of seven FLS training subjects and four out of six VBLaST training subjects achieved the “senior” performance level. Furthermore, both the FLS and VBLaST trained groups had significantly lower transfer task completion times on ex vivo peritoneal tissue models (p < 0.05).

Conclusion

We characterized task performance scores for trained VBLaST and FLS subjects via CUSUM analysis of the learning curves and showed evidence that both groups have significant improvements in surgical motor skill. Furthermore, we showed that learned surgical skills in the FLS and VBLaST environments transfer not only to the different simulation environments, but also to ex vivo tissue models.

Keywords

Learning curve Virtual surgical simulation Surgical training Surgical performance metrics Surgical skill transfer Laparoscopy 

Notes

Acknowledgements

This work is supported by NIBIB 1R01EB014305, NHLBI 1R01HL119248, and NCI 1R01CA197491 Grants awarded to Suvranu De. The authors would like to thank the medical student subjects and their dedication for this study. The authors would also like to thank the anatomical gift program and the gross anatomy lab at University of Buffalo for their support regarding the ex vivo cadaveric samples.

Compliance with ethical standards

Disclosures

Drs. Arun Nemani, Woojin Ahn, Clairice Cooper, Steven Schwaitzberg, and Suvranu De have no conflict of interest or financial ties to disclose.

References

  1. 1.
    Darzi A, Smith S, Taffinder N (1999) Assessing operative skill. Needs to become more objective. BMJ 318:887–888CrossRefPubMedPubMedCentralGoogle Scholar
  2. 2.
    Wanzel KR, Hamstra SJ, Anastakis DJ, Matsumoto ED, Cusimano MD (2002) Effect of visual-spatial ability on learning of spatially-complex surgical skills. Lancet 359:230–231. doi: 10.1016/S0140-6736(02)07441-X CrossRefPubMedGoogle Scholar
  3. 3.
    Aggarwal R, Grantcharov T, Moorthy K, Milland T, Papasavas P, Dosis A, Bello F, Darzi A (2007) An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 245:992–999. doi: 10.1097/01.sla.0000262780.17950.e5 CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113. doi: 10.1016/j.amjsurg.2005.04.004 CrossRefPubMedGoogle Scholar
  5. 5.
    Doyle JD, Webber EM, Sidhu RS (2007) A universal global rating scale for the evaluation of technical skills in the operating room. Am J Surg 193:551–555. doi: 10.1016/j.amjsurg.2007.02.003 CrossRefPubMedGoogle Scholar
  6. 6.
    Martin JA, Regehr G, Reznick R, Macrae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278. doi: 10.1046/j.1365-2168.1997.02502.x CrossRefPubMedGoogle Scholar
  7. 7.
    Hogle NJ, Chang L, Strong VEM, Welcome AOU, Sinaan M, Bailey R, Fowler DL (2009) Validation of laparoscopic surgical skills training outside the operating room: a long road. Surg Endosc 23:1476–1482. doi: 10.1007/s00464-009-0379-5 CrossRefPubMedGoogle Scholar
  8. 8.
    Moorthy K, Munz Y (2003) Objective assessment of technical skills in surgery. Br Med J 327:1032–1037. doi: 10.1136/bmj.327.7422.1032 CrossRefGoogle Scholar
  9. 9.
    Fraser SA, Klassen DR, Feldman LS, Ghitulescu GA, Stanbridge D, Fried GM (2003) Evaluating laparoscopic skills: setting the pass/fail score for the MISTELS system. Surg Endosc 17:964–967. doi: 10.1007/s00464-002-8828-4 CrossRefPubMedGoogle Scholar
  10. 10.
    Soper NJ, Fried GM (2008) The fundamentals of laparoscopic surgery: its time has come. Bull Am Coll Surg 93:30–32PubMedGoogle Scholar
  11. 11.
    Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, Schirmer B, Hoffman K (2004) Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery 135:21–27. doi: 10.1016/S0039-6060(03)00156-9 CrossRefPubMedGoogle Scholar
  12. 12.
    Fried GM (2008) FLS assessment of competency using simulated laparoscopic tasks. J Gastrointest Surg 12:210–212. doi: 10.1007/s11605-007-0355-0 CrossRefPubMedGoogle Scholar
  13. 13.
    Feldman LS, Sherman V, Fried GM (2004) Using simulators to assess laparoscopic competence: ready for widespread use? Special section: competency-when, why, how? Surgery 135:28–42CrossRefPubMedGoogle Scholar
  14. 14.
    Feldman LS, Hagarty SE, Ghitulescu G, Stanbridge D, Fried GM (2004) Relationship between objective assessment of technical skills and subjective in-training evaluations in surgical residents. J Am Coll Surg 198:105–110. doi: 10.1016/j.jamcollsurg.2003.08.020 CrossRefPubMedGoogle Scholar
  15. 15.
    Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240:518–525. doi: 10.1097/01.SLA.0000136941.46529.56 CrossRefPubMedPubMedCentralGoogle Scholar
  16. 16.
    Vassiliou MC, Ghitulescu GA, Feldman LS, Stanbridge D, Leffondré K, Sigman HH, Fried GM (2006) The MISTELS program to measure technical skill in laparoscopic surgery: evidence for reliability. Surg Endosc 20:744–747. doi: 10.1007/s00464-005-3008-y CrossRefPubMedGoogle Scholar
  17. 17.
    Zhang L, Sankaranarayanan G, Arikatla VS, Ahn W, Grosdemouge C, Rideout JM, Epstein SK, De S, Schwaitzberg SD, Jones DB, Cao CGL (2013) Characterizing the learning curve of the VBLaST-PT© (Virtual Basic Laparoscopic Skill Trainer). Surg Endosc 27:3603–3615. doi: 10.1007/s00464-013-2932-5 CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–463. doi: 10.1097/01.SLA.0000028969.51489.B4 CrossRefPubMedPubMedCentralGoogle Scholar
  19. 19.
    Gallagher AG, Ritter EM, Champion H, Higgins G, Fried MP, Moses G, Smith CD, Satava RM (2005) Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 241:364–372CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Maciel A, Liu Y, Ahn W, Singh TP, Dunnican W, De S (2008) Development of the VBLaST™: a virtual basic laparoscopic skill trainer. Int J Med Robot Comput Assist Surg 4:131–138. doi: 10.1002/rcs.185 CrossRefGoogle Scholar
  21. 21.
    Arikatla VS, Sankaranarayanan G, Ahn W, Chellali A, De S, Caroline GL, Hwabejire J, DeMoya M, Schwaitzberg S, Jones DB (2013) Face and construct validation of a virtual peg transfer simulator. Surg Endosc 27:1721–1729. doi: 10.1007/s00464-012-2664-y CrossRefPubMedGoogle Scholar
  22. 22.
    Sankaranarayanan G, Lin H, Arikatla VS, Mulcare M, Zhang L, Derevianko A, Lim R, Fobert D, Cao C, Schwaitzberg SD, Jones DB, De S (2010) Preliminary face and construct validation study of a virtual basic laparoscopic skill trainer. J Laparoendosc Adv Surg Tech A 20:153–157. doi: 10.1089/lap.2009.0030 CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Chellali A, Ahn W, Sankaranarayanan G, Flinn JT, Schwaitzberg SD, Jones DB, De S, Cao CGL (2015) Preliminary evaluation of the pattern cutting and the ligating loop virtual laparoscopic trainers. Surg Endosc 29:815–821. doi: 10.1007/s00464-014-3764-7 CrossRefPubMedGoogle Scholar
  24. 24.
    Williams SM, Parry BR, Schlup MM (1992) Quality control: an application of the CUSUM. BMJ 304:1359–1361CrossRefPubMedPubMedCentralGoogle Scholar
  25. 25.
    Biau DJ, Resche-Rigon M, Godiris-Petit G, Nizard RS, Porcher R (2007) Quality control of surgical and interventional procedures: a review of the CUSUM. Qual Saf Health Care 16:203–207. doi: 10.1136/qshc.2006.020776 CrossRefPubMedPubMedCentralGoogle Scholar
  26. 26.
    Faul F, Erdfelder E, Lang A-G, Buchner A (2007) G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 39:175–191. doi: 10.3758/BF03193146 CrossRefPubMedGoogle Scholar
  27. 27.
    Fraser SA, Feldman LS, Stanbridge D, Fried GM (2005) Characterizing the learning curve for a basic laparoscopic drill. Surg Endosc 19:1572–1578. doi: 10.1007/s00464-005-0150-5 CrossRefPubMedGoogle Scholar
  28. 28.
    Bartlett A, Parry B (2001) CUSUM analysis of trends in operative selection and conversion rates for laparoscopic cholecystectomy. ANZ J Surg 71:453–456. doi: 10.1046/j.1440-1622.2001.02163.x CrossRefPubMedGoogle Scholar
  29. 29.
    Kye B-H, Kim J-G, Cho H-M, Kim H-J, Suh Y-J, Chun C-S (2011) Learning curves in laparoscopic right-sided colon cancer surgery: a comparison of first-generation colorectal surgeon to advance laparoscopically trained surgeon. J Laparoendosc Adv Surg Tech 21:789–796. doi: 10.1089/lap.2011.0086 CrossRefGoogle Scholar
  30. 30.
    Okrainec A, Ferri LE, Feldman LS, Fried GM (2011) Defining the learning curve in laparoscopic paraesophageal hernia repair: a CUSUM analysis. Surg Endosc 25:1083–1087. doi: 10.1007/s00464-010-1321-6 CrossRefPubMedGoogle Scholar
  31. 31.
    Hyltander A, Liljegren E, Rhodin PH, Lönroth H (2002) The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc 16:1324–1328. doi: 10.1007/s00464-001-9184-5 CrossRefPubMedGoogle Scholar
  32. 32.
    Korndorffer JR, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ (2005) Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg 201:23–29. doi: 10.1016/j.jamcollsurg.2005.02.021 CrossRefPubMedGoogle Scholar
  33. 33.
    McCluney AL, Vassiliou MC, Kaneva PA, Cao J, Stanbridge DD, Feldman LS, Fried GM (2007) FLS simulator performance predicts intraoperative laparoscopic skill. Surg Endosc 21:1991–1995. doi: 10.1007/s00464-007-9451-1 CrossRefPubMedGoogle Scholar
  34. 34.
    Sroka G, Feldman LS, Vassiliou MC, Kaneva PA, Fayez R, Fried GM (2010) Fundamentals of Laparoscopic Surgery simulator training to proficiency improves laparoscopic performance in the operating room—a randomized controlled trial. Am J Surg 199:115–120. doi: 10.1016/j.amjsurg.2009.07.035 CrossRefPubMedGoogle Scholar
  35. 35.
    Rosen J, Brown JD, Chang L, Sinanan MN, Hannaford B (2006) Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model. IEEE Trans Biomed Eng 53:399–413. doi: 10.1109/TBME.2005.869771 CrossRefPubMedGoogle Scholar
  36. 36.
    Lin HC, Shafran I, Yuh D, Hager GD (2006) Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. Comput Aided Surg 11:220–230. doi: 10.3109/10929080600989189 CrossRefPubMedGoogle Scholar
  37. 37.
    Leff DR, Orihuela-Espina F, Elwell CE, Athanasiou T, Delpy DT, Darzi AW, Yang G-Z (2011) Assessment of the cerebral cortex during motor task behaviours in adults: a systematic review of functional near infrared spectroscopy (fNIRS) studies. Neuroimage 54:2922–2936. doi: 10.1016/j.neuroimage.2010.10.058 CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Arun Nemani
    • 1
  • Woojin Ahn
    • 1
  • Clairice Cooper
    • 2
  • Steven Schwaitzberg
    • 2
  • Suvranu De
    • 1
  1. 1.Rensselaer Polytechnic InstituteTroyUSA
  2. 2.University at Buffalo School of Medicine and Biomedical SciencesBuffaloUSA

Personalised recommendations