Robotic minimally invasive surgery (RMIS) lacks the haptic (kinesthetic and tactile) cues that surgeons are accustomed to receiving in open and laparoscopic surgery. We previously introduced a method for adding tactile and audio feedback of tool vibrations to RMIS systems, creating sensations similar to what one feels and hears when using a laparoscopic tool. Our prior work showed that surgeons performing box-trainer tasks significantly preferred having this feedback and believed that it helped them concentrate on the task, but we did not know how well our approach would work in a clinically relevant setting. This study constituted the first in vivo test of our system.
Accelerometers that measure tool vibrations were mounted to the patient-side manipulators of a da Vinci S surgical system. The measured vibrations were recorded and presented to the surgeon through vibrotactile and audio channels while two transperitoneal nephrectomies and two mid-ureteral dissections with uretero-ureterostomy were completed on a porcine model. We examined 30 minutes of resulting video to identify and tag manipulation events, aiming to determine whether our system can measure significant and meaningful tool vibrations during in vivo procedures.
A total of 1,404 manipulation events were identified. Analysis of each event’s accelerations indicated that 82 % of these events resulted in significant vibrations. The magnitude of the accelerations measured for different manipulation events varied widely, with hard contact causing the largest cues.
This study demonstrates the feasibility of providing tool vibration feedback during in vivo RMIS. Significant tool vibrations were reliably measured for the majority of events during standard urological procedures on a porcine model, while real-time, naturalistic tactile and audio tool vibration feedback was provided to the surgeon. The feedback system’s modules were easily implemented outside the sterile field of the da Vinci S and did not interfere with the surgical procedure.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
DiMaio S, Hanuschik M, Kreaden U (2011) The da Vinci Surgical System. In: Rosen J, Hannaford B, Satava RM (eds) Surgical robotics. Springer, New York, pp 199–218
Bholat OS, Haluck RS, Murray WB, Gorman PJ, Krummel TM (1999) Tactile feedback is present during minimally invasive surgery. J Am Coll Surg 189:349–355
Sung GT, Gill IS (2001) Robotic laparoscopic surgery: a comparison of the da Vinci and Zeus systems. Urology 58:893–898
Meireles O, Horgan S (2011) Applications of surgical robotics in general surgery. In: Rosen J, Hannaford B, Satava RM (eds) Surgical robotics. Springer, New York, pp 791–812
Costi R, Himpens J, Bruyns J, Cadière GB (2003) Robotic fundoplication: from theoretic advantages to real problems. J Am Coll Surg 197:500–507
Nezhat F (2008) Minimally invasive surgery in gynecologic oncology: laparoscopy versus robotics. Gynecol Oncol 111:S29–S32
Okamura AM, Verner LN, Yamamoto T, Gwilliam JC, Griffiths PG (2011) Force feedback and sensory substitution for robot-assisted surgery. In: Rosen J, Hannaford B, Satava RM (eds) Surgical robotics. Springer, New York, pp 419–448
Culjat MO, Bisley JW, King CH, Wottawa C, Fan RE, Dutson EP, Grundfest WS (2011) Tactile feedback in surgical robotics. In: Rosen J, Hannaford B, Satava RM (eds) Surgical robotics. Springer, New York, pp 449–468
Hagen ME, Meehan JJ, Inan I, Morel P (2008) Visual clues act as a substitute for haptic feedback in robotic surgery. Surg Endosc 22:1505–1508
Wickens CD (2002) Multiple resources and performance prediction. Theor Issues Ergon Sci 3:159–177
Vitense HS, Jacko JA, Emery VK (2003) Multimodal feedback: an assessment of performance and mental workload. Ergonomics 46:68–87
Prewett MS, Yang L, Stilson FRB, Gray AA, Coovert MD, Burke J, Redden E, Elliot LR (2006) The benefits of multimodal information: a meta-analysis comparing visual and visual-tactile feedback. Proc Multimodal Interfaces 333–338
Cao CGL, Zhou M, Jones DB, Schwaitzberg SD (2007) Can surgeons think and operate with haptics at the same time? J Gastrointest Surg 11:1564–1569
Mayer H, Nagy I, Knoll A, Braun EU, Bauernschmitt R, Lange R (2007) Haptic feedback in a telepresence system for endoscopic heart surgery. Presence Teleoperators Virtual Environ 16:459–470
van der Meijden O, Schijven M (2009) The value of haptic feedback in conventional and robot-assisted minimal invasive surgery and virtual reality training: a current review. Surg Endosc 23:1180–1190
Tan GY, Goel RK, Kaouk JH, Tewari AK (2009) Technological advances in robotic-assisted laparoscopic surgery. Urol Clin N Am 36:237–249
Okamura AM (2009) Haptic feedback in robot-assisted minimally invasive surgery. Curr Opin Urol 19:102–107
Wagner CR, Stylopoulos N, Jackson PG, Howe RD (2007) The benefit of force feedback in surgery: examination of blunt dissection. Presence Teleoperators Virtual Environ 16:252–262
King CH, Culjat MO, Franco ML, Lewis CE, Dutson EP, Grundfest WS, Bisley JW (2009) Tactile feedback induces reduced grasping force in robotic surgery. IEEE Trans Haptics 2:103–110
Kontarinis DA, Howe RD (1996) Tactile display of vibratory information in teleoperation and virtual environments. Presence Teleoperators Virtual Environ 4:387–402
Johnson KO (2001) The roles and functions of cutaneous mechanoreceptors. Curr Opin Neurobiol 11:455–461
Johansson RS, Flanagan JR (2009) Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat Rev Neurosci 10:345–359
McMahan W, Romano JM, Rahuman AMA, Kuchenbecker KJ (2010) High frequency acceleration feedback significantly increases the realism of haptically rendered textured surfaces. Proc IEEE Haptics Symp 141–148
McMahan W, Gewirtz J, Standish D, Martin P, Kunkel JA, Lilavois M, Wedmid A, Lee DI, Kuchenbecker KJ (2011) Tool contact acceleration feedback for telerobotic surgery. IEEE Trans Haptics 4:210–220
Hayward V, Maclean KE (2007) Do it yourself haptics: part I. IEEE Robot Automation Mag 14:88–104
Kuchenbecker KJ, Gewirtz J, McMahan W, Satndish D, Martin P, Bohren J, Mendoza PJ, Lee DI (2010) VerroTouch: high-frequency acceleration feedback for telerobotic surgery. In: Kappers AML, van Erp JBF, Bergmann Tiest WM, van der Helm FCT (eds) Haptics: generating and perceiving tangible sensations. Proceedings of EuroHaptics Part I, vol 6191. Springer, New York, pp 189–196
Bethea BT, Okamura AM, Kitagawa M, Fitton TP, Cattaneo SM, Gott VL, Baumgartner WA, Yuh DD (2004) Application of haptic feedback to robotic surgery. J Laparoendosc Adv Surg Tech A 14:191–195
Bark K, Gomez ED, Rivera C, McMahan W, Remington AC, Murayama KM, Lee DI, Dumon KR, Williams NN, Kuchenbecker KJ (2012) Surgical instrument vibrations are a construct-valid measure of technical skill in robotic peg transfer and suturing tasks. In: Yang GZ, Darzi A (eds) Proc. Hamlyn Symposium on Medical Robotics, pp 50–51
Judkins TN, Oleynikov D, Stergiou N (2008) Enhanced robotic surgical training using augmented visual feedback. Surg Innov 15:59–68
The authors thank Aly Kozieja, April Laskow, and Thad Henninger for their assistance in completing the reported procedures. We also thank Paul Martin, Dorsey Standish, and Pierre J. Mendoza for their contributions to the development of the VerroTouch system. This research was supported by a Coulter Translational Research Award, the Pennsylvania Department of Health via Health Research Formula Funds, and the National Science Foundation via grant #IIS-0845670. The University of Pennsylvania institutional animal care and use committee approved this investigation via protocol #802129.
Karlin Bark, William McMahan, Austin Remington, Jamie Gewirtz, and Alexei Wedmid have no conflicts of interest or financial ties to disclose. Professor Katherine J. Kuchenbecker has lectured at Lankenau Hospital in Philadelphia, receiving financial compensation. Professor Kuchenbecker has no stock ownership, equity interests, patent-licensing arrangements, or the like that might pose a conflict of interest in connection with this work, nor have her disclosures influenced the scientific work. Dr. David I. Lee has served as meeting participant and lecturer at Intuitive Surgical, Ethicon, and Aureon, and participated in scientific studies and trials at Johnson and Johnson and Pfizer. Dr. Lee has no stock ownership, equity interests, patent-licensing arrangements, or the like that might pose a conflict of interest in connection with this work, nor have his disclosures influenced the scientific work.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Bark, K., McMahan, W., Remington, A. et al. In vivo validation of a system for haptic feedback of tool vibrations in robotic surgery. Surg Endosc 27, 656–664 (2013). https://doi.org/10.1007/s00464-012-2452-8
- Technology assessment