In vivo validation of a system for haptic feedback of tool vibrations in robotic surgery

Surgical Endoscopy volume 27, pages656–664(2013)Cite this article

Abstract

Background

Robotic minimally invasive surgery (RMIS) lacks the haptic (kinesthetic and tactile) cues that surgeons are accustomed to receiving in open and laparoscopic surgery. We previously introduced a method for adding tactile and audio feedback of tool vibrations to RMIS systems, creating sensations similar to what one feels and hears when using a laparoscopic tool. Our prior work showed that surgeons performing box-trainer tasks significantly preferred having this feedback and believed that it helped them concentrate on the task, but we did not know how well our approach would work in a clinically relevant setting. This study constituted the first in vivo test of our system.

Methods

Accelerometers that measure tool vibrations were mounted to the patient-side manipulators of a da Vinci S surgical system. The measured vibrations were recorded and presented to the surgeon through vibrotactile and audio channels while two transperitoneal nephrectomies and two mid-ureteral dissections with uretero-ureterostomy were completed on a porcine model. We examined 30 minutes of resulting video to identify and tag manipulation events, aiming to determine whether our system can measure significant and meaningful tool vibrations during in vivo procedures.

Results

A total of 1,404 manipulation events were identified. Analysis of each event’s accelerations indicated that 82 % of these events resulted in significant vibrations. The magnitude of the accelerations measured for different manipulation events varied widely, with hard contact causing the largest cues.

Conclusions

This study demonstrates the feasibility of providing tool vibration feedback during in vivo RMIS. Significant tool vibrations were reliably measured for the majority of events during standard urological procedures on a porcine model, while real-time, naturalistic tactile and audio tool vibration feedback was provided to the surgeon. The feedback system’s modules were easily implemented outside the sterile field of the da Vinci S and did not interfere with the surgical procedure.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    DiMaio S, Hanuschik M, Kreaden U (2011) The da Vinci Surgical System. In: Rosen J, Hannaford B, Satava RM (eds) Surgical robotics. Springer, New York, pp 199–218

    Google Scholar 

  2. 2.

    Bholat OS, Haluck RS, Murray WB, Gorman PJ, Krummel TM (1999) Tactile feedback is present during minimally invasive surgery. J Am Coll Surg 189:349–355

    PubMed  Article  CAS  Google Scholar 

  3. 3.

    Sung GT, Gill IS (2001) Robotic laparoscopic surgery: a comparison of the da Vinci and Zeus systems. Urology 58:893–898

    PubMed  Article  CAS  Google Scholar 

  4. 4.

    Meireles O, Horgan S (2011) Applications of surgical robotics in general surgery. In: Rosen J, Hannaford B, Satava RM (eds) Surgical robotics. Springer, New York, pp 791–812

    Google Scholar 

  5. 5.

    Costi R, Himpens J, Bruyns J, Cadière GB (2003) Robotic fundoplication: from theoretic advantages to real problems. J Am Coll Surg 197:500–507

    PubMed  Article  Google Scholar 

  6. 6.

    Nezhat F (2008) Minimally invasive surgery in gynecologic oncology: laparoscopy versus robotics. Gynecol Oncol 111:S29–S32

    PubMed  Article  Google Scholar 

  7. 7.

    Okamura AM, Verner LN, Yamamoto T, Gwilliam JC, Griffiths PG (2011) Force feedback and sensory substitution for robot-assisted surgery. In: Rosen J, Hannaford B, Satava RM (eds) Surgical robotics. Springer, New York, pp 419–448

    Google Scholar 

  8. 8.

    Culjat MO, Bisley JW, King CH, Wottawa C, Fan RE, Dutson EP, Grundfest WS (2011) Tactile feedback in surgical robotics. In: Rosen J, Hannaford B, Satava RM (eds) Surgical robotics. Springer, New York, pp 449–468

    Google Scholar 

  9. 9.

    Hagen ME, Meehan JJ, Inan I, Morel P (2008) Visual clues act as a substitute for haptic feedback in robotic surgery. Surg Endosc 22:1505–1508

    PubMed  Article  CAS  Google Scholar 

  10. 10.

    Wickens CD (2002) Multiple resources and performance prediction. Theor Issues Ergon Sci 3:159–177

    Article  Google Scholar 

  11. 11.

    Vitense HS, Jacko JA, Emery VK (2003) Multimodal feedback: an assessment of performance and mental workload. Ergonomics 46:68–87

    PubMed  Article  CAS  Google Scholar 

  12. 12.

    Prewett MS, Yang L, Stilson FRB, Gray AA, Coovert MD, Burke J, Redden E, Elliot LR (2006) The benefits of multimodal information: a meta-analysis comparing visual and visual-tactile feedback. Proc Multimodal Interfaces 333–338

  13. 13.

    Cao CGL, Zhou M, Jones DB, Schwaitzberg SD (2007) Can surgeons think and operate with haptics at the same time? J Gastrointest Surg 11:1564–1569

    PubMed  Article  Google Scholar 

  14. 14.

    Mayer H, Nagy I, Knoll A, Braun EU, Bauernschmitt R, Lange R (2007) Haptic feedback in a telepresence system for endoscopic heart surgery. Presence Teleoperators Virtual Environ 16:459–470

    Article  Google Scholar 

  15. 15.

    van der Meijden O, Schijven M (2009) The value of haptic feedback in conventional and robot-assisted minimal invasive surgery and virtual reality training: a current review. Surg Endosc 23:1180–1190

    PubMed  Article  Google Scholar 

  16. 16.

    Tan GY, Goel RK, Kaouk JH, Tewari AK (2009) Technological advances in robotic-assisted laparoscopic surgery. Urol Clin N Am 36:237–249

    Article  Google Scholar 

  17. 17.

    Okamura AM (2009) Haptic feedback in robot-assisted minimally invasive surgery. Curr Opin Urol 19:102–107

    PubMed  Article  Google Scholar 

  18. 18.

    Wagner CR, Stylopoulos N, Jackson PG, Howe RD (2007) The benefit of force feedback in surgery: examination of blunt dissection. Presence Teleoperators Virtual Environ 16:252–262

    Article  Google Scholar 

  19. 19.

    King CH, Culjat MO, Franco ML, Lewis CE, Dutson EP, Grundfest WS, Bisley JW (2009) Tactile feedback induces reduced grasping force in robotic surgery. IEEE Trans Haptics 2:103–110

    Article  Google Scholar 

  20. 20.

    Kontarinis DA, Howe RD (1996) Tactile display of vibratory information in teleoperation and virtual environments. Presence Teleoperators Virtual Environ 4:387–402

    Google Scholar 

  21. 21.

    Johnson KO (2001) The roles and functions of cutaneous mechanoreceptors. Curr Opin Neurobiol 11:455–461

    PubMed  Article  CAS  Google Scholar 

  22. 22.

    Johansson RS, Flanagan JR (2009) Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat Rev Neurosci 10:345–359

    PubMed  Article  CAS  Google Scholar 

  23. 23.

    McMahan W, Romano JM, Rahuman AMA, Kuchenbecker KJ (2010) High frequency acceleration feedback significantly increases the realism of haptically rendered textured surfaces. Proc IEEE Haptics Symp 141–148

  24. 24.

    McMahan W, Gewirtz J, Standish D, Martin P, Kunkel JA, Lilavois M, Wedmid A, Lee DI, Kuchenbecker KJ (2011) Tool contact acceleration feedback for telerobotic surgery. IEEE Trans Haptics 4:210–220

    Article  Google Scholar 

  25. 25.

    Hayward V, Maclean KE (2007) Do it yourself haptics: part I. IEEE Robot Automation Mag 14:88–104

    Article  Google Scholar 

  26. 26.

    Kuchenbecker KJ, Gewirtz J, McMahan W, Satndish D, Martin P, Bohren J, Mendoza PJ, Lee DI (2010) VerroTouch: high-frequency acceleration feedback for telerobotic surgery. In: Kappers AML, van Erp JBF, Bergmann Tiest WM, van der Helm FCT (eds) Haptics: generating and perceiving tangible sensations. Proceedings of EuroHaptics Part I, vol 6191. Springer, New York, pp 189–196

  27. 27.

    Bethea BT, Okamura AM, Kitagawa M, Fitton TP, Cattaneo SM, Gott VL, Baumgartner WA, Yuh DD (2004) Application of haptic feedback to robotic surgery. J Laparoendosc Adv Surg Tech A 14:191–195

    PubMed  Article  Google Scholar 

  28. 28.

    Bark K, Gomez ED, Rivera C, McMahan W, Remington AC, Murayama KM, Lee DI, Dumon KR, Williams NN, Kuchenbecker KJ (2012) Surgical instrument vibrations are a construct-valid measure of technical skill in robotic peg transfer and suturing tasks. In: Yang GZ, Darzi A (eds) Proc. Hamlyn Symposium on Medical Robotics, pp 50–51

  29. 29.

    Judkins TN, Oleynikov D, Stergiou N (2008) Enhanced robotic surgical training using augmented visual feedback. Surg Innov 15:59–68

    PubMed  Article  Google Scholar 

Download references

Acknowledgments

The authors thank Aly Kozieja, April Laskow, and Thad Henninger for their assistance in completing the reported procedures. We also thank Paul Martin, Dorsey Standish, and Pierre J. Mendoza for their contributions to the development of the VerroTouch system. This research was supported by a Coulter Translational Research Award, the Pennsylvania Department of Health via Health Research Formula Funds, and the National Science Foundation via grant #IIS-0845670. The University of Pennsylvania institutional animal care and use committee approved this investigation via protocol #802129.

Disclosures

Karlin Bark, William McMahan, Austin Remington, Jamie Gewirtz, and Alexei Wedmid have no conflicts of interest or financial ties to disclose. Professor Katherine J. Kuchenbecker has lectured at Lankenau Hospital in Philadelphia, receiving financial compensation. Professor Kuchenbecker has no stock ownership, equity interests, patent-licensing arrangements, or the like that might pose a conflict of interest in connection with this work, nor have her disclosures influenced the scientific work. Dr. David I. Lee has served as meeting participant and lecturer at Intuitive Surgical, Ethicon, and Aureon, and participated in scientific studies and trials at Johnson and Johnson and Pfizer. Dr. Lee has no stock ownership, equity interests, patent-licensing arrangements, or the like that might pose a conflict of interest in connection with this work, nor have his disclosures influenced the scientific work.

Author information

Affiliations

  1. Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania, 144 Towne Building, 220 South 33rd Street, Philadelphia, PA, 19104, USA

    Karlin Bark, William McMahan, Austin Remington, Jamie Gewirtz & Katherine J. Kuchenbecker

  2. Department of Surgery, Division of Urology, University of Pennsylvania, Philadelphia, PA, USA

    Alexei Wedmid & David I. Lee

Authors
  1. Karlin Bark
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. William McMahan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Austin Remington
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jamie Gewirtz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alexei Wedmid
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. David I. Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Katherine J. Kuchenbecker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Katherine J. Kuchenbecker.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (MOV 26384 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Bark, K., McMahan, W., Remington, A. et al. In vivo validation of a system for haptic feedback of tool vibrations in robotic surgery. Surg Endosc 27, 656–664 (2013). https://doi.org/10.1007/s00464-012-2452-8

Download citation

Keywords

  • Touch
  • Technology assessment
  • Biomedical
  • Robotics/instrumentation
  • Feedback/sensory