Skip to main content
SpringerLink
Log in
Book cover

Surgical Robotics pp 419–448Cite as

Force Feedback and Sensory Substitution for Robot-Assisted Surgery

  • Chapter
  • First Online:

Abstract

It is hypothesized that the lack of haptic (force and tactile) feedback presented to the surgeon is a limiting factor in the performance of teleoperated robot-assisted minimally invasive surgery. This chapter reviews the technical challenges of creating force feedback in robot-assisted surgical systems and describes recent results in creating and evaluating the effectiveness of this feedback in mock surgical tasks. In the design of a force-feedback teleoperator, the importance of hardware design choices and their relationship to controller design are emphasized. In addition, the practicality and necessity of force feedback in all degrees of freedom of the teleoperator are considered in the context of surgical tasks and the operating room environment. An alternative to direct force feedback to the surgeon’s hands is sensory substitution/augmented reality, in which graphical displays are used to convey information about the forces between the surgical instrument and the patient, or about the mechanical properties of the patient’s tissue. Experimental results demonstrate that the effectiveness of direct and graphical force feedback depend on the nature of the surgical task and the experience level of the surgeon.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   229.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    The four-channel framework describes single-input/single-output hardware P m and P s . To extend the framework to multi-degree-of-freedom master and slave devices, one may treat each degree of freedom as an independent scalar feedback loop; however, this assumption ignores interactions between degrees of freedom. Robust, optimal state-space control design such as H ∞ and μ -synthesis [32] provide an alternative integrated multivariable approach.

  2. 2.

    The tested device differs from the commercial system in its custom mounting, motor amplifiers, data acquisition, and controllers.

  3. 3.

    While the magnitude of the measured frequency response is relatively flat over the normalized bandwidth of 10 − 2 to 10 − 1 in Fig. 18.7, the ability to move the distal end of the master robot in fact diminishes rapidly above 10 − 2. The measured response is between a proximal motor and a co-located sensor rather than the robot endpoint. The anti-resonance zeros become poles in the response from motor command to the robot endpoint position.

References

  1. Akinbiyi, T., Reiley, C.E., Saha, S., Burschka, D., Hasser, C.J., Yuh, D.D., Okamura, A.M.: Dynamic augmented reality for sensory substitution in robot-assisted surgical systems. In: 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 567–570 (2006)

    Google Scholar 

  2. Anderson, R.J., Spong, M.W.: Bilateral control of teleoperators with time delay. IEEE Trans. Automat. Contr. 34(5), 494–501 (1989)

    Article  MathSciNet  Google Scholar 

  3. Barbagli, F., Salisbury, K.: The effect of sensor/actuator asymmetries in haptic interfaces. In: Proceedings of 11th Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, pp. 140–147 (2003)

    Google Scholar 

  4. Barbagli, F., Salisbury, K., Ho, C., Spence, C., Tan, H.: Haptic discrimination of force direction and the influence of visual information. ACM Trans. Appl. Percept. 3(2), 125–135 (2006)

    Article  Google Scholar 

  5. Bethea, B.T., Okamura, A.M., Kitagawa, M., Fitton, T.P., Cattaneo, S.M., Gott, V.L., Baumgartner, W.A., Yuh, D.D.: Application of haptic feedback to robotic surgery. J. Laparoendosc. Adv. Surg. Tech. 14(3), 191–195 (2004)

    Article  Google Scholar 

  6. Cavusoglu, M.C., Sherman, A., Tendick, F.: Design of bilateral teleoperation controllers for haptic exploration and telemanipulation of soft environments. IEEE Trans. Rob. Autom. 18(4), 641–647 (2002)

    Article  Google Scholar 

  7. Daniel, R.W., McAree, P.R.: Fundamental limits of performance for force reflecting teleoperation. Int. J. Rob. Res. 17(8), 811–830 (1998)

    Article  Google Scholar 

  8. Diolaiti, N., Melchiorri, C., Stramigioli, S.: Contact impedance estimation for robotic systems. IEEE Trans. Robot. 21(5), 925–935 (2005)

    Article  Google Scholar 

  9. Gersem, G.D., Brussel, H.V., Tendick, F.: Reliable and enhanced stiffness perception in soft-tissue telemanipulation. Int. J. Rob. Res. 24(10), 805–822 (2005)

    Article  Google Scholar 

  10. Griffiths, P.G., Gillespie, R.B.: Characterizing teleoperator behavior for feedback design and performance analysis. In: Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 273–280 (2008)

    Google Scholar 

  11. Griffiths, P.G., Gillespie, R.B., Freudenberg, J.S.: A fundamental tradeoff between performance and sensitivity in haptic rendering. IEEE Trans. Robot. 24(3), 537–548 (2008)

    Article  Google Scholar 

  12. Gwilliam, J., Mahvash, M., Vagvolgyi, B., Vacharat, A., Yuh, D.D., Okamura, A.M.: Effects of haptic and graphical force feedback for teleoperated palpation. In: IEEE International Conference on Robotics and Automation, pp. 677–682 (2009)

    Google Scholar 

  13. Hannaford, B.: A design framework for teleoperators with kinesthetic feedback. IEEE Trans. Rob. Autom. 5(4), 426–434 (1989)

    Article  Google Scholar 

  14. Hashtrudi-Zaad, K., Salcudean, S.E.: Analysis of control architectures for teleoperation systems with impedance/admittance master and slave manipulators. Int. J. Rob. Res. 20(6), 419–445 (2001)

    Article  Google Scholar 

  15. Haykin, S.S.: Active Network Theory. Addison-Wesley, London (1970)

    MATH  Google Scholar 

  16. Hoyt, K., Castaneda, B., Zhang, M., Nigwekar, P., di Sant’agnese, P.A., Joseph, J.V., Strang, J., Rubens, D.J., Parker, K.J.: Tissue elasticity properties as biomarkers for prostate cancer. Cancer Biomark. 4(4–5), 213–225 (2008)

    Google Scholar 

  17. Khalil, H.K.: Nonlinear Systems, 3rd edn. Prentice Hall, Upper Saddle River, NJ (2002)

    MATH  Google Scholar 

  18. Kim, K., Cavusoglu, M.C., Chung, W.K.: Quantitative comparison of bilateral teleoperation systems using μ-synthesis. IEEE Trans. Robot. 23(4), 776–789 (2007)

    Article  Google Scholar 

  19. King, C.H., Culjat, M.O., Franco, M.L., Lewis, C.E., Dutson, E.P., Grundfest, W.S., Bisley, J.W.: Tactile feedback induces reduced grasping force in robot-assisted surgery. IEEE Trans. Haptics 2(2), 103–110 (2009)

    Article  Google Scholar 

  20. Kitagawa, M., Bethea, B.T., Gott, V.L., Okamura, A.M.: Analysis of suture manipulation forces for teleoperation with force feedback. In: T. Dohi, R. Kikinis (eds.) Medical Image Computing and Computer Assisted Intervention. Lecture Notes in Computer Science, vol. 2488, pp. 155–162. Springer, Berlin (2002)

    Google Scholar 

  21. Krouskop, T.A., Wheeler, T.M., Kallel, F., Garra, B.S., Hall, T.: Elastic moduli of breast and prostate tissues under compression. Ultrasonic imaging 20(4), 260–274 (1998)

    Google Scholar 

  22. Lawrence, D.: Stability and transparency in bilateral teleoperation. IEEE Trans. Rob. Autom. 9(5), 624–637 (1993)

    Article  MathSciNet  Google Scholar 

  23. Liu, H., Noonan, D.P., Challacombe, B.J., Dasgupta, P., Seneviratne, L.D., Althoefer, K.: Rolling Mechanical Imaging for Tissue Abnormality Localization During Minimally Invasive Surgery. IEEE Trans. Biomed. Eng. 57(2), 404–14 (2010)

    Google Scholar 

  24. Mahvash, M., Gwilliam, J., Agarwal, R., Vagvolgyi, B., Su, L.M., Yuh, D.D., Okamura, A.M.: Force-feedback surgical teleoperator: Controller design and palpation experiments. In: 16th Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, pp. 465–471 (2008)

    Google Scholar 

  25. Miller, A.P., Peine, W.J., Son, J.S., Hammoud, Z.T.: Tactile imaging system for localizing lung nodules during video assisted thoracoscopic surgery. In: IEEE International Conference on Robotics and Automation, pp. 2996–3001 (2007)

    Google Scholar 

  26. Niemeyer, G., Slotine, J.E.: Telemanipulation with time delays. Int. J. Rob. Res. 23(9), 873–890 (2004)

    Article  Google Scholar 

  27. Okamura, A.M.: Methods for haptic feedback in teleoperated robot-assisted surgery. Ind. Rob. 31(6), 499–508 (2004)

    Article  Google Scholar 

  28. Ortmaier, T., Deml, B., Kuebler, B., Passig, G., Reintsema, D., Seibold, U.: Robot assisted force feedback surgery. In: Ferre, M., Buss, M., Aracil, R., Melchiorri, C., Balaguer, C. (eds.) Advances in Telerobotics, Springer Tracts on Advanced Robotics STAR 31, pp. 361–379. Springer, Berlin (2007)

    Google Scholar 

  29. Reiley, C.E., Akinbiyi, T., Burschka, D., Chang, D.C., Okamura, A.M., Yuh, D.D.: Effects of visual force feedback on robot-assisted surgical task performance. J. Thorac. Cardiovasc. Surg. 135(1), 196–202 (2008)

    Article  Google Scholar 

  30. Saha, S.: Appropriate degrees of freedom of force sensing in robot-assisted minimally invasive surgery. M.S. thesis, Department of Biomedical Engineering, The Johns Hopkins University (2006)

    Google Scholar 

  31. Semere, W., Kitagawa, M., Okamura, A.M.: Teleoperation with sensor/actuator asymmetry: Task performance with partial force feedback. In: Proceedings of 12th Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, pp. 121–127 (2004)

    Google Scholar 

  32. Skogestad, S., Postlethwaite, I.: Multivariable Feedback Control: Analysis and Design. Wiley, New York (1997)

    MATH  Google Scholar 

  33. Tan, H., Barbagli, F., Salisbury, K., Ho, C., Spence, C.: Force-direction discrimination is not influenced by reference force direction. Haptics-e, Electron J. Haptics Res. (www.haptics-e.org) 4(1) (2006)

  34. Tavakoli, M., Aziminejad, A., Patel, R., Moallem, M.: High-fidelity bilateral teleoperation systems and the effect of multimodal haptics. IEEE Trans. Syst. Man Cybern. B Cybern. 37(6), 1512–1528 (2007)

    Article  Google Scholar 

  35. Tavakoli, M., Howe, R.D.: Haptic effects of surgical teleoperator flexibility. Int. J. Rob. Res. 28(10), 1289–1301 (2004)

    Article  Google Scholar 

  36. Taylor, R., Jensen, P., Whitcomb, L., Barnes, A., Kumar, R., Stoianovici, D., Gupta, P., Wang, Z., Dejuan, E., Kavoussi, L.: Steady-hand robotic system for microsurgical augmentation. Int. J. Rob. Res. 18(12), 1201–1210 (1999)

    Article  Google Scholar 

  37. Tobergte, A., Konietschke, R., Hirzinger, G.: Planning and control of a teleoperation system for research in minimally invasive robotic surgery. In: IEEE International Conference on Robotics and Automation, pp. 4225–4232 (2009)

    Google Scholar 

  38. Trejos, A.L., Jayender, J., Perri, M.T., Naish, M.D., Patel, R.V., Malthaner, R.A.: Robot-assisted Tactile Sensing for Minimally Invasive Tumor Localization. Int. J. Rob. Res. 28(9), 1118–1133 (2009)

    Article  Google Scholar 

  39. Verner, L.N., Okamura, A.M.: Force & torque feedback vs force only feedback. In: Third Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (World Haptics), pp. 406–410 (2009)

    Google Scholar 

  40. Verner, L.N.: Sensor/actuator asymmetries in telemanipulators. Ph.D. in Mechanical Engineering, The Johns Hopkins University (2009)

    Google Scholar 

  41. Verner, L.N., Okamura, A.M.: Effects of translational and gripping force feedback are decoupled in a 4-degree-of-freedom telemanipulator. In: Second Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (World Haptics), pp. 286–291 (2007)

    Google Scholar 

  42. Verner, L.N., Okamura, A.M.: Telemanipulators with sensor/actuator asymmetries fail the robustness criterion. In: Proceedings of 16th Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, pp. 267–271 (2008)

    Google Scholar 

  43. Wagner, C.R., Stylopoulos, N., Jackson, P.G., Howe, R.D.: The benefit of force feedback in surgery: Examination of blunt dissection. Presence: Teleoperators and Virtual Environments 16(3), 252–262 (2007)

    Article  Google Scholar 

  44. Xu, K., Simaan, N.: An investigation of the intrinsic force sensing capabilities of continuum robots. IEEE Trans. Robot. 24(3), 576–587 (2008)

    Article  Google Scholar 

  45. Yamamoto, T., Bernhardt, M., Peer, A., Buss, M., Okamura, A.M.: Multi-estimator technique for environment parameter estimation during telemanipulation. In: IEEE International Conference on Biomedical Robotics and Biomechatronics, pp. 217–223 (2008)

    Google Scholar 

  46. Yamamoto, T., Vagvolgyi, B., Balaji, K., Whitcomb, L., Okamura, A.M.: Tissue property estimation and graphical display for teleoperated robot-assisted surgery. In: IEEE International Conference on Robotics and Automation, pp. 4239–4245 (2009)

    Google Scholar 

  47. Yokokohji, Y., Yoshikawa, T.: Bilateral control of master-slave manipulators for ideal kinesthetic coupling–formulation and experiment. IEEE Trans. Rob. Autom. 10(5), 605–20 (1994)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by Johns Hopkins University, National Science Foundation grants 0347464, 9731478, and 0722943, and National Institutes of Health grant EB002004. The authors thank Dr. David Yuh, Dr. Li-Ming Su, Dr. Mohsen Mahvash, Carol Reiley, Balazs Vagvolgyi, Masaya Kitagawa and Wagahta Semere for their contributions to this work, and Intuitive Surgical, Inc. for access to surgical robotics hardware.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, Laboratory for Computional Sensing and Robotics, Johns Hopkins University, Baltimore, MD, 21218, USA

    Allison M. Okamura

Authors
  1. Allison M. Okamura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lawton N. Verner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tomonori Yamamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. James C. Gwilliam
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Paul G. Griffiths
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Allison M. Okamura .

Editor information

Editors and Affiliations

  1. Baskin School of Engineering SOE-3, Department of Computer Engineering, University of California Santa Cruz, 1156 High Street, Santa Cruz, 95064, California, USA

    Jacob Rosen

  2. , Department of Electrical Engineering, University of Washington, Box 325500, Seattle, 98195-2500, Washington, USA

    Blake Hannaford

  3. Medical Center, Dept. Surgery, University of Washington, Seattle, 98195, Washington, USA

    Richard M. Satava

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Okamura, A.M., Verner, L.N., Yamamoto, T., Gwilliam, J.C., Griffiths, P.G. (2011). Force Feedback and Sensory Substitution for Robot-Assisted Surgery. In: Rosen, J., Hannaford, B., Satava, R. (eds) Surgical Robotics. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-1126-1_18

Download citation

  • DOI: https://doi.org/10.1007/978-1-4419-1126-1_18

  • Published:

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4419-1125-4

  • Online ISBN: 978-1-4419-1126-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation