Advertisement

Journal of Intelligent & Robotic Systems

, Volume 95, Issue 1, pp 165–192 | Cite as

A Scaled Bilateral Teleoperation System for Robotic-Assisted Surgery with Time Delay

  • Jing Guo
  • Chao LiuEmail author
  • Philippe Poignet
Article
  • 68 Downloads

Abstract

The master-slave teleoperated robotic systems have advanced the surgeries in the past decades. Time delay is usually caused due to the data transmission between communication channel connecting the master and slave in bilateral teleoperation, which is crucial because even small time delay could destabilize the whole teleoperation system. Motivated to solve the instability caused by time delay in bilateral teleoperation, wave variable transformation (WVT) structure has been proposed to passivate the delayed communication channel. However, conventional WVT structure provides poor velocity, position and force tracking performances which are not sufficient for surgical applications. In this paper, a new wave variable compensation (WVC) structure is proposed to improve the tracking performances with less conservative condition and comprehensive analysis to keep stable and improved tracking performance is also provided. In order to better facilitate certain surgical procedures with special requirements, e.g. robotic-assisted neurosurgery, velocity/position and force scalings are designed in the proposed structure with guaranteed system passivity, and transparency of the scaled WVC structure is also analyzed. Simulation and experimental studies were carried out to verify the performance of the proposed structure with time delay. System performance comparisons with several existing wave based bilateral teleoperation structures are also provided through simulation studies to show the improvements brought by the proposed teleoperation structure.

Keywords

Robotic-assisted surgery Time delay Bilateral teleoperation Wave variable Passivity Transparency 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

This work was partially supported by Guangzhou Elite Project and National Natural Science Foundation of China (Grant 61803103).

References

  1. 1.
    Nguan, C., Miller, B., Patel, R., Luke, P.P., Schlachta, C.M.: Pre-clinical remote telesurgery trial of a da vinci telesurgery prototype. Int. J. Med. Rob. Comput. Assisted Surg. 4(4), 304–309 (2008)Google Scholar
  2. 2.
    Panait, L., Akkary, E., Bell, R.L., Roberts, K.E., Dudrick, S.J., Duffy, A.J.: The role of haptic feedback in laparoscopic simulation training. J. Surg. Res. 156(2), 312–316 (2009)Google Scholar
  3. 3.
    Tahmasebi, A.M., Hashtrudi-Zaad, K., Thompson, D., Abolmaesumi, P.: A framework for the design of a novel haptic-based medical training simulator. IEEE Trans. Inf. Technol. Biomed. 12(5), 658–666 (2008)Google Scholar
  4. 4.
    Artigas, J., Hirzinger, G.: A brief history of dlr’s space telerobotics and force-feedback teleoperation. Acta Polytech. Hung. 13(1), 239–249 (2016)Google Scholar
  5. 5.
    Marturi, N., Rastegarpanah, A., Takahashi, C., Adjigble, M., Stolkin, R., Zurek, S., Kopicki, M., Talha, M., Kuo, J.A., Bekiroglu, Y.: Towards advanced robotic manipulation for nuclear decommissioning: a pilot study on tele-operation and autonomy. In: 2016 International Conference on Robotics and Automation for Humanitarian Applications (RAHA), pp. 1–8. IEEE (2016)Google Scholar
  6. 6.
    Murphy, R.R., Dreger, K.L., Newsome, S., Rodocker, J., Steimle, E., Kimura, T., Makabe, K., Matsuno, F., Tadokoro, S., Kon, K.: Use of remotely operated marine vehicles at minamisanriku and rikuzentakata Japan for disaster recovery. In: 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pp. 19–25. IEEE (2011)Google Scholar
  7. 7.
    Hokayem, P.F., Spong, M.W.: Bilateral teleoperation: an historical survey. Automatica 42(12), 2035–2057 (2006)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Li, Z., Xia, Y., Sun, F.: Adaptive fuzzy control for multilateral cooperative teleoperation of multiple robotic manipulators under random Network-Induced delays. IEEE Trans. Fuzzy Systems 22(2), 437–450 (2014)Google Scholar
  9. 9.
    Li, Z., Xia, Y., Wang, D., Zhai, D., Su, C.Y., Zhao, X.: Neural Network-Based control of networked trilateral teleoperation with geometrically unknown constraints. IEEE Trans. Cybernetics 46(5), 1051–1064 (2016)Google Scholar
  10. 10.
    Bergeles, C., Yang, G.-Z.: From passive tool holders to microsurgeons: safer, smaller, smarter surgical robots. IEEE Trans. Biomed. Eng. 61(5), 1565–1576 (2014)Google Scholar
  11. 11.
    Vitiello, V., Lee, S.-L., Cundy, T.P., Yang, G.-Z.: Emerging robotic platforms for minimally invasive surgery. IEEE Rev. Biomed. Eng. 6, 111–126 (2013)Google Scholar
  12. 12.
    Taylor, R.H., Menciassi, A., Fichtinger, G., Fiorini, P., Dario, P.: Medical robotics and computer-integrated surgery. In: Springer Handbook of Robotics, pp. 1657–1684. Springer (2016)Google Scholar
  13. 13.
    Karas, C.S., Chiocca, E.A.: Neurosurgical robotics: a review of brain and spine applications. J. Robot. Surg. 1(1), 39–43 (2007)Google Scholar
  14. 14.
    Burgner, J., Rucker, D.C., Gilbert, H.B., Swaney, P.J., Russell, P.T., Weaver, K.D., Webster, R.J.: A telerobotic system for transnasal surgery. IEEE/ASME Trans. Mechatron. 19(3), 996–1006 (2014)Google Scholar
  15. 15.
    Mohr, F.W., Falk, V., Diegeler, A., Walther, T., Gummert, J.F., Bucerius, J., Jacobs, S., Autschbach, R.: Computer-enhanced robotic cardiac surgery: experience in 148 patients. J. Thorac. Cardiovasc. Surg. 121(5), 842–853 (2001)Google Scholar
  16. 16.
    Okamura, A.M.: Methods for haptic feedback in teleoperated robot-assisted surgery. Industrial Robot: An International Journal 31(6), 499–508 (2004)Google Scholar
  17. 17.
    Tavakoli, M.: Haptics for teleoperated surgical robotic systems. World Scientific Publishing Co. Inc., Singapore (2008)Google Scholar
  18. 18.
    Christopher, R., Nicholas, S., Robert, D.: The role of force feedback in surgery: analysis of blunt dissection. In: 10Th Symposium on Haptic Interface for Virtual Environment and Teleoperator Systems, vol. 1, pp 18–125. IEEE Computer Society, Orlando (2002)Google Scholar
  19. 19.
    Meli, L., Pacchierotti, C., Prattichizzo, D.: Sensory subtraction in robot-assisted surgery: fingertip skin deformation feedback to ensure safety and improve transparency in bimanual haptic interaction. IEEE Trans. Biomed. Eng. 61(4), 1318–1327 (2014)Google Scholar
  20. 20.
    ANSI-AAMI-ST79, Comprehensive guide to steam sterilization and sterility assurance in health care facilities. In: A1:2010, A2:2011, Association for the Advancement of Medical Instrumentation Arlington, VAGoogle Scholar
  21. 21.
    Konstantinova, J., Jiang, A., Althoefer, K., Dasgupta, P., Nanayakkara, T.: Implementation of tactile sensing for palpation in robot-assisted minimally invasive surgery: a review. IEEE Sens. J 14(8), 2490–2501 (2014)Google Scholar
  22. 22.
    Hinterseer, P., Hirche, S., Chaudhuri, S., Steinbach, E., Buss, M.: Perception-based data reduction and transmission of haptic data in telepresence and teleaction systems. IEEE Trans. Signal Process. 56(2), 588–597 (2008)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Kuschel, M., Kremer, P., Buss, M.: Passive haptic data-compression methods with perceptual coding for bilateral presence systems. IEEE Trans. Syst. Man Cybern. Part A Syst. Humans 39(6), 1142–1151 (2009)Google Scholar
  24. 24.
    Kokkonis, G., Psannis, K., Roumeliotis, M.s, Kontogiannis, S.: A survey of transport protocols for haptic applications. In: 2012 16th Panhellenic Conference on Informatics (PCI), pp. 192–197. IEEE (2012)Google Scholar
  25. 25.
    Tortora, G., Dario, P., Menciassi, A.: Array of robots augmenting the kinematics of endocavitary surgery. IEEE/ASME Trans. Mechatron. 19(6), 1821–1829 (2014)Google Scholar
  26. 26.
    Guo, J., Liu, C., Poignet, P.: Scaled position-force tracking for wireless teleoperation of miniaturized surgical robotic system. In: Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE, pp. 361–365. IEEE (2014)Google Scholar
  27. 27.
    Shine, T.S.J., et al.: Specialized Operating Room (Chapter 13), Operation Room Design Manual, pp 44–56. American Society of Anesthesiologists, Illinois, USA (2012)Google Scholar
  28. 28.
    Anderson, R.J., Spong, M.W.: Bilateral control of teleoperators with time delay. IEEE Trans. Autom. Control 34(5), 494–501 (1989)MathSciNetGoogle Scholar
  29. 29.
    Niemeyer, G., Slotine, J.-J.: Stable adaptive teleoperation. IEEE J. Ocean. Eng. 16(1), 152–162 (1991)Google Scholar
  30. 30.
    Baier, H., Schmidt, G.: Transparency and stability of bilateral kinesthetic teleoperation with time-delayed communication. J. Intell. Robot. Syst. 40(1), 1–22 (2004)Google Scholar
  31. 31.
    Nuño, E., Basañez, L., Ortega, R.: Passivity-based control for bilateral teleoperation: a tutorial. Automatica 47(3), 485–495 (2011)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Deng, Q.-W., Wei, Q., Li, Z.-X.: Analysis of absolute stability for time-delay teleoperation systems. Int. J. Autom. Comput. 4(2), 203–207 (2007)Google Scholar
  33. 33.
    Franken, M., Stramigioli, S., Misra, S., Secchi, C., Macchelli, A.: Bilateral telemanipulation with time delays: a two-layer approach combining passivity and transparency. IEEE Trans. Robot. 27(4), 741–756 (2011)Google Scholar
  34. 34.
    Yang, C., Wang, X., Li, Z., Li, Y., Su, C.-Y.: Teleoperation control based on combination of wave variable and neural networks. IEEE Trans. Syst. Man Cybern. Syst. 47(8), 2125–2136 (2017)Google Scholar
  35. 35.
    Kawashima, K., Tadano, K., Wang, C., Sankaranarayanan, G., Hannaford, B.: Bilateral teleoperation with time delay using modified wave variable based controller. In: 2009. ICRA’09 IEEE International Conference on Robotics and Automation, pp. 4326-4331. IEEE (2009)Google Scholar
  36. 36.
    Bate, L., Cook, C.D., Li, Z.: Reducing wave-based teleoperator reflections for unknown environments. IEEE Trans. Ind. Electron. 2(58), 392–397 (2011)Google Scholar
  37. 37.
    Ye, Y., Liu, P.X.: Improving haptic feedback fidelity in wave-variable-based teleoperation orientated to telemedical applications. IEEE Trans. Instrum. Meas. 58(8), 2847–2855 (2009)Google Scholar
  38. 38.
    Ye, Y., Liu, P.X.: Improving trajectory tracking in wave-variable-based teleoperation. IEEE/ASME Trans. Mechatron. 15(2), 321–326 (2010)Google Scholar
  39. 39.
    Li, H., Kawashima, K.: Achieving stable tracking in wave-variable-based teleoperation. IEEE/ASME Trans. Mechatron. 19(5), 1574–1582 (2014)Google Scholar
  40. 40.
    Zhu, J., He, X., Gueaieb, W.: Trends in the control schemes for bilateral teleoperation with time delay. In: Kamel, M., Karray, F., Gueaieb, W., Khamis, A. (eds.) Autonomous and Intelligent Systems. Lecture Notes in Computer Science, vol. 6752. Springer, Berlin (2011)Google Scholar
  41. 41.
    Muradore, R., Fiorini, P.: A review of bilateral teleoperation algorithms. Acta Polytechnica Hungarica 13 (1), 191–208 (2016)Google Scholar
  42. 42.
    Ghavifekr, A.A., Ghiasi, A.R., Badamchizadeh, M.A.: Discrete-time control of bilateral teleoperation systems: a review. Robotica 36(4), 552–569 (2018)Google Scholar
  43. 43.
    Guo, J., Liu, C., Poignet, P.: Stable and enhanced position-force tracking for bilateral teleoperation with time delay. In: Control Conference (ECC), 2015 European, pp. 1980-1985. IEEE (2015)Google Scholar
  44. 44.
    Lawrence, D.A.: Stability and transparency in bilateral teleoperation. IEEE Trans. Robot. Autom. 9(5), 624–637 (1993)Google Scholar
  45. 45.
    Munir, S., Book, W.J.: Wave-based teleoperation with prediction. In: American Control Conference, 2001. Proceedings of the 2001, vol. 6, pp. 4605–4611. IEEE (2001)Google Scholar
  46. 46.
    Niemeyer, G.D.: Using wave variables in time delayed force reflecting teleoperation, Ph.D. thesis Massachusetts Institute of Technology (1996)Google Scholar
  47. 47.
    Ching, H., Book, W.J.: Internet-based bilateral teleoperation based on wave variable with adaptive predictor and direct drift control. J. Dyn. Syst. Meas. Control. 128(1), 86–93 (2006)Google Scholar
  48. 48.
    Moreira, P., Zemiti, N., Liu, C., Poignet, P.: Viscoelastic model based force control for soft tissue interaction and its application in physiological motion compensation. Comput. Methods Prog. Biomed. 116(2), 52–67 (2014)Google Scholar
  49. 49.
    Sanchez, L.A., Le, M., Liu, C., Zemiti, N., Poignet, P.: The impact of interaction model on stability and transparency in bilateral teleoperation for medical applications. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 1607–1613. IEEE (2012)Google Scholar
  50. 50.
    Hashtrudi-Zaad, K., Salcudean, S.E.: Analysis of control architectures for teleoperation systems with impedance/admittance master and slave manipulators. Int. J. Robot. Res. 20(6), 419–445 (2001)Google Scholar
  51. 51.
    Downing, S.W., Herzog, W.A., McLaughlin, J.S., Gilbert, T.P.: Beating-heart mitral valve surgery: preliminary model and methodology. J. Thorac. Cardiovasc. Surg. 123(6), 1141–1146 (2002)Google Scholar
  52. 52.
    Mitsuishi, M.: Medical robot and master slave system for minimally invasive surgery. In: 2007. CME 2007. IEEE/ICME International Conference on Complex Medical Engineering, pp. 8–13. IEEE (2007)Google Scholar
  53. 53.
    Puvanesarajah, V., Liauw, J.A., Lo, S.-f., Lina, I.A., Witham, T.F.: Techniques and accuracy of thoracolumbar pedicle screw placement. World J. Orthod. 5(2), 112 (2014)Google Scholar
  54. 54.
    Tavakoli, M., Patel, R.V., Moallem, M.: Bilateral control of a teleoperator for soft tissue palpation: design and experiments. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006. pp. 3280-3285. IEEE (2006)Google Scholar
  55. 55.
    Christiansson, G.A., Van Der Helm, F.C.: The low-stiffness teleoperator slave—a trade-off between stability and performance. Int. J. Robot. Res. 26(3), 287–299 (2007)Google Scholar
  56. 56.
    Colonnese, N., Okamura, A.M.: Analysis of effective impedance transmitted to the operator in position-exchange bilateral teleoperation. In: World Haptics Conference (WHC), pp. 328–333. IEEE (2017)Google Scholar
  57. 57.
    Moreira, P., Patil, S., Alterovitz, R., Misra, S.: Needle steering in biological tissue using ultrasound-based online curvature estimation. In: Proceedings of IEEE Int Conf Robot Autom, pp. 4368–4373. Hong Kong (2014)Google Scholar
  58. 58.
    Jian, B., Gao, W., Kacher, D., Nevo, E., Fetics, B., Lee, T.C., Jayender, J.: Kalman filter-based EM-optical sensor fusion for needle deflection estimation. Int. J. Comput. Assist. Radiol. Surg. 13(4), 573–583 (2018)Google Scholar

Copyright information

© Springer Nature B.V. 2018

Authors and Affiliations

  1. 1.School of AutomationGuangdong University of TechnologyGuangzhouChina
  2. 2.LIRMMCNRS-University of MontpellierMontpellierFrance

Personalised recommendations