Visual Servoing for Beating Heart Surgery

  • Wael Bachta
  • Pierre Renaud
  • Ezio Malis
  • Koichi Hashimoto
  • Jacques Gangloff
Part of the Lecture Notes in Control and Information Sciences book series (LNCIS, volume 401)


Off-pump coronary artery bypass grafting (CABG) is still a technically challenging procedure. The existing mechanical stabilizers used for local suppression of the heart excursion have demonstrated significant residual motion, which could lead to a lack of accuracy in the surgical task, particularly in a minimally invasive surgery (MIS) context. Robots can help the surgeon by actively compensating for the heart motion using visual servoing. Various sensors like endoscopic camera, ultrasound imaging or even magnetic resonance imaging (MRI) can be used to provide the feedback of the visual loop. Advanced control approaches like predictive, repetitive or robust control can enhance the compensation accuracy. They rely on a model that uses physiological inputs to predict the motion of the myocardium in real-time.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    Bachta, W., Renaud, P., Cuvillon, L., Laroche, E., Forgione, A., Gangloff, J.: Motion prediction for computer-assisted beating heart surgery. IEEE Transactions on Biomedical Engineering (to appear) Google Scholar
  3. 3.
    Bachta, W., Renaud, P., Laroche, E., Forgione, A., Gangloff, J.: Cardiolock: an active cardiac stabilizer, first in vivo experiments using a new robotized device. Computer Aided Surgery 13(5), 243–254 (2008)Google Scholar
  4. 4.
    Bachta, W., Renaud, P., Laroche, E., Gangloff, J.: Cardiolock2: Parallel singularities for the design of an active heart stabilizer. In: IEEE International Conference on Robotics and Automation (2009)Google Scholar
  5. 5.
    Baker, S., Matthews, I.: Lucas-kanade 20 years on: A unifying framework. Int. J. of Computer Vision 56(3) (2004)Google Scholar
  6. 6.
    Bebek, O., Cavusoglu, M.: Intelligent control algorithms for robotic-assisted beating heart surgery. IEEE Transactions on Robotics 23(3), 468–480 (2007)CrossRefGoogle Scholar
  7. 7.
    Benhimane, S., Malis, E.: Homography-based 2D visual tracking and servoing. Int. J. Robotics Research 26(7), 661–676 (2007)CrossRefGoogle Scholar
  8. 8.
    Boyd, W.D., Desai, N.D., Rizzo, D.F.D., Novick, R.J., McKenzie, F.N., Menkis, A.H.: Off-pump surgery decreases post-operative complications and resource utilization in the elderly. Annals of Thoracic Surgery 68, 1490–1493 (1999)CrossRefGoogle Scholar
  9. 9.
    Camacho, E.F., Bordons, C.: Model Predictive Control. Springer, London (1999)Google Scholar
  10. 10.
    Cattin, P., Dave, H., Grunenfelder, J., Szekely, G., Turina, M., Zund, G.: Trajectory of coronary motion and its significance in robotic motion cancellation. European Journal of Cardio-thoracic Surgery 25(5), 786–790 (2004)CrossRefGoogle Scholar
  11. 11.
    Choi, D., Riviere, C.: Flexure-based manipulator for active handhled microsurgical instrument. In: 27th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (2005)Google Scholar
  12. 12.
    Crick, S.J., Sheppard, M.N., Ho, S.Y., Gebstein, L., Anderson, R.H.: Anatomy of the pig heart: comparisons with normal human cardiac structure. Journal of Anatomy 193(1), 105 (1998)CrossRefGoogle Scholar
  13. 13.
    Cuvillon, L., Gangloff, J., De Mathelin, M., Forgione, A.: Towards robotized beating heart TECABG: assessment of the heart dynamics using high-speed vision. Computer aided surgery 11(5), 267–277 (2006)CrossRefGoogle Scholar
  14. 14.
    Dario, P., Hannaford, B., Menciassi, A.: Smart surgical tools and augmenting devices. IEEE Transactions on Robotics and Automation 19(5), 782–792 (2003)CrossRefGoogle Scholar
  15. 15.
    Doyle, J., Glover, K., Khargonekar, P., Francis, B.: State-space solutions to standard H2 and H ∞  control problems. IEEE Transactions on Automatic Control 34(8), 831–847 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Duchemin, G., Poignet, P., Dombre, E., Pierrot, F.: Medically safe and sound [human-friendly robot dependability]. IEEE Robotics and Automation Magazine 11(2), 46–55 (2004)CrossRefGoogle Scholar
  17. 17.
    Espiau, B., Chaumette, F., Rives, P.: A new approach to visual servoing in robotics. IEEE Transactions on Robotics and Automation 8(3), 313–326 (1992)CrossRefGoogle Scholar
  18. 18.
    Falk, V., Walther, T., Stein, H., Jacobs, S., Walther, C., Rastan, A., Wimmer-Greinecker, G., Mohr, F.: Facilated endoscopic beating heart coronary artery bypass grafting using a magnetic coupling device. The Journal of Thoracic and Cardiovascular Surgery 126(5), 1575–1579 (2003)CrossRefGoogle Scholar
  19. 19.
    Ginhoux, R., Gangloff, J., de Mathelin, M., Soler, L., Sanchez, M., Marescaux, J.: Active filtering of physiological motion in robotized surgery using predictive control. IEEE Transactions on Robotics 21(1), 67–79 (2005)CrossRefGoogle Scholar
  20. 20.
  21. 21.
    Gummert, J., Opfermann, U., Jacobs, S., Walther, T., Kempfert, J., Mohr, F., Falk, V.: Anastomotic devices for coronary artery bypass grafting: technological options and potential pitfalls. Computers in Biology and Medicine 37, 1384–1393 (2007)CrossRefGoogle Scholar
  22. 22.
    Hager, G., Belhumeur, P.: Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(10), 1125–1139 (1998)CrossRefGoogle Scholar
  23. 23.
  24. 24.
    Jin, H., Favaro, P., Soatto, S.: Real-time feature tracking and outlier rejection with changes in illumination. In: Proc. IEEE Int. Conf. on Computer Vision, vol. 1, pp. 684–689 (2001)Google Scholar
  25. 25.
    Jurie, F., Dhome, M.: Hyperplane approximation for template matching. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 996–1000 (2002)CrossRefGoogle Scholar
  26. 26.
    Kettler, D., Plowes, R., Novotny, P., Vasilyev, N., del Nido, P., Howe, R.: An active motion compensation instrument for beating heart mitral valve surgery. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 2007. IROS 2007, pp. 1290–1295 (2007)Google Scholar
  27. 27.
    Lemma, M., Mangini, A., Redaelli, A., Acocella, F.: Do cardiac stabilizers really stabilize? experimental quantitative analysis of mechanical stabilization. Interactive Cardiovascular and Thoracic Surgery (2005)Google Scholar
  28. 28.
    Loisance, D., Nakashima, K., Kirsch, M.: Computer-assisted coronary surgery: lessons from an initial experience. Interactive Cardiovascular and Thoracic Surgery 4, 398–401 (2005)CrossRefGoogle Scholar
  29. 29.
    Malis, E.: An efficient unified approach to direct visual tracking of rigid and deformable surfaces. In: Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 2729–2734 (2007)Google Scholar
  30. 30.
  31. 31.
    Nakamura, Y., Kishi, K., Kawakami, H.: Heartbeat synchronization for robotic cardiac surgery. In: IEEE International Conference on Robotics and Automation, vol. 2, pp. 2014–2019 (2001)Google Scholar
  32. 32.
    Noce, A., Triboulet, J., Poignet., P.: Efficient tracking of the heart using texture. In: Proceedings of IEEE International Conference of the Engineering in Medicine and Biology Society (EMBS 2007), Lyon, France, vol. 1, pp. 4480–4483 (2007)Google Scholar
  33. 33.
    OECD: Health data: Statistics and indicators for 30 countries,
  34. 34.
    Ortmaier, T.: Motion compensation in minimally invasive robotic surgery. Ph.D. thesis, Technischen Universität München (2002)Google Scholar
  35. 35.
    Patronik, N., Zenati, M., Riviere, C.: Preliminary evaluation of a mobile robotic device for navigation and intervention on the beating heart. Computer Aided Surgery 10(5), 225–232 (2005)CrossRefGoogle Scholar
  36. 36.
    Ranftl, A., Cuvillon, L., Gangloff, J., Sloten, J.: High speed visual servoing with ultrasonic motors. In: IEEE Int. Conf. on Robotics and Automation, pp. 4472–4477 (2007)Google Scholar
  37. 37.
    Salcudean, S., Lichtenstein, S., Trejos, A., Sassani F.and Gihuly, T.: Moving tracking platform for relative motion cancellation for surgery. US Patent 6368332B1, April 9 (2002)Google Scholar
  38. 38.
    Stoyanov, D., Mylonas, G.P., Deligianni, F., Darzi, A.: Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3750, pp. 139–146. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  39. 39.
    Sugimoto, S., Okutomi, M.: A direct and efficient method for piecewise-planar surface reconstruction from stereo images. In: Proc. IEEE Society Conf. on Comptuer Vision and Pattern Recognition, pp. 1–8 (2007)Google Scholar
  40. 40.
    Takata, U.: Thesis. Ph.D. thesis, Graduate School of Frontier Sciences, The University of Tokyo (2009)Google Scholar
  41. 41.
    Thakral, A., Wallace, J., Tolmin, D., Seth, N., Thakor, N.: Surgical motion adaptive robotic technology (S.M.A.R.T): Taking the motion out of physiological motion. In: Niessen, W.J., Viergever, M.A. (eds.) MICCAI 2001. LNCS, vol. 2208, pp. 317–325. Springer, Heidelberg (2001)Google Scholar
  42. 42.
    Trejos, A., Salcudean, S., Sassani, F., Lichtenstein, S.: On the feasibility of a moving support for surgery on the beating heart. In: Taylor, C., Colchester, A. (eds.) MICCAI 1999. LNCS, vol. 1679, pp. 1088–1097. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  43. 43.
    Van Dijk, D., Jansen, E., Hijman, R.E.A.: Cognitive outcome after off-pump and on-pump coronary artery bypass graft surgery. Journal of the American Medical Association 287(11), 1405–1412 (2002)CrossRefGoogle Scholar
  44. 44.
    Yuen, S., Novotny, P., Howe, R.: Quasiperiodic predictive filtering for robot-assisted beating heart surgery. In: IEEE International Conference on Robotics and Automation, pp. 3875–3880 (2008)Google Scholar

Copyright information

© Springer London 2010

Authors and Affiliations

  • Wael Bachta
    • 1
  • Pierre Renaud
    • 1
  • Ezio Malis
    • 2
  • Koichi Hashimoto
    • 3
  • Jacques Gangloff
    • 1
  1. 1.LSIITUniversity of StrasbourgIllkirchFrance
  2. 2.INRIASophia AntipolisFrance
  3. 3.Tohoku UniversitySendaiJapan

Personalised recommendations