A Gesture-Based Interface for Remote Surgery

  • Irene Cheng
  • Richard Moreau
  • Nathaniel Rossol
  • Arnaud Leleve
  • Patrick Lermusiux
  • Antoine Millon
  • Anup BasuEmail author


There has been a great deal of research activity in computer- and robot-assisted surgeries in recent years. Some of the advances have included robotic hip surgery, image-guided endoscopic surgery, and the use of intra-operative MRI to assist in neurosurgery. However, most of the work in the literature assumes that all of the expert surgeons are physically present close to the location of a surgery. A new direction that is now worth investigating is assisting in performing surgeries remotely. As a first step in this direction, this chapter presents a system that can detect movement of hands and fingers, and thereby detect gestures, which can be used to control a catheter remotely. Our development is aimed at performing remote endovascular surgery by controlling the movement of a catheter through blood vessels. Our hand movement detection is facilitated by sensors, like LEAP, which can track the position of fingertips and the palm. In order to make the system robust to occlusions, we have improved the implementation by optimally integrating the input from two different sensors. Following this step, we identify high-level gestures, like push and turn, to enable remote catheter movements. To simulate a realistic environment we have fabricated a flexible endovascular mold, and also a phantom of the abdominal region with the endovascular mold integrated inside. A mechanical device that can remotely control a catheter based on movement primitives extracted from gestures has been built. Experimental results are shown demonstrating the accuracy of the system.


Gesture recognition Remote control of catheter Endovascular surgery 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



The financial support from Alberta Innovates and INSA, Lyon, France in conducting this research is gratefully acknowledged.


  1. 1.
    S. Guo, H. Kondo, J. Wang, J. Guo, T. Tamiya, A new catheter operating system for medical applications. Proceedings IEEE International Conference on Complex Medical Engineering, Beijing, China, 2007, pp. 82–86Google Scholar
  2. 2.
    L.W. Klein et al., The catheterization laboratory and interventional vascular suite of the future: Anticipating innovations in design and function. Catheter Cardiovasc. Interv. 77, 447–455 (2011)CrossRefGoogle Scholar
  3. 3.
    V. De Sars et al., A practical approach to the design and control of active endoscopes. Mechatronics 20(2), 251–264 (2009)CrossRefGoogle Scholar
  4. 4.
    M. Simi, G. Sardi, P. Valdastri, A. Menciassi, P. Dario, Magnetic levitation camera robot for endoscopic surgery. IEEE International Conference on Robotics and Automation, Shanghai, China, 2011, pp. 5279–5284Google Scholar
  5. 5.
    J. Hao, J. Zhao, M. Li, Spatial continuity incorporated multi-attribute fuzzy clustering algorithm for blood vessels segmentation. Science China Inf. Sci. 53, 752–759 (2010)MathSciNetCrossRefGoogle Scholar
  6. 6.
    M. Mahvash, P. Dupont, Stiffness control of surgical manipulators. IEEE Trans. Robotics 27(2), 334–345 (2011)CrossRefGoogle Scholar
  7. 7.
    B. Lindbald, Commentary on ‘impact of shaggy aorta in patients with abdominal aortic aneurysm following open or endovascular aneurysm repair. Eur. J. Vasc. Endovasc. Surg. 52(5), 620 (Nov. 2016)CrossRefGoogle Scholar
  8. 8.
    S. Monastiriotis, M. Comito, N. Labropoulos, Radiation exposure in endovascular repair of abdominal and thoracic aortic aneurysms. J. Vasc. Surg. 62(3), 753–761 (Sep 2015)CrossRefGoogle Scholar
  9. 9.
    S. Yoshinaga, K. Mabuchi, A.J. Sigurdson, M.M. Doody, E. Ron, Cancer risks among radiologists and radiologic technologists: Review of epidemiologic studies. Radiology 233, 313–321 (2004)CrossRefGoogle Scholar
  10. 10.
    T.R. Koenig, D. Wolff, F.A. Mettler, L.K. Wagner, Skin injuries from fluoroscopically guided procedures. Am. J. Roentgenol. 177, 3–20 (July 2001)CrossRefGoogle Scholar
  11. 11.
    W.K. Wong et al., Endovascular stent graft repair is an effective and safe alternative therapy for arteriovenous graft Pseudoaneurysms. Eur. J. Vasc. Endovasc. Surg. 52(5), 682–688 (November 2016)CrossRefGoogle Scholar
  12. 12.
    M.R. Smeds, A.A. Duncan, M.P. Harlander-Locke, P.F. Lawrence, S. Lyden, J. Fatima, M.K. Eskandari, Treatment and outcomes of aortic Endograft infection. J. Vasc. Surg. 63(2), 332–340 (February 2016)CrossRefGoogle Scholar
  13. 13.
    X. Chaufour, J. Gaudric, Y. Goueffic, R.H. Khodja, P. Feugier, S. Malikov, G. Beraud, J.-B. Ricco, A multicenter experience with infected abdominal aortic Endograft explantation. J. Vasc. Surg. 65(2), 372–380 (2017)CrossRefGoogle Scholar
  14. 14.
    R. Aggarwal et al., Virtual reality simulation training can improve inexperienced surgeons’ endovascular skills. Eur. J. Vasc. Endovasc. Surg. 31(6), 588–593 (2006)CrossRefGoogle Scholar
  15. 15.
    A. Saratzis, T. Calderbank, D. Sidloff, M.J. Bown, R.S. Davies, Role of simulation in endovascular aneurysm repair (EVAR) training: A preliminary study. Eur. J. Vasc. Endovasc. Surg. 53(2), 193–198 (2017)CrossRefGoogle Scholar
  16. 16.
    S. Sinceri, M. Carbone, M. Marconi, A. Moglia, M. Ferrari, V. Ferrari, Basic endovascular skills trainer: A surgical simulator for the training of novice practitioners of endovascular procedures. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2015, 5102–5105 (2015)Google Scholar
  17. 17.
    L. Shi, I. Cheng, A. Basu, Anatomy preserving 3D model decomposition based on robust skeleton-surface node correspondence. IEEE International Conference on Multimedia and Expo, 6 p 2011Google Scholar
  18. 18.
    I. Cheng, A. Firouzmanesh, A. Leleve, R. Shen, R. Moreau, V. Brizzi, M.-T. Pham, P. Lermusiaux, T. Redarce, A. Basu, Enhanced Segmentation and Skeletonization for Endovascular Surgical Planning (SPIE Medical Imaging, San Diego, February 2012), p. 7Google Scholar
  19. 19.
    N. Rossol, I. Cheng, A. Basu, A multisensor technique for gesture recognition through intelligent skeletal pose analysis. IEEE Trans. Human Mach. Syst. 46(3), 350–359 (2016)CrossRefGoogle Scholar
  20. 20.
    J. D. Hoelscher, MS Thesis, Development of a Robust, Accurate Ultrasonic Tracking System for Image Guided Surgery, Southern Illinois University Carbondale, 2008Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Irene Cheng
    • 1
  • Richard Moreau
    • 2
  • Nathaniel Rossol
    • 1
  • Arnaud Leleve
    • 2
  • Patrick Lermusiux
    • 3
  • Antoine Millon
    • 3
  • Anup Basu
    • 1
    Email author
  1. 1.Department of Computing ScienceUniversity of AlbertaEdmontonCanada
  2. 2.Ampere Lab, INSALyonFrance
  3. 3.CHU HospitalLyonFrance

Personalised recommendations