Multiple Device Segmentation for Fluoroscopic Imaging Using Multi-task Learning

  • Katharina BreiningerEmail author
  • Tobias Würfl
  • Tanja Kurzendorfer
  • Shadi Albarqouni
  • Marcus Pfister
  • Markus Kowarschik
  • Nassir Navab
  • Andreas Maier
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11043)


For endovascular aortic repair (EVAR), integrating preoperative information of the aortic anatomy with intraoperative fluoroscopy can aid in reducing radiation exposure, contrast agent and procedure time. However, the quality of this fusion may deteriorate over the course of the intervention due to patient movement or deformation of the vasculature caused by interventional tools. Automatically detecting the instruments present in the X-ray image can help to assess the degree of deterioration, trigger automatic re-registration or aid in automatic workflow phase detection and process modeling. In this work, we investigate a flexible approach to segment different devices based on fully convolutional neural networks using multi-task learning. We evaluate the proposed approach on a set of 38 X-ray images acquired during EVAR interventions by targeting the segmentation of aortic stents, stiff guidewires and pigtail catheters. We compare the results to the performance of single-task networks. We manage to keep similar performance compared to single-task networks with Dice coefficients between 0.95 and 0.80 depending on the device, while speeding up computation by a factor of two.


Endovascular Aortic Repair Multi-task learning Convolutional Neural Networks Stents Catheters Guidewires X-ray 


  1. 1.
    Ambrosini, P., Ruijters, D., Niessen, W.J., Moelker, A., van Walsum, T.: Fully automatic and real-time catheter segmentation in X-ray fluoroscopy. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 577–585. Springer, Cham (2017). Scholar
  2. 2.
    Baur, C., Albarqouni, S., Demirci, S., Navab, N., Fallavollita, P.: CathNets: detection and single-view depth prediction of catheter electrodes. In: Zheng, G., Liao, H., Jannin, P., Cattin, P., Lee, S.-L. (eds.) MIAR 2016. LNCS, vol. 9805, pp. 38–49. Springer, Cham (2016). Scholar
  3. 3.
    Breininger, K., Albarqouni, S., Kurzendorfer, T., Pfister, M., Kowarschik, M., Maier, A.: Intraoperative stent segmentation in X-ray fluoroscopy for endovascular aortic repair. IJCARS 13, 1221–1231 (2018). Scholar
  4. 4.
    Demirci, S., et al.: 3D stent recovery from one X-ray projection. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6891, pp. 178–185. Springer, Heidelberg (2011). Scholar
  5. 5.
    Hoffmann, M.: Electrophysiology catheter detection and reconstruction from two views in fluoroscopic images. IEEE Trans. Med. Imaging 35(2), 567–579 (2015). Scholar
  6. 6.
    Kauffmann, C.: Source of errors and accuracy of a two-dimensional/three-dimensional fusion road map for endovascular aneurysm repair of abdominal aortic aneurysm. JVIR 26(4), 544–551 (2015). Scholar
  7. 7.
    Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: International Conference on Learning Representations (ICLR) (2015).
  8. 8.
    Lessard, S., et al.: Automatic detection of selective arterial devices for advanced visualization during abdominal aortic aneurysm endovascular repair. Med. Eng. Phys. 37(10), 979–986 (2015). Scholar
  9. 9.
    McNally, M.M., Scali, S.T., Feezor, R.J., Neal, D., Huber, T.S., Beck, A.W.: Three-dimensional fusion computed tomography decreases radiation exposure, procedure time, and contrast use during fenestrated endovascular aortic repair. J. Vasc. Surg. 61(2), 309–316 (2015). Scholar
  10. 10.
    Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: IEEE International Conference on 3DVision (2016)Google Scholar
  11. 11.
    Moeskops, P., et al.: Deep learning for multi-task medical image segmentation in multiple modalities. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 478–486. Springer, Cham (2016). Scholar
  12. 12.
    Panuccio, G., et al.: Computer-aided endovascular aortic repair using fully automated two-and three-dimensional fusion imaging. J. Vasc. Surg. 64, 1587–1594 (2016). Scholar
  13. 13.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  14. 14.
    Tacher, V., et al.: Image guidance for endovascular repair of complex aortic aneurysms: comparison of two-dimensional and three-dimensional angiography and image fusion. JVIR 24(11), 1698–1706 (2013). Scholar
  15. 15.
    Toth, D., Pfister, M., Maier, A., Kowarschik, M., Hornegger, J.: Adaption of 3D models to 2D X-ray images during endovascular abdominal aneurysm repair. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 339–346. Springer, Cham (2015). Scholar
  16. 16.
    Volpi, D., Sarhan, M.H., Ghotbi, R., Navab, N., Mateus, D., Demirci, S.: Online tracking of interventional devices for endovascular aortic repair. IJCARS 10(6), 773–781 (2015). Scholar
  17. 17.
    Wang, C.: Segmentation of multiple structures in chest radiographs using multi-task fully convolutional networks. In: Sharma, P., Bianchi, F.M. (eds.) SCIA 2017. LNCS, vol. 10270, pp. 282–289. Springer, Cham (2017). Scholar
  18. 18.
    Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 94–108. Springer, Cham (2014). Scholar
  19. 19.
    Zhong, X., Hoffmann, M., Strobel, N., Maier, A.: Improved semi-automatic basket catheter reconstruction from two X-ray views. In: Tolxdorff, T., Deserno, T.M., Handels, H., Meinzer, H.P. (eds.) Bildverarbeitung für die Medizin 2016. I, pp. 26–31. Springer, Heidelberg (2016). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Katharina Breininger
    • 1
    Email author
  • Tobias Würfl
    • 1
  • Tanja Kurzendorfer
    • 1
    • 2
  • Shadi Albarqouni
    • 3
  • Marcus Pfister
    • 2
  • Markus Kowarschik
    • 2
  • Nassir Navab
    • 3
    • 4
  • Andreas Maier
    • 1
    • 5
  1. 1.Pattern Recognition LabFriedrich-Alexander-Universität Erlangen-NürnbergErlangenGermany
  2. 2.Siemens Healthcare GmbHForchheimGermany
  3. 3.Computer Aided Medical Procedures (CAMP)Technische Universität MünchenMunichGermany
  4. 4.Whiting School of EngineeringJohns Hopkins UniversityBaltimoreUSA
  5. 5.School of Advanced Optical Techniques (SAOT)ErlangenGermany

Personalised recommendations