Dual-robot ultrasound-guided needle placement: closing the planning-imaging-action loop

  • 624 Accesses

  • 8 Citations



Precise needle placement is an important task during several medical procedures. Ultrasound imaging is often used to guide the needle toward the target region in soft tissue. This task remains challenging due to the user’s dependence on image quality, limited field of view, moving target, and moving needle. In this paper, we present a novel dual-robot framework for robotic needle insertions under robotic ultrasound guidance.


We integrated force-controlled ultrasound image acquisition, registration of preoperative and intraoperative images, vision-based robot control, and target localization, in combination with a novel needle tracking algorithm. The framework allows robotic needle insertion to target a preoperatively defined region of interest while enabling real-time visualization and adaptive trajectory planning to provide safe and quick interactions. We assessed the framework by considering both static and moving targets embedded in water and tissue-mimicking gelatin.


The presented dual-robot tracking algorithms allow for accurate needle placement, namely to target the region of interest with an error around 1 mm.


To the best of our knowledge, we show the first use of two independent robots, one for imaging, the other for needle insertion, that are simultaneously controlled using image processing algorithms. Experimental results show the feasibility and demonstrate the accuracy and robustness of the process.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6


  1. 1.

    Abolhassani N, Patel R, Moallem M (2007) Needle insertion into soft tissue: a survey. Med Eng Phys 29(4):413–431

  2. 2.

    Curley SA, Izzo F, Delrio P, Ellis LM, Granchi J, Vallone P, Fiore F, Pignata S, Daniele B, Cremona F (1999) Radiofrequency ablation of unresectable primary and metastatic hepatic malignancies: results in 123 patients. Ann Surg 230(1):1

  3. 3.

    Yılmaz S, Özdoğan M, Cevener M, Ozluk A, Kargi A, Kendiroglu F, Ogretmen I, Yildiz A (2016) Use of cryoablation beyond the prostate, Insights into imaging, pp. 1–10

  4. 4.

    Taylor RH (2006) A perspective on medical robotics. Proc IEEE 94(9):1652–1664

  5. 5.

    Esposito M, Busam B, Hennersperger C, Rackerseder J, Lu A, Navab N, Frisch B (2015) Cooperative robotic gamma imaging: enhancing us-guided needle biopsy, in Medical Image Computing and Computer-Assisted Intervention-MICCAI. Springer, Berlin

  6. 6.

    Graumann C, Fuerst B, Hennersperger C, Bork F, Navab N (2016) Robotic ultrasound trajectory planning for volume of interest coverage. In: IEEE international conference on robotics and automation (ICRA)

  7. 7.

    Zettinig O, Fuerst B, Kojcev R, Esposito M, Salehi M, Wein W, Rackerseder J, Sinibaldi E, Frisch B, Navab N (2016) Toward real-time 3d ultrasound registration-based visual servoing for interventional navigation. In: IEEE international conference on robotics and automation (ICRA)

  8. 8.

    Hong J, Dohi T, Hashizume M, Konishi K, Hata N (2004) An ultrasound-driven needle-insertion robot for percutaneous cholecystostomy. Phys Med Biol 49(3):441

  9. 9.

    Wei Z, Ding M, Downey D, Fenster A (2005) 3d trus guided robot assisted prostate brachytherapy. In Medical image computing and computer-assisted intervention-MICCAI. Springer, Berlin, pp 17–24

  10. 10.

    DiMaio SP, Salcudean S (2005) Needle steering and motion planning in soft tissues. IEEE Trans Biomed Eng 52(6):965–974

  11. 11.

    Alterovitz R, Goldberg K, Okamura A (2005) Planning for steerable bevel-tip needle insertion through 2d soft tissue with obstacles, in Robotics and Automation, 2005. Proceedings of the IEEE international conference on IEEE, pp. 1640–1645

  12. 12.

    Abayazid M, Vrooijink GJ, Patil S, Alterovitz R, Misra S (2014) Experimental evaluation of ultrasound-guided 3D needle steering in biological tissue. Int J Comput Assist Radiol Surg 9(6):931–939

  13. 13.

    Niccolini M, Castelli V, Diversi C, Kang B, Mussa F, Sinibaldi E (2016) Development and preliminary assessment of a robotic platform for neuroendoscopy based on a lightweight robot. Int J Med Robot Comput Assist Surg 12(1):4–17

  14. 14.

    Shiu YC, Ahmad S (1989) Calibration of wrist-mounted robotic sensors by solving homogeneous transform equations of the form AX=XB. IEEE Trans Robot Autom 5(1):16–29

  15. 15.

    Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334

  16. 16.

    Mercier L, Langø T, Lindseth F, Collins LD (2005) A review of calibration techniques for freehand 3-d ultrasound systems. Ultrasound Med Biol 31(2):143–165

  17. 17.

    Rusu RB, Marton ZC, Blodow N, Dolha M, Beetz M (2008) Towards 3D point cloud based object maps for household environments. Rob Auton Syst 56(11):927–941

  18. 18.

    Karamalis A, Wein W, Kutter O, Navab N (2009) Fast hybrid freehand ultrasound volume reconstruction. In: SPIE medical imaging. International society for optics and photonics, pp 726 114–726 114

  19. 19.

    Albu-Schäffer A, Ott C, Frese U, Hirzinger G (2003) Cartesian impedance control of redundant robots: Recent results with the DLR-Light-Weight-arms. In Proceedings on international conference on robotics and automation, vol. 3. IEEE, 2003, pp. 3704–3709

  20. 20.

    Wein W, Ladikos A, Fuerst B, Shah A, Sharma K, Navab N (2013) Global registration of ultrasound to mri using the lc2 metric for enabling neurosurgical guidance. In: Medical image computing and computer-assisted intervention–MICCAI. Springer, Berlin, pp. 34–41

  21. 21.

    Fuerst B, Wein W, Müller M, Navab N (2014) Automatic ultrasound-MRI registration for neurosurgery using the 2D and 3D LC2 metric. Med Image Anal 18(8):1312–1319

  22. 22.

    Powell MJ (2009) The BOBYQA algorithm for bound constrained optimization without derivatives

  23. 23.

    Chatelain P, Krupa A, Marchal M (2013) Real-time needle detection and tracking using a visually servoed 3d ultrasound probe. In Robotics and Automation (ICRA), 2013 IEEE International Conference on IEEE, pp 1676–1681

  24. 24.

    Otsu N (1975) A threshold selection method from gray-level histograms. Automatica 11(285–296):23–27

  25. 25.

    Uherčík M, Liebgott H, Kybic J, Cachard C (2009) Needle localization methods in 3D ultrasound data. In International congress on ultrasonics, pp. 11–17

  26. 26.

    Julier s, Uhlmann J (2004) Unscented filtering and nonlinear estimation. In Proceedings of the IEEE, vol 92, no 3, pp. 401–422

  27. 27.

    Bischoff R, Kurth J, Schreiber G, Koeppe R, Albu-Schaeffer A, Beyer A, Eiberger O, Haddadin S, Stemmer A, Grunwald G, Hirzinger G (2010) The kuka-dlr lightweight robot arm—a new reference platform for robotics research and manufacturing, In Robotics (ISR), 2010 41st International Symposium on and 2010 6th German Conference on Robotics (ROBOTIK), June 2010, pp. 1–8

  28. 28.

    Schreiber G, Stemmer A, Bischoff R (2010) The fast research interface for the KUKA lightweight robot. In: IEEE workshop on innovative robot control architectures for demanding applications how to modify and enhance commercial controllers

  29. 29.

    Lasso A, Heffter T, Rankin A, Pinter C, Ungi T, Fichtinger G (2014) Plus: Open-source toolkit for ultrasound-guided intervention systems. IEEE Trans Biomed Eng 10:2527–2537

  30. 30.

    Cook JR, Bouchard RR, Emelianov SY (2011) Tissue-mimicking phantoms for photoacoustic and ultrasonic imaging. Biomed Opt Express 2(11):3193–3206

  31. 31.

    Hall TJ, Bilgen M, Insana MF, Krouskop TA (1997) Phantom materials for elastography. IEEE Trans Ultrason Ferroelectr Freq Control 44(6):1355–1365

  32. 32.

    Künzli BM, Abitabile P, Maurer CA (2011) Radiofrequency ablation of liver tumors: actual limitations and potential solutions in the future. World J Hepatol 3(1):8–14

  33. 33.

    Bhan SN, Pautler SE, Shayegan B, Voss MD, Goeree RA, You JJ (2013) Active surveillance, radiofrequency ablation, or cryoablation for the nonsurgical management of a small renal mass: a cost-utility analysis. Ann Surg Oncol 20(11):3675–3684

Download references


The authors wish to thank Wolfgang Wein and his team (ImFusion GmbH, Munich, Germany) for the great support and opportunity to use the ImFusion framework. This work was partially funded by the Bayerische Forschungsstiftung award number AZ-1072-13 (project RoBildOR).

Author information

Correspondence to Risto Kojcev.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors. This article does not contain patient data.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 12286 KB)

Supplementary material 1 (mp4 12286 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kojcev, R., Fuerst, B., Zettinig, O. et al. Dual-robot ultrasound-guided needle placement: closing the planning-imaging-action loop. Int J CARS 11, 1173–1181 (2016) doi:10.1007/s11548-016-1408-1

Download citation


  • Robotic system and software
  • Software architecture
  • Robotics
  • Robotic architecture and devices
  • Ultrasound
  • Instrument and patient localization and tracking