Precise needle placement is an important task during several medical procedures. Ultrasound imaging is often used to guide the needle toward the target region in soft tissue. This task remains challenging due to the user’s dependence on image quality, limited field of view, moving target, and moving needle. In this paper, we present a novel dual-robot framework for robotic needle insertions under robotic ultrasound guidance.
We integrated force-controlled ultrasound image acquisition, registration of preoperative and intraoperative images, vision-based robot control, and target localization, in combination with a novel needle tracking algorithm. The framework allows robotic needle insertion to target a preoperatively defined region of interest while enabling real-time visualization and adaptive trajectory planning to provide safe and quick interactions. We assessed the framework by considering both static and moving targets embedded in water and tissue-mimicking gelatin.
The presented dual-robot tracking algorithms allow for accurate needle placement, namely to target the region of interest with an error around 1 mm.
To the best of our knowledge, we show the first use of two independent robots, one for imaging, the other for needle insertion, that are simultaneously controlled using image processing algorithms. Experimental results show the feasibility and demonstrate the accuracy and robustness of the process.
This is a preview of subscription content, log in to check access.
Buy single article
Instant unlimited access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Abolhassani N, Patel R, Moallem M (2007) Needle insertion into soft tissue: a survey. Med Eng Phys 29(4):413–431
Curley SA, Izzo F, Delrio P, Ellis LM, Granchi J, Vallone P, Fiore F, Pignata S, Daniele B, Cremona F (1999) Radiofrequency ablation of unresectable primary and metastatic hepatic malignancies: results in 123 patients. Ann Surg 230(1):1
Yılmaz S, Özdoğan M, Cevener M, Ozluk A, Kargi A, Kendiroglu F, Ogretmen I, Yildiz A (2016) Use of cryoablation beyond the prostate, Insights into imaging, pp. 1–10
Taylor RH (2006) A perspective on medical robotics. Proc IEEE 94(9):1652–1664
Esposito M, Busam B, Hennersperger C, Rackerseder J, Lu A, Navab N, Frisch B (2015) Cooperative robotic gamma imaging: enhancing us-guided needle biopsy, in Medical Image Computing and Computer-Assisted Intervention-MICCAI. Springer, Berlin
Graumann C, Fuerst B, Hennersperger C, Bork F, Navab N (2016) Robotic ultrasound trajectory planning for volume of interest coverage. In: IEEE international conference on robotics and automation (ICRA)
Zettinig O, Fuerst B, Kojcev R, Esposito M, Salehi M, Wein W, Rackerseder J, Sinibaldi E, Frisch B, Navab N (2016) Toward real-time 3d ultrasound registration-based visual servoing for interventional navigation. In: IEEE international conference on robotics and automation (ICRA)
Hong J, Dohi T, Hashizume M, Konishi K, Hata N (2004) An ultrasound-driven needle-insertion robot for percutaneous cholecystostomy. Phys Med Biol 49(3):441
Wei Z, Ding M, Downey D, Fenster A (2005) 3d trus guided robot assisted prostate brachytherapy. In Medical image computing and computer-assisted intervention-MICCAI. Springer, Berlin, pp 17–24
DiMaio SP, Salcudean S (2005) Needle steering and motion planning in soft tissues. IEEE Trans Biomed Eng 52(6):965–974
Alterovitz R, Goldberg K, Okamura A (2005) Planning for steerable bevel-tip needle insertion through 2d soft tissue with obstacles, in Robotics and Automation, 2005. Proceedings of the IEEE international conference on IEEE, pp. 1640–1645
Abayazid M, Vrooijink GJ, Patil S, Alterovitz R, Misra S (2014) Experimental evaluation of ultrasound-guided 3D needle steering in biological tissue. Int J Comput Assist Radiol Surg 9(6):931–939
Niccolini M, Castelli V, Diversi C, Kang B, Mussa F, Sinibaldi E (2016) Development and preliminary assessment of a robotic platform for neuroendoscopy based on a lightweight robot. Int J Med Robot Comput Assist Surg 12(1):4–17
Shiu YC, Ahmad S (1989) Calibration of wrist-mounted robotic sensors by solving homogeneous transform equations of the form AX=XB. IEEE Trans Robot Autom 5(1):16–29
Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334
Mercier L, Langø T, Lindseth F, Collins LD (2005) A review of calibration techniques for freehand 3-d ultrasound systems. Ultrasound Med Biol 31(2):143–165
Rusu RB, Marton ZC, Blodow N, Dolha M, Beetz M (2008) Towards 3D point cloud based object maps for household environments. Rob Auton Syst 56(11):927–941
Karamalis A, Wein W, Kutter O, Navab N (2009) Fast hybrid freehand ultrasound volume reconstruction. In: SPIE medical imaging. International society for optics and photonics, pp 726 114–726 114
Albu-Schäffer A, Ott C, Frese U, Hirzinger G (2003) Cartesian impedance control of redundant robots: Recent results with the DLR-Light-Weight-arms. In Proceedings on international conference on robotics and automation, vol. 3. IEEE, 2003, pp. 3704–3709
Wein W, Ladikos A, Fuerst B, Shah A, Sharma K, Navab N (2013) Global registration of ultrasound to mri using the lc2 metric for enabling neurosurgical guidance. In: Medical image computing and computer-assisted intervention–MICCAI. Springer, Berlin, pp. 34–41
Fuerst B, Wein W, Müller M, Navab N (2014) Automatic ultrasound-MRI registration for neurosurgery using the 2D and 3D LC2 metric. Med Image Anal 18(8):1312–1319
Powell MJ (2009) The BOBYQA algorithm for bound constrained optimization without derivatives
Chatelain P, Krupa A, Marchal M (2013) Real-time needle detection and tracking using a visually servoed 3d ultrasound probe. In Robotics and Automation (ICRA), 2013 IEEE International Conference on IEEE, pp 1676–1681
Otsu N (1975) A threshold selection method from gray-level histograms. Automatica 11(285–296):23–27
Uherčík M, Liebgott H, Kybic J, Cachard C (2009) Needle localization methods in 3D ultrasound data. In International congress on ultrasonics, pp. 11–17
Julier s, Uhlmann J (2004) Unscented filtering and nonlinear estimation. In Proceedings of the IEEE, vol 92, no 3, pp. 401–422
Bischoff R, Kurth J, Schreiber G, Koeppe R, Albu-Schaeffer A, Beyer A, Eiberger O, Haddadin S, Stemmer A, Grunwald G, Hirzinger G (2010) The kuka-dlr lightweight robot arm—a new reference platform for robotics research and manufacturing, In Robotics (ISR), 2010 41st International Symposium on and 2010 6th German Conference on Robotics (ROBOTIK), June 2010, pp. 1–8
Schreiber G, Stemmer A, Bischoff R (2010) The fast research interface for the KUKA lightweight robot. In: IEEE workshop on innovative robot control architectures for demanding applications how to modify and enhance commercial controllers
Lasso A, Heffter T, Rankin A, Pinter C, Ungi T, Fichtinger G (2014) Plus: Open-source toolkit for ultrasound-guided intervention systems. IEEE Trans Biomed Eng 10:2527–2537
Cook JR, Bouchard RR, Emelianov SY (2011) Tissue-mimicking phantoms for photoacoustic and ultrasonic imaging. Biomed Opt Express 2(11):3193–3206
Hall TJ, Bilgen M, Insana MF, Krouskop TA (1997) Phantom materials for elastography. IEEE Trans Ultrason Ferroelectr Freq Control 44(6):1355–1365
Künzli BM, Abitabile P, Maurer CA (2011) Radiofrequency ablation of liver tumors: actual limitations and potential solutions in the future. World J Hepatol 3(1):8–14
Bhan SN, Pautler SE, Shayegan B, Voss MD, Goeree RA, You JJ (2013) Active surveillance, radiofrequency ablation, or cryoablation for the nonsurgical management of a small renal mass: a cost-utility analysis. Ann Surg Oncol 20(11):3675–3684
The authors wish to thank Wolfgang Wein and his team (ImFusion GmbH, Munich, Germany) for the great support and opportunity to use the ImFusion framework. This work was partially funded by the Bayerische Forschungsstiftung award number AZ-1072-13 (project RoBildOR).
Conflict of interest
The authors declare that they have no conflict of interest.
Human and animal rights
This article does not contain any studies with human participants or animals performed by any of the authors. This article does not contain patient data.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Kojcev, R., Fuerst, B., Zettinig, O. et al. Dual-robot ultrasound-guided needle placement: closing the planning-imaging-action loop. Int J CARS 11, 1173–1181 (2016) doi:10.1007/s11548-016-1408-1
- Robotic system and software
- Software architecture
- Robotic architecture and devices
- Instrument and patient localization and tracking