Single Shot Needle Tip Localization in 2D Ultrasound

  • Cosmas MwikirizeEmail author
  • John L. Nosher
  • Ilker Hacihaliloglu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11768)


We present a novel real-time technique for dynamic localization of the needle tip in 2D ultrasound during challenging interventions in which the tip is imperceptible or shaft information is unavailable. We first enhance the needle tip from time-series ultrasound data through digital subtraction of consecutive frames. The enhanced tip image is then fed to a cascade of similar convolutional neural networks: a tip classifier and a tip location regressor. The classifier ascertains tip motion and the regressor directly outputs the coordinates of the tip. Since we do not require needle shaft information, the method achieves efficient localization of both in-plane and out-of-plane needles. Our approach is trained and evaluated on an ex vivo dataset collected using two different ultrasound machines, with in-plane and out-of-plane insertion of 17G and 22G needles in bovine, porcine and chicken tissue. We use 12, 000 frames extracted from 40 video sequences for training and validation, and 500 frames from 20 sequences as test data. The framework achieves a tip localization error of \(0.55\,\pm \,0.07\) mm, and overall processing time of 0.015 s (67 fps). Validation studies against state-of-the-art achieved \(29\%\) and \(509\%\) improvement in accuracy and processing rate respectively. Because of the real-time execution time and accurate tip localization, we believe that our approach is potentially a breakthrough for real-time needle tip localization in challenging ultrasound-guided interventions.


Needle enhancement Needle localization Ultrasound Deep learning 



This work was accomplished with funding support from the North American Spine Society 2017 young investigator award.

Supplementary material

Supplementary material 1 (mp4 42212 KB)


  1. 1.
    Mwikirize, C., Nosher, J.L., Hacihaliloglu, I.: Learning needle tip localization from digital subtraction in 2D ultrasound. Int. J. CARS 14(6), 1017–1026 (2019)CrossRefGoogle Scholar
  2. 2.
    Mwikirize, C., Nosher, J.L., Hacihaliloglu, I.: Convolution neural networks for real-time needle detection and localization in 2D ultrasound. Int. J. CARS 13(5), 647–657 (2018)CrossRefGoogle Scholar
  3. 3.
    Beigi, P., Rohling, R., Salcudean, S., Ng, G.: CASPER: computer-aided segmentation of imperceptible motion-a learning-based tracking of an invisible needle in ultrasound. Int. J. CARS 12(11), 1857–66 (2017)CrossRefGoogle Scholar
  4. 4.
    Pourtaherian, A., et al.: Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks. Int. J. CARS 13(9), 1321–1333 (2018)CrossRefGoogle Scholar
  5. 5.
    Arif, M., Moelker, A., Walsum, T.V.: Automatic needle detection and real-time bi-planar needle visualization during 3D ultrasound scanning of the liver. Med. Image Anal. 53, 104–110 (2019)CrossRefGoogle Scholar
  6. 6.
    Xiang, Y., Schmidt, T., Narayanan, V., Fox, D.: PoseCNN: a convolutional neural network for 6D object pose estimation in cluttered scenes arXiv:1711.00199v3 (2017)
  7. 7.
    Agarwal, N., Krohn-Grimberghe, A., Vyas, R.: Facial key points detection using deep convolutional neural network - NaimishNet arXiv:1710.00977v1 (2017)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Biomedical EngineeringRutgers UniversityPiscatawayUSA
  2. 2.Department of RadiologyRutgers Robert Wood Johnson Medical SchoolNew BrunswickUSA

Personalised recommendations