Automatic segmentation of bone surfaces from ultrasound using a filter-layer-guided CNN

  • Ahmed Z. Alsinan
  • Vishal M. Patel
  • Ilker HacihalilogluEmail author
Original Article



Ultrasound (US) provides real-time, two-/three-dimensional safe imaging. Due to these capabilities, it is considered a safe alternative to intra-operative fluoroscopy in various computer-assisted orthopedic surgery (CAOS) procedures. However, interpretation of the collected bone US data is difficult due to high levels of noise, various imaging artifacts, and bone surfaces response appearing several millimeters (mm) in thickness. For US-guided CAOS procedures, it is an essential objective to have a segmentation mechanism, that is both robust and computationally inexpensive.


In this paper, we present our development of a convolutional neural network-based technique for segmentation of bone surfaces from in vivo US scans. The novelty of our proposed design is that it utilizes fusion of feature maps and employs multi-modal images to abate sensitivity to variations caused by imaging artifacts and low intensity bone boundaries. B-mode US images, and their corresponding local phase filtered images are used as multi-modal inputs for the proposed fusion network. Different fusion architectures are investigated for fusing the B-mode US image and the local phase features.


The proposed methods was quantitatively and qualitatively evaluated on 546 in vivo scans by scanning 14 healthy subjects. We achieved an average F-score above 95% with an average bone surface localization error of 0.2 mm. The reported results are statistically significant compared to state-of-the-art.


Reported accurate and robust segmentation results make the proposed method promising in CAOS applications. Further extensive validations are required in order to fully understand the clinical utility of the proposed method.


Orthopedic surgery Segmentation Ultrasound Bone Deep learning 



This work was supported in part by 2017 North American Spine Society Young Investigator Award.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.


  1. 1.
    Baka N, Leenstra S, van Walsum T (2017) Ultrasound aided vertebral level localization for lumbar surgery. IEEE Trans Med Imaging 36(10):2138–2147CrossRefGoogle Scholar
  2. 2.
    Cernazanu-Glavan C, Holban S (2013) Segmentation of bone structure in x-ray images using convolutional neural network. Adv Electr Comput Eng 13(1):87–94CrossRefGoogle Scholar
  3. 3.
    Feichtenhofer C, Pinz A, Zisserman A (2016) Convolutional two-stream network fusion for video action recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1933–1941Google Scholar
  4. 4.
    Hacihaliloglu I (2017) Enhancement of bone shadow region using local phase-based ultrasound transmission maps. Int J Comput Assist Radiol Surg 12(6):951–960CrossRefGoogle Scholar
  5. 5.
    Hacihaliloglu I (2017) Localization of bone surfaces from ultrasound data using local phase information and signal transmission maps. In: International workshop and challenge on computational methods and clinical applications in musculoskeletal imaging. Springer, pp 1–11Google Scholar
  6. 6.
    Hacihaliloglu I (2017) Ultrasound imaging and segmentation of bone surfaces: a review. Technology 5(2):74–80CrossRefGoogle Scholar
  7. 7.
    Hacihaliloglu I, Guy P, Hodgson AJ, Abugharbieh R (2014) Volume-specific parameter optimization of 3d local phase features for improved extraction of bone surfaces in ultrasound. Int J Med Robot Comput Assist Surg 10(4):461–473CrossRefGoogle Scholar
  8. 8.
    Hacihaliloglu I, Rasoulian A, Rohling RN, Abolmaesumi P (2014) Local phase tensor features for 3-d ultrasound to statistical shape \(+\) pose spine model registration. IEEE Trans Med Imaging 33(11):2167–2179CrossRefGoogle Scholar
  9. 9.
    Hazirbas C, Ma L, Domokos C, Cremers D (2016) Fusenet: incorporating depth into semantic segmentation via fusion-based CNN architecture. In: Asian conference on computer vision. Springer, pp 213–228Google Scholar
  10. 10.
    Jain V, Bollmann B, Richardson M, Berger DR, Helmstaedter MN, Briggman KL, Denk W, Bowden JB, Mendenhall JM, Abraham WC et al (2010) Boundary learning by optimization with topological constraints. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2488–2495Google Scholar
  11. 11.
    Laina I, Rupprecht C, Belagiannis V, Tombari F, Navab N (2016) Deeper depth prediction with fully convolutional residual networks. In: 2016 Fourth international conference on 3D vision (3DV), pp 239–248Google Scholar
  12. 12.
    Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440Google Scholar
  13. 13.
    Organization WH (2003) The burden of musculoskeletal conditions at the start of the new millennium: report of a who scientific group. WHO Technical Report Series 919Google Scholar
  14. 14.
    Rand WM (1971) Objective criteria for the evaluation of clustering methods. J Am Stat Assoc 66(336):846–850CrossRefGoogle Scholar
  15. 15.
    Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241Google Scholar
  16. 16.
    Salehi M, Prevost R, Moctezuma JL, Navab N, Wein W (2017) Precise ultrasound bone registration with learning-based segmentation and speed of sound calibration. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 682–690Google Scholar
  17. 17.
    United States Bone and Joint Initiative (2014) The burden of musculoskeletal diseases in the United States (BMUS), 3rd edn. Rosemont, IL. Accessed on 13March 2018
  18. 18.
    Valada A, Vertens J, Dhall A, Burgard W (2017) Adapnet: Adaptive semantic segmentation in adverse environmental conditions. In: 2017 IEEE International conference on robotics and automation (ICRA). IEEE, pp 4644–4651Google Scholar
  19. 19.
    Villa M, Dardenne G, Nasan M, Letissier H, Hamitouche C, Stindel E (2018) FCN-based approach for the automatic segmentation of bone surfaces in ultrasound images. Int J Comput Assist Radiol Surg 13(11):1707–1716CrossRefGoogle Scholar
  20. 20.
    Wang P, Patel VM, Hacihaliloglu I (2018) Simultaneous segmentation and classification of bone surfaces from ultrasound using a multi-feature guided CNN. In: Medical image computing and computer assisted interventionGoogle Scholar

Copyright information

© CARS 2019

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringRutgers UniversityPiscatawayUSA
  2. 2.Department of Biomedical EngineeringRutgers UniversityPiscatawayUSA
  3. 3.Rutgers University Robert Wood Johnson Medical SchoolNew BrunswickUSA

Personalised recommendations