Coupling Convolutional Neural Networks and Hough Voting for Robust Segmentation of Ultrasound Volumes

  • Christine Kroll
  • Fausto Milletari
  • Nassir Navab
  • Seyed-Ahmad Ahmadi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9796)


This paper analyses the applicability and performance of Convolutional Neural Networks (CNN) to localise and segment anatomical structures in medical volumes under clinically realistic constraints: small amount of available training data, the need of a short processing time and limited computational resources. Our segmentation approach employs CNNs for simultaneous classification and feature extraction. A Hough voting strategy has been developed in order to automatically localise and segment the anatomy of interest. Our results show (i) improved robustness, due to the inclusion of prior shape knowledge, (ii) highly accurate segmentation even when only small datasets are available during training, (iii) speed and computational requirements that match those that are usually present in clinical settings.


Training Volume Convolutional Neural Network Transcranial Ultrasound Segmentation Volume Semantic Segmentation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This study was funded by the Lüneburg Heritage and Deutsche Forschungsgesellschaft (DFG) Grant BO 1895/4-1. We gratefully acknowledge the support of NVIDIA Corporation in donating a Tesla K40 GPU for this study.


  1. 1.
    Ahmadi, S.A., Baust, M., Karamalis, A., Plate, A., Bötzel, K., Klein, T., Navab, N.: Midbrain segmentation in transcranial 3D ultrasound for Parkinson diagnosis. Med. Image Comput. Comput. Assist. Interv. 14(Pt 3), 362–369 (2011)Google Scholar
  2. 2.
    Berg, D., Seppi, K., Behnke, S., Liepelt, I., Schweitzer, K., Stockner, H., Wollenweber, F., Gaenslen, A., Mahlknecht, P., Spiegel, J., Godau, J., Huber, H., Srulijes, K., Kiechl, S., Bentele, M., Gasperi, A., Schubert, T., Hiry, T., Probst, M., Schneider, V., Klenk, J., Sawires, M., Willeit, J., Maetzler, W., Fassbender, K., Gasser, T., Poewe, W.: Enlarged substantia nigra hyperechogenicity and risk for Parkinson disease: a 37-month 3-center study of 1847 older persons. Arch. Neurol. 68(7), 932–937 (2011)CrossRefGoogle Scholar
  3. 3.
    Cernazanu-Glavan, C., Holban, S.: Segmentation of bone structure in X-ray images using convolutional neural network. Adv. Electr. Comput. Eng. 13(1), 87–94 (2013)CrossRefGoogle Scholar
  4. 4.
    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv preprint arXiv:1412.7062 (2014)
  5. 5.
    Ciresan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: Advances in Neural Information Processing Systems, pp. 2843–2851 (2012)Google Scholar
  6. 6.
    Engel, K., Toennies, K.D.: Segmentation of the midbrain in transcranial sonographies using a two-component deformable model. In: 12th Annual Conference on Medical Image Understanding and Analysis, pp. 3–7. Citeseer (2009)Google Scholar
  7. 7.
    Farfade, S.S., Saberian, M.J., Li, L.J.: Multi-view face detection using deep convolutional neural networks. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 643–650. ACM (2015)Google Scholar
  8. 8.
    Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.M., Larochelle, H.: Brain tumor segmentation with deep neural networks. arXiv preprint arXiv:1505.03540 (2015)
  9. 9.
    Kim, M., Wu, G., Shen, D.: Unsupervised deep learning for hippocampus segmentation in 7.0 Tesla MR images. In: Wu, G., Zhang, D., Shen, D., Yan, P., Suzuki, K., Wang, F. (eds.) MLMI 2013. LNCS, vol. 8184, pp. 1–8. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  10. 10.
    Lee, N., Laine, A.F., Klein, A.: Towards a deep learning approach to brain parcellation. In: 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 321–324. IEEE (2011)Google Scholar
  11. 11.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Google Scholar
  12. 12.
    Melinščak, M., Prentašić, P., Lončarić, S.: Retinal vessel segmentation using deep neural networks. In: VISAPP 2015 (10th International Conference on Computer Vision Theory and Applications) (2015)Google Scholar
  13. 13.
    Middleton, I., Damper, R.I.: Segmentation of magnetic resonance images using a combination of neural networks and active contour models. Med. Eng. Phys. 26(1), 71–86 (2004)CrossRefGoogle Scholar
  14. 14.
    Milletari, F., Ahmadi, S.A., Kroll, C., Hennersperger, C., Tombari, F., Shah, A., Plate, A., Bötzel, K., Navab, N.: Robust Segmentation of Various Anatomies in 3D Ultrasound Using Hough Forests and Learned Data Representations. Medical Image Computing and Computer Assisted Interventions (2015, to appear)Google Scholar
  15. 15.
    Milletari, F., Ahmadi, S.A., Kroll, C., Plate, A., Rozanski, V., Maiostre, J., Levin, J., Dietrich, O., Ertl-Wagner, B., Bötzel, K., et al.: Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound. arXiv preprint arXiv:1601.07014 (2016)
  16. 16.
    Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. arXiv preprint arXiv:1606.04797 (2016)
  17. 17.
    Muja, M., Lowe, D.G.: Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans. Pattern Anal. Mach. Intell. 36, 2227–2240 (2014)CrossRefGoogle Scholar
  18. 18.
    Plate, A., Ahmadi, S.A., Pauly, O., Klein, T., Navab, N., Bötzel, K.: Three-dimensional sonographic examination of the midbrain for computer-aided diagnosis of movement disorders. Ultrasound Med. Biol. 38(12), 2041–2050 (2012)CrossRefGoogle Scholar
  19. 19.
    Prasoon, A., Petersen, K., Igel, C., Lauze, F., Dam, E., Nielsen, M.: Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part II. LNCS, vol. 8150, pp. 246–253. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  20. 20.
    Riegler, G., Ferstl, D., Rüther, M., Bischof, H.: Hough networks for head pose estimation and facial feature localization. J. Comput. Vis. 101(3), 437–458 (2013)CrossRefGoogle Scholar
  21. 21.
    Ronneberger, O., Philipp, F., Thomas, B.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24574-4_28 CrossRefGoogle Scholar
  22. 22.
    Roth, H.R., Lu, L., Seff, A., Cherry, K.M., Hoffman, J., Wang, S., Liu, J., Turkbey, E., Summers, R.M.: A new 2.5D representation for Lymph node detection using random sets of deep convolutional neural network observations. In: Hata, N., Barillot, C., Hornegger, J., Howe, R., Golland, P. (eds.) MICCAI 2014, Part I. LNCS, vol. 8673, pp. 520–527. Springer, Heidelberg (2014)Google Scholar
  23. 23.
    Sakalauskas, A., Lukoševičius, A., Laučkaitė, K., Jegelevičius, D., Rutkauskas, S.: Automated segmentation of transcranial sonographic images in the diagnostics of Parkinsons disease. Ultrasonics 53(1), 111–121 (2013)CrossRefGoogle Scholar
  24. 24.
    Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: Overfeat: integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013)
  25. 25.
    Song, Y., Zhang, L., Chen, S., Ni, D., Lei, B., Wang, T.: Accurate segmentation of cervical cytoplasm and nuclei based on multiscale convolutional network and graph partitioning. IEEE Trans. Biomed. Eng. 62(10), 2421–2433 (2015)CrossRefGoogle Scholar
  26. 26.
    Turaga, S.C., Murray, J.F., Jain, V., Roth, F., Helmstaedter, M., Briggman, K., Denk, W., Seung, H.S.: Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Comput. 22(2), 511–538 (2010)CrossRefzbMATHGoogle Scholar
  27. 27.
    Urban, G., Bendszus, M., Hamprecht, F., Kleesiek, J.: Multi-modal brain tumor segmentation using deep convolutional neural networks. In: MICCAI BraTS (Brain Tumor Segmentation) Challenge. Proceedings, Winning Contribution, pp. 31–35 (2014)Google Scholar
  28. 28.
    Walter, U., Dressler, D., Probst, T., Wolters, A., Abu-Mugheisib, M., Wittstock, M., Benecke, R.: Transcranial brain sonography findings in discriminating between parkinsonism and idiopathic Parkinson disease. Arch. Neurol. 64(11), 1635–1640 (2007)CrossRefGoogle Scholar
  29. 29.
    Wang, S., Yin, Y., Cao, G., Wei, B., Zheng, Y., Yang, G.: Hierarchical retinal blood vessel segmentation based on feature and ensemble learning. Neurocomputing 149, 708–717 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Christine Kroll
    • 1
  • Fausto Milletari
    • 1
  • Nassir Navab
    • 1
    • 2
  • Seyed-Ahmad Ahmadi
    • 3
  1. 1.Computer Aided Medical ProceduresTechnische Universität MünchenMunichGermany
  2. 2.Computer Aided Medical ProceduresJohns Hopkins UniversityBaltimoreUSA
  3. 3.Department of NeurologyKlinikum Grosshadern, Ludwig-Maximilians-Universität MünchenMunichGermany

Personalised recommendations