Detection of Sclerotic Spine Metastases via Random Aggregation of Deep Convolutional Neural Network Classifications

  • Holger R. RothEmail author
  • Jianhua Yao
  • Le Lu
  • James Stieger
  • Joseph E. Burns
  • Ronald M. Summers
Part of the Lecture Notes in Computational Vision and Biomechanics book series (LNCVB, volume 20)


Automated detection of sclerotic metastases (bone lesions) in Computed Tomography (CT) images has potential to be an important tool in clinical practice and research. State-of-the-art methods show performance of 79 % sensitivity or true-positive (TP) rate, at 10 false-positives (FP) per volume. We design a two-tiered coarse-to-fine cascade framework to first operate a highly sensitive candidate generation system at a maximum sensitivity of \(\sim \)92 % but with high FP level (\(\sim \)50 per patient). Regions of interest (ROI) for lesion candidates are generated in this step and function as input for the second tier. In the second tier we generate \(N\) 2D views, via scale, random translations, and rotations with respect to each ROI centroid coordinates. These random views are used to train a deep Convolutional Neural Network (CNN) classifier. In testing, the CNN is employed to assign individual probabilities for a new set of \(N\) random views that are averaged at each ROI to compute a final per-candidate classification probability. This second tier behaves as a highly selective process to reject difficult false positives while preserving high sensitivities. We validate the approach on CT images of 59 patients (49 with sclerotic metastases and 10 normal controls). The proposed method reduces the number of FP/vol. from 4 to 1.2, 7 to 3, and 12 to 9.5 when comparing a sensitivity rates of 60, 70, and 80 % respectively in testing. The Area-Under-the-Curve (AUC) is 0.834. The results show marked improvement upon previous work.


Image Patch Convolutional Neural Network Classification Probability Compute Tomography Volume CADe Scheme 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported by the Intramural Research Program of the NIH Clinical Center.


  1. 1.
    Msaouel, P., Pissimissis, N., Halapas, A., Koutsilieris, M.: Mechanisms of bone metastasis in prostate cancer: clinical implications. Best Pract. Res. Clin. Endocrinol. Metabol. 22(2), 341–355 (2008)CrossRefGoogle Scholar
  2. 2.
    Hitron, A., Adams, V.: The pharmacological management of skeletal-related events from metastatic tumors. Orthopedics 32(3), 188 (2009)CrossRefGoogle Scholar
  3. 3.
    Coleman, R.: Metastatic bone disease: clinical features, pathophysiology and treatment strategies. Cancer Treat. Rev. 27(3), 165–176 (2001)CrossRefGoogle Scholar
  4. 4.
    Saylor, P., Smith, M.: Bone health and prostate cancer. Prostate Cancer Prostatic Dis. 13(1), 20–27 (2010)CrossRefGoogle Scholar
  5. 5.
    Keller, E.T., Brown, J.: Prostate cancer bone metastases promote both osteolytic and osteoblastic activity. J. Cell. Biochem. 91(4), 718–729 (2004)CrossRefGoogle Scholar
  6. 6.
    Lee, R.J., Saylor, P.J., Smith, M.R.: Treatment and prevention of bone complications from prostate cancer. Bone 48(1), 88–95 (2011)CrossRefGoogle Scholar
  7. 7.
    Guise, T.A., Mundy, G.R.: Cancer and bone 1. Endocr. Rev. 19(1), 18–54 (1998)Google Scholar
  8. 8.
    Wiese, T., Yao, J., Burns, J.E., Summers, R.M.: Detection of sclerotic bone metastases in the spine using watershed algorithm and graph cut. In: SPIE Medical Imaging, International Society for Optics and Photonics (2012) 831512–831512Google Scholar
  9. 9.
    Burns, J.E., Yao, J., Wiese, T.S., Muñoz, H.E., Jones, E.C., Summers, R.M.: Automated detection of sclerotic metastases in the thoracolumbar spine at ct. Radiology 268(1), 69–78 (2013)CrossRefGoogle Scholar
  10. 10.
    Hammon, M., Dankerl, P., Tsymbal, A., Wels, M., Kelm, M., May, M., Suehling, M., Uder, M., Cavallaro, A.: Automatic detection of lytic and blastic thoracolumbar spine metastases on computed tomography. Eur. Radiol. 23(7), 1862–1870 (2013)CrossRefGoogle Scholar
  11. 11.
    Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25 (2012)Google Scholar
  12. 12.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional neural networks. arXiv:1311.2901 (2013)
  13. 13.
    Prasoon, A., Petersen, K., Igel, C., Lauze, F., Dam, E., Nielsen, M.: Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. MICCAI (2013)Google Scholar
  14. 14.
    Roth, H.R., Lu, L., Seff, A., Cherry, K.M., Hoffman, J., Wang, S., Liu, J., Turkbey, E., Summers, R.M.: A new 2.5D representation for lymph node detection using random sets of deep convolutional neural network observations. MICCAI 2014 (in-print), arXiv:1406.2639 [cs.CV] (2014)
  15. 15.
    Cireşan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Mitosis detection in breast cancer histology images with deep neural networks. MICCAI (2013)Google Scholar
  16. 16.
    Lu, L., Liu, M., Ye, X., Yu, S., Huang, H.: Coarse-to-fine classification via parametric and nonparametric models for computer-aided diagnosis. In: Proceedings of the 20th ACM international conference on Information and knowledge management, ACM (2011) 2509–2512Google Scholar
  17. 17.
    Wiese, T., Burns, J., Yao, J., Summers, R.M.: Computer-aided detection of sclerotic bone metastases in the spine using watershed algorithm and support vector machines. In: Biomedical Imaging: From Nano to Macro, 2011 IEEE International Symposium on, IEEE (2011) 152–155Google Scholar
  18. 18.
    Yao, J., O’Connor, S.D., Summers, R.M.: Automated spinal column extraction and partitioning. In: Biomedical Imaging: Nano to Macro, 2006. 3rd IEEE International Symposium on, IEEE (2006) 390–393Google Scholar
  19. 19.
    Yao, J., O’Connor, S.D., Summers, R.: Computer aided lytic bone metastasis detection using regular ct images. In: Medical Imaging, International Society for Optics and Photonics (2006) 614459–614459Google Scholar
  20. 20.
    Yao, J., Summers, R.M., Hara, A.K.: Optimizing the support vector machines (svm) committee configuration in a colonic polyp cad system. In: Medical Imaging, International Society for Optics and Photonics (2005) 384–392Google Scholar
  21. 21.
    Göktürk, S.B., Tomasi, C., Acar, B., Beaulieu, C.F., Paik, D.S., Jeffrey, R.B., Yee, J., Napel, Y.: A statistical 3-d pattern processing method for computer-aided detection of polyps in ct colonography. IEEE Trans. Med. Imag. 20, 1251–1260 (2001)CrossRefGoogle Scholar
  22. 22.
    Wan, L., Zeiler, M., Zhang, S., Cun, Y.L., Fergus, R.: Regularization of neural networks using dropconnect. In: Proceedings of International Conference on Machine Learning (ICML-13) (2013)Google Scholar
  23. 23.
    Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)

Copyright information

© Springer International Publishing Switzerland (outside the USA) 2015

Authors and Affiliations

  • Holger R. Roth
    • 1
    Email author
  • Jianhua Yao
    • 1
  • Le Lu
    • 1
  • James Stieger
    • 1
  • Joseph E. Burns
    • 2
  • Ronald M. Summers
    • 1
  1. 1.Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging SciencesNational Institutes of Health Clinical CenterBethesdaUSA
  2. 2.Department of Radiological SciencesUniversity of CaliforniaIrvine, OrangeUSA

Personalised recommendations