Journal of Digital Imaging

, Volume 32, Issue 5, pp 793–807 | Cite as

A Deep Learning-Based Approach for the Detection and Localization of Prostate Cancer in T2 Magnetic Resonance Images

  • Ruba AlkadiEmail author
  • Fatma Taher
  • Ayman El-baz
  • Naoufel Werghi


We address the problem of prostate lesion detection, localization, and segmentation in T2W magnetic resonance (MR) images. We train a deep convolutional encoder-decoder architecture to simultaneously segment the prostate, its anatomical structure, and the malignant lesions. To incorporate the 3D contextual spatial information provided by the MRI series, we propose a novel 3D sliding window approach, which preserves the 2D domain complexity while exploiting 3D information. Experiments on data from 19 patients provided for the public by the Initiative for Collaborative Computer Vision Benchmarking (I2CVB) show that our approach outperforms traditional pattern recognition and machine learning approaches by a significant margin. Particularly, for the task of cancer detection and localization, the system achieves an average AUC of 0.995, an accuracy of 0.894, and a recall of 0.928. The proposed mono-modal deep learning-based system performs comparably to other multi-modal MR-based systems. It could improve the performance of a radiologist in prostate cancer diagnosis and treatment planning.


Magnetic resonance imaging Prostate cancer Deep convolutional encoder-decoder 



The authors would also like to thank Dr. Waleed Hassen and Dr. Eric Vens from Cleveland Clinic, Abu Dhabi, and Dr. Salah El-Rai from Sheikh Khalifa General Hospital for their support and collaboration.

Funding Information

This work is support by a research grant from Al-Jalila foundation ref. AJF-201616.

Compliance with Ethical Standards

Conflict of interests

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.


  1. 1.
    Alkadi R, Taher F, El-Baz A, Naoufel W: Early diagnosis and staging of prostate cancer using magnetic resonance imaging: State of the art and perspectives. In: Prostate cancer imaging: An engineering and clinical perspective, chapter 2. Taylor & Francis, In-pressGoogle Scholar
  2. 2.
    Badrinarayanan V, Kendall A, Cipolla R: Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39 (12): 2481–2495, 2017CrossRefGoogle Scholar
  3. 3.
    Chan I, Wells W, Mulkern RV, Haker S, Zhang J, Zou KH, Maier SE, Tempany C: Detection of prostate cancer by integration of line-scan diffusion, t2-mapping and t2-weighted magnetic resonance imaging; a multichannel statistical classifier. Med Phys 30 (9): 2390–2398, 2003CrossRefGoogle Scholar
  4. 4.
    Chawla NV, Bowyer KW, Hall LO, Philip Kegelmeyer W: Smote: synthetic minority over-sampling technique. J Artif Intell Res 16: 321–357, 2002CrossRefGoogle Scholar
  5. 5.
    Csurka G, Larlus D, Perronnin F, Meylan F: What is a good evaluation measure for semantic segmentation?. In: BMVC, volume 27, p 2013. Citeseer, 2013Google Scholar
  6. 6.
    Drozdzal M, Chartrand G, Vorontsov E, Shakeri M, Di Jorio L, An T, Romero A, Bengio Y, Pal C, Kadoury S: Learning normalized inputs for iterative estimation in medical image segmentation. Med Image Anal 44: 1–13, 2018CrossRefGoogle Scholar
  7. 7.
    Eigen D, Fergus R: Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture.. In: Proceedings of the IEEE international conference on computer vision, 2015, pp 2650–2658Google Scholar
  8. 8.
    Ferlay J, Soerjomataram I, Dikshit R, Eser S, Mathers C, Rebelo M, Parkin DM, Forman D, Bray F: Cancer incidence and mortality worldwide: sources, methods and major patterns in globocan 2012. Int J Cancer 136 (5): E359–86, 2015CrossRefGoogle Scholar
  9. 9.
    Fütterer JJ: Multiparametric mri in the detection of clinically significant prostate cancer. Korean J Radiol 18 (4): 597–606, 2017CrossRefGoogle Scholar
  10. 10.
    Garcia-Garcia A, Orts-Escolano S, Oprea S, Villena-Martinez V, Garcia-Rodriguez J: A review on deep learning techniques applied to semantic segmentation, 2017. arXiv:1704.06857
  11. 11.
    Greenspan H, van Ginneken B, Summers RM: Guest editorial deep learning in medical imaging overview and future promise of an exciting new technique. IEEE Trans Med Imaging 35 (5): 1153–1159, 2016CrossRefGoogle Scholar
  12. 12.
    Guo Y, Gao Y, Shen D: Deformable mr prostate segmentation via deep feature learning and sparse patch matching. IEEE Trans Med Imaging 35 (4): 1077–1089, 2016CrossRefGoogle Scholar
  13. 13.
    Hall MA: Correlation-based feature selection of discrete and numeric class machine learning, 2000Google Scholar
  14. 14.
    Han H, Wang W-Y, Mao B-H: Borderline-smote: a new over- sampling method in imbalanced data sets learning.. In: International Conference on Intelligent Computing, pp 878–887. Springer , 2005Google Scholar
  15. 15.
    He K, Zhang X, Ren S, Sun J: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification.. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034, 2015Google Scholar
  16. 16.
    Kiraly AP, Nader CA, Tuysuzoglu A, Grimm R, Kiefer B, El-Zehiry N, Kamen A: Deep convolutional encoder-decoders for prostate cancer detection and classification.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 489–497. Springer, 2017Google Scholar
  17. 17.
    Kohl S, Bonekamp D, Schlemmer H-P, Yaqubi K, Hohenfellner M, Hadaschik B, Radtke J-P, Maier-Hein K: Adversarial networks for the detection of aggressive prostate cancer, 2017. arXiv:1702.08014 1702.08014
  18. 18.
    Kumar D, Wong A, Clausi DA: Lung nodule classification using deep features in ct images.. In: 2015 12th conference on computer and robot vision (CRV), pp 133–138. IEEE, 2015Google Scholar
  19. 19.
    Lemaitre G: Computer-aided diagnosis for prostate cancer using multi-parametric magnetic resonance imaging. PhD thesis, Ph. D. dissertation, Universitat de Girona and Université de Bourgogne, 2016Google Scholar
  20. 20.
    Lemaître G, Martí R, Freixenet J, Vilanova JC, Walker PM, Meriaudeau F: Computer-aided detection and diagnosis for prostate cancer based on mono and multi-parametric mri: A review. Comput Biol Med 60: 8–31, 2015CrossRefGoogle Scholar
  21. 21.
    Lemaitre G, Martí R, Rastgoo M, Mériaudeau F: Computer-aided detection for prostate cancer detection based on multi-parametric magnetic resonance imaging.. In: Engineering in Medicine and Biology Society (EMBC), 2017 39th Annual International Conference of the IEEE, pp 3138–3141. IEEE, 2017Google Scholar
  22. 22.
    Litjens G, Debats O, Barentsz J, Karssemeijer N, Huisman H: Computer-aided detection of prostate cancer in mri. IEEE Trans Med Imaging 33 (5): 1083–1092, 2014CrossRefGoogle Scholar
  23. 23.
    Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI: A survey on deep learning in medical image analysis. Med Image Anal 42: 60–88, 2017CrossRefGoogle Scholar
  24. 24.
    Long J, Shelhamer E, Darrell T: Fully convolutional networks for semantic segmentation.. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp 3431–3440Google Scholar
  25. 25.
    Lv D, Guo X, Wang X, Zhang J, Fang J: Computerized characterization of prostate cancer by fractal analysis in mr images. J Magn Reson Imaging 30 (1): 161–168, 2009CrossRefGoogle Scholar
  26. 26.
    Mani I, Zhang I: knn approach to unbalanced data distributions: a case study involving information extraction.. In: Proceedings of workshop on learning from imbalanced datasets, vol 126, 2003Google Scholar
  27. 27.
    Mazurowski MA, Buda M, Saha A, Bashir MR: Deep learning in radiology: an overview of the concepts and a survey of the state of the art, 2018. arXiv:1802.08717
  28. 28.
    Puech P, Betrouni N, Makni N, Dewalle A-S, Villers A, Lemaitre L: Computer-assisted diagnosis of prostate cancer using dce-mri data: design, implementation and preliminary results. Int J Comput. Assist Radiol Surg 4 (1): 1–10, 2009CrossRefGoogle Scholar
  29. 29.
    Qi CR, Hao SU, Nießner M, Dai A, Yan M, Guibas LJ: Volumetric and multi-view cnns for object classification on 3d data.. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp 5648–5656Google Scholar
  30. 30.
    Rampun A, Zheng L, Malcolm P, Tiddeman B, Zwiggelaar R: Computer-aided detection of prostate cancer in t2-weighted mri within the peripheral zone. Phys Med Biol 61 (13): 4796, 2016CrossRefGoogle Scholar
  31. 31.
    Reda I, Shalaby A, Khalifa F, Elmogy M, Aboulfotouh A, El-Ghar MA, Hosseini-Asl E, Werghi N, Keynton R, El-Baz A: Computer-aided diagnostic tool for early detection of prostate cancer.. In: IEEE international conference on image processing (ICIP), pp 2668–2672. IEEE, 2016Google Scholar
  32. 32.
    Ronneberger O, Fischer P, Brox T: U-net: Convolutional networks for biomedical image segmentation.. In: International Conference on Medical image computing and computer-assisted intervention, pp 234–241. Springer, 2015Google Scholar
  33. 33.
    Roth HR, Lu Le, Liu J, Yao J, Seff A, Cherry K, Kim L, Summers RM: Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans Med Imaging 35 (5): 1170–1181, 2016CrossRefGoogle Scholar
  34. 34.
    Roth HR, Lu L, Seff A, Cherry KM, Hoffman J, Wang S, Liu J, Turkbey E, Summers RM: A new 2.5 d representation for lymph node detection using random sets of deep convolutional neural network observations.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 520–527. Springer, 2014Google Scholar
  35. 35.
    Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L: ImageNet large scale visual recognition challenge. Int J Comput Vis 115 (3): 211–252, 2015CrossRefGoogle Scholar
  36. 36.
    Simonyan K, Zisserman A: Very deep convolutional networks for large-scale image recognition, 2014. arXiv:1409.1556
  37. 37.
    Smith MR, Martinez T, Giraud-Carrier C: An instance level analysis of data complexity. Mach Learn 95 (2): 225–256, 2014CrossRefGoogle Scholar
  38. 38.
    Tiwari P, Kurhanewicz J, Madabhushi A: Multi-kernel graph embedding for detection, gleason grading of prostate cancer via mri/mrs. Med Image Anal 17 (2): 219–235, 2013CrossRefGoogle Scholar
  39. 39.
    Tiwari P, Rosen M, Madabhushi A: A hierarchical spectral clustering and nonlinear dimensionality reduction scheme for detection of prostate cancer from magnetic resonance spectroscopy (mrs). Med Phys 36 (9Part1): 3927–3939, 2009CrossRefGoogle Scholar
  40. 40.
    Tiwari P, Viswanath S, Kurhanewicz J, Sridhar A, Madabhushi A: Multimodal wavelet embedding representation for data combination (maweric): integrating magnetic resonance imaging and spectroscopy for prostate cancer detection. NMR Biomed 25 (4): 607–619, 2012CrossRefGoogle Scholar
  41. 41.
    Trigui R, Mitéran J, Walker PM, Sellami L, Ben Hamida A: Automatic classification and localization of prostate cancer using multi-parametric mri/mrs. Biomed Signal Process Control 31: 189–198, 2017CrossRefGoogle Scholar
  42. 42.
    Tustison NJ, Avants BB, Cook PA, Zheng Y, Egan A, Yushkevich PA, Gee JC: N4itk: Improved n3 bias correction. IEEE Trans Med Imaging 29 (6): 1310–1320, 2010CrossRefGoogle Scholar
  43. 43.
    Viswanath S, Bloch BN, Genega E, Rofsky N, Lenkinski R, Chappelow J, Toth R, Madabhushi A: A comprehensive segmentation, registration, and cancer detection scheme on 3 tesla in vivo prostate dce-mri.. In: International conference on medical image computing and computer-assisted intervention, pp 662–669. Springer, 2008Google Scholar
  44. 44.
    Viswanath SE, Bloch NB, Chappelow JC, Toth R, Rofsky NM, Genega EM, Lenkinski RE, Madabhushi A: Central gland and peripheral zone prostate tumors have significantly different quantitative imaging signatures on 3 tesla endorectal, in vivo t2- weighted mr imagery. J Magn Reson Imaging 36 (1): 213–224, 2012CrossRefGoogle Scholar
  45. 45.
    Vos PC, Barentsz JO, Karssemeijer N, Huisman HJ: Automatic computer-aided detection of prostate cancer based on multiparametric magnetic resonance image analysis. Phys Med Biol 57 (6): 1527, 2012CrossRefGoogle Scholar
  46. 46.
    Wang L, Zwiggelaar R: 3d texton based prostate cancer detection using multiparametric magnetic resonance imaging.. In: Annual conference on medical image understanding and analysis, pp 309– 319. Springer, 2017Google Scholar
  47. 47.
    Wang Z, Liu C, Cheng D, Wanga L, Yang X, Chengb K-TT: Automated detection of clinically significant prostate cancer in mp-mri images based on an end-to-end deep neural network. IEEE Transactions on Medical Imaging, 2018Google Scholar
  48. 48.
    Zhirong WU, Song S, Khosla A, Fisher YU, Zhang L, Tang X, Xiao J: 3d shapenets; A deep representation for volumetric shapes.. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp 1912–1920Google Scholar
  49. 49.
    Yang X, Liu C, Wang Z, Yang J, Min HL, Wang L, Cheng K-TT: Co-trained convolutional neural networks for automated detection of prostate cancer in multi-parametric mri. Med Image Anal 42: 212–227, 2017CrossRefGoogle Scholar
  50. 50.
    Yang X, Wang Z, Liu C, Le HM, Chen J, Cheng K-TT, Wang L: Joint detection and diagnosis of prostate cancer in multi-parametric mri based on multimodal convolutional neural networks.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 426–434. Springer, 2017Google Scholar
  51. 51.
    Lequan YU, Yang X, Chen H, Qin J, Heng P-A: Volumetric convnets with mixed residual connections for automated prostate segmentation from 3d mr images.. In: AAAI, 2017, pp 66–72Google Scholar
  52. 52.
    Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A: Learning deep features for discriminative localization.. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2921–2929. IEEE, 2016Google Scholar

Copyright information

© Society for Imaging Informatics in Medicine 2018

Authors and Affiliations

  1. 1.Khalifa University of Science and TechnologyAbu DhabiUnited Arab Emirates
  2. 2.University of LouisvilleLouisvilleUSA

Personalised recommendations