European Radiology

, Volume 28, Issue 10, pp 4407–4417 | Cite as

Computer-aided diagnosis prior to conventional interpretation of prostate mpMRI: an international multi-reader study

  • Matthew D. Greer
  • Nathan Lay
  • Joanna H. Shih
  • Tristan Barrett
  • Leonardo Kayat Bittencourt
  • Samuel Borofsky
  • Ismail Kabakus
  • Yan Mee Law
  • Jamie Marko
  • Haytham Shebel
  • Francesca V. Mertan
  • Maria J. Merino
  • Bradford J. Wood
  • Peter A. Pinto
  • Ronald M. Summers
  • Peter L. Choyke
  • Baris TurkbeyEmail author
Magnetic Resonance



To evaluate if computer-aided diagnosis (CAD) prior to prostate multi-parametric MRI (mpMRI) can improve sensitivity and agreement between radiologists.


Nine radiologists (three each high, intermediate, low experience) from eight institutions participated. A total of 163 patients with 3-T mpMRI from 4/2012 to 6/2015 were included: 110 cancer patients with prostatectomy after mpMRI, 53 patients with no lesions on mpMRI and negative TRUS-guided biopsy. Readers were blinded to all outcomes and detected lesions per PI-RADSv2 on mpMRI. After 5 weeks, readers re-evaluated patients using CAD to detect lesions. Prostatectomy specimens registered to MRI were ground truth with index lesions defined on pathology. Sensitivity, specificity and agreement were calculated per patient, lesion level and zone—peripheral (PZ) and transition (TZ).


Index lesion sensitivity was 78.2% for mpMRI alone and 86.3% for CAD-assisted mpMRI (p = 0.013). Sensitivity was comparable for TZ lesions (78.7% vs 78.1%; p = 0.929); CAD improved PZ lesion sensitivity (84% vs 94%; p = 0.003). Improved sensitivity came from lesions scored PI-RADS < 3 as index lesion sensitivity was comparable at PI-RADS ≥ 3 (77.6% vs 78.1%; p = 0.859). Per patient specificity was 57.1% for CAD and 70.4% for mpMRI (p = 0.003). CAD improved agreement between all readers (56.9% vs 71.8%; p < 0.001).


CAD-assisted mpMRI improved sensitivity and agreement, but decreased specificity, between radiologists of varying experience.

Key Points

• Computer-aided diagnosis (CAD) assists clinicians in detecting prostate cancer on MRI.

• CAD assistance improves agreement between radiologists in detecting prostate cancer lesions.

• However, this CAD system induces more false positives, particularly for less-experienced clinicians and in the transition zone.

• CAD assists radiologists in detecting cancer missed on MRI, suggesting a path for improved diagnostic confidence.


Prostate cancer MRI scans Image interpretation computer assisted Computer-assisted diagnosis 



area under the curve


computer-aided diagnosis


dynamic contrast-enhanced imaging


diffusion-weighted imaging


Gleason score


index of specific agreement


multi-parametric MRI


Prostate Imaging Reporting and Data System


prostate-specific antigen


peripheral zone




transrectal ultrasound


transition zone



The study has received funding by the Intramural Research Program of the NIH, National Cancer Institute, Center for Cancer Research (Grant ZIA BC 010655).

Compliance with ethical standards


The scientific guarantor of this publication is Baris Turkbey, MD.

Conflict of interest

The authors of this manuscript declare relationships with the following companies: Bradford Wood, Philips and InVivo; Ronald Summers, Ping An and iCAD.

Statistics and biometry

One of the authors, Dr. Joanna Shih, has significant statistical expertise.

Ethical approval

Institutional review board approval was obtained.

Informed consent

Written informed consent was obtained from all patients in this study.

Study subjects or cohorts overlap

Some study subjects or cohorts have been previously reported in Greer MD, Shih JH, Lay N, et al. Validation of the dominant sequence paradigm and role of dynamic contrast-enhanced imaging in PI-RADS Version 2. Radiology. 2017;285:859–869.


• retrospective

• diagnostic study

• multicentre study

Supplementary material

330_2018_5374_MOESM1_ESM.docx (18 kb)
ESM 1 (DOCX 18 kb)


  1. 1.
    Society AC (2016) Cancer facts & figures 2016. American Cancer Society, AtlantaGoogle Scholar
  2. 2.
    Graif T, Loeb S, Roehl KA et al (2007) Under diagnosis and over diagnosis of prostate cancer. J Urol 178:88–92CrossRefGoogle Scholar
  3. 3.
    Draisma G, Etzioni R, Tsodikov A et al (2009) Lead time and overdiagnosis in prostate-specific antigen screening: importance of methods and context. J Natl Cancer Inst 101:374–383CrossRefGoogle Scholar
  4. 4.
    Brown AM, Elbuluk O, Mertan F et al (2015) Recent advances in image-guided targeted prostate biopsy. Abdom Imaging 40:1788–1799CrossRefGoogle Scholar
  5. 5.
    Meng X, Rosenkrantz AB, Mendhiratta N et al (2016) Relationship between prebiopsy multiparametric magnetic resonance imaging (MRI), biopsy indication, and MRI-ultrasound fusion-targeted prostate biopsy outcomes. Eur Urol 69:512–517CrossRefGoogle Scholar
  6. 6.
    Siddiqui MM, Rais-Bahrami S, Turkbey B et al (2015) Comparison of MR/ultrasound fusion-guided biopsy with ultrasound-guided biopsy for the diagnosis of prostate cancer. Jama 313:390–397CrossRefGoogle Scholar
  7. 7.
    Schoots IG, Roobol MJ, Nieboer D, Bangma CH, Steyerberg EW, Hunink MG (2015) Magnetic resonance imaging-targeted biopsy may enhance the diagnostic accuracy of significant prostate cancer detection compared to standard transrectal ultrasound-guided biopsy: a systematic review and meta-analysis. Eur Urol 68:438–450CrossRefGoogle Scholar
  8. 8.
    Borkowetz A, Platzek I, Toma M et al (2016) Direct comparison of multiparametric MRI and final histopathology in patients with proven prostate cancer in MRI/ultrasound-fusion biopsy. BJU Int. CrossRefGoogle Scholar
  9. 9.
    Rosenkrantz AB, Deng FM, Kim S et al (2012) Prostate cancer: multiparametric MRI for index lesion localization–a multiple-reader study. AJR Am J Roentgenol 199:830–837CrossRefGoogle Scholar
  10. 10.
    Borofsky S, George AK, Gaur S et al (2017) What are we missing? False-negative cancers at multiparametric MR imaging of the prostate. Radiology.
  11. 11.
    Garcia-Reyes K, Passoni NM, Palmeri ML et al (2015) Detection of prostate cancer with multiparametric MRI (mpMRI): effect of dedicated reader education on accuracy and confidence of index and anterior cancer diagnosis. Abdom Imaging 40:134–142CrossRefGoogle Scholar
  12. 12.
    Gaziev G, Wadhwa K, Barrett T et al (2016) Defining the learning curve for multiparametric magnetic resonance imaging (MRI) of the prostate using MRI-transrectal ultrasonography (TRUS) fusion-guided transperineal prostate biopsies as a validation tool. BJU Int 117:80–86CrossRefGoogle Scholar
  13. 13.
    Wang S, Burtt K, Turkbey B, Choyke P, Summers RM (2014) Computer aided-diagnosis of prostate cancer on multiparametric MRI: a technical review of current research. Biomed Res Int 2014:789561PubMedPubMedCentralGoogle Scholar
  14. 14.
    Liu L, Tian Z, Zhang Z, Fei B (2016) Computer-aided detection of prostate cancer with MRI: technology and applications. Acad Radiol. CrossRefGoogle Scholar
  15. 15.
    Litjens GJ, Elliott R, Shih NN et al (2015) Computer-extracted features can distinguish noncancerous confounding disease from prostatic adenocarcinoma at multiparametric MR imaging. Radiology.
  16. 16.
    Litjens GJ, Barentsz JO, Karssemeijer N, Huisman HJ (2015) Clinical evaluation of a computer-aided diagnosis system for determining cancer aggressiveness in prostate MRI. Eur Radiol 25:3187–3199CrossRefGoogle Scholar
  17. 17.
    Peng Y, Jiang Y, Yang C et al (2013) Quantitative analysis of multiparametric prostate MR images: differentiation between prostate cancer and normal tissue and correlation with Gleason score–a computer-aided diagnosis development study. Radiology 267:787–796CrossRefGoogle Scholar
  18. 18.
    Tiwari P, Kurhanewicz J, Madabhushi A (2013) Multi-kernel graph embedding for detection, Gleason grading of prostate cancer via MRI/MRS. Med Image Anal 17:219–235CrossRefGoogle Scholar
  19. 19.
    Niaf E, Lartizien C, Bratan F et al (2014) Prostate focal peripheral zone lesions: characterization at multiparametric MR imaging–influence of a computer-aided diagnosis system. Radiology 271:761–769CrossRefGoogle Scholar
  20. 20.
    Niaf E, Rouviere O, Mege-Lechevallier F, Bratan F, Lartizien C (2012) Computer-aided diagnosis of prostate cancer in the peripheral zone using multiparametric MRI. Phys Med Biol 57:3833–3851CrossRefGoogle Scholar
  21. 21.
    Hambrock T, Vos PC, Hulsbergen-van de Kaa CA, Barentsz JO, Huisman HJ (2013) Prostate cancer: computer-aided diagnosis with multiparametric 3-T MR imaging–effect on observer performance. Radiology 266:521–530CrossRefGoogle Scholar
  22. 22.
    Berbaum KS, Krupinski EA, Schartz KM et al (2015) Satisfaction of search in chest radiography 2015. Acad Radiol 22:1457–1465CrossRefGoogle Scholar
  23. 23.
    Lay N, Tsehay Y, Greer MD et al (2017) Detection of prostate cancer in multiparametric MRI using random forest with instance weighting. J Med Imaging (Bellingham) 4:024506CrossRefGoogle Scholar
  24. 24.
    Medixant (2015) RadiAnt DICOM Viewer,
  25. 25.
    Radiology ACo (2015) MR Prostate Imaging Reporting and Data System version 2.0.Google Scholar
  26. 26.
    Shah V, Pohida T, Turkbey B et al (2009) A method for correlating in vivo prostate magnetic resonance imaging and histopathology using individualized magnetic resonance-based molds. Rev Sci Instrum 80:104301CrossRefGoogle Scholar
  27. 27.
    Kim YW, Mansfield LT (2014) Fool me twice: delayed diagnoses in radiology with emphasis on perpetuated errors. AJR Am J Roentgenol 202:465–470CrossRefGoogle Scholar
  28. 28.
    Muller BG, Shih JH, Sankineni S et al (2015) Prostate cancer: interobserver agreement and accuracy with the revised Prostate Imaging Reporting and Data System at multiparametric MR imaging. Radiology.
  29. 29.
    Rosenkrantz AB, Ginocchio LA, Cornfeld D et al (2016) Interobserver reproducibility of the PI-RADS Version 2 Lexicon: a multicenter study of six experienced prostate radiologists. Radiology.
  30. 30.
    Rosenkrantz AB, Oto A, Turkbey B, Westphalen AC (2016) Prostate Imaging Reporting and Data System (PI-RADS), Version 2: a critical look. AJR Am J Roentgenol 206:1179–1183CrossRefGoogle Scholar
  31. 31.
    Hansen NL, Koo BC, Gallagher FA et al (2016) Comparison of initial and tertiary centre second opinion reads of multiparametric magnetic resonance imaging of the prostate prior to repeat biopsy. Eur Radiol. CrossRefGoogle Scholar

Copyright information

© This is a U.S. government work and its text is not subject to copyright protection in the United States; however, its text may be subject to foreign copyright protection 2018 2018

Authors and Affiliations

  • Matthew D. Greer
    • 1
  • Nathan Lay
    • 2
  • Joanna H. Shih
    • 3
  • Tristan Barrett
    • 4
  • Leonardo Kayat Bittencourt
    • 5
  • Samuel Borofsky
    • 6
  • Ismail Kabakus
    • 7
  • Yan Mee Law
    • 8
  • Jamie Marko
    • 9
  • Haytham Shebel
    • 10
  • Francesca V. Mertan
    • 1
  • Maria J. Merino
    • 11
  • Bradford J. Wood
    • 12
  • Peter A. Pinto
    • 13
  • Ronald M. Summers
    • 2
  • Peter L. Choyke
    • 1
  • Baris Turkbey
    • 1
    Email author
  1. 1.Molecular Imaging ProgramNCI, NIHBethesdaUSA
  2. 2.Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging SciencesNational Institutes of Health Clinical CenterBethesdaUSA
  3. 3.Biometric Research ProgramNCI, NIHBethesdaUSA
  4. 4.Department of RadiologyUniversity of Cambridge School of MedicineCambridgeUK
  5. 5.Universidade Federal Fluminense and CDPI Clinics/DASARio de JaneiroBrazil
  6. 6.George Washington University HospitalWashingtonUSA
  7. 7.Hacettepe UniversityAnkaraTurkey
  8. 8.Singapore General HospitalSingaporeSingapore
  9. 9.Radiology and Imaging Sciences Department, Clinical CenterNIHBethesdaUSA
  10. 10.Department of Radiology, Nephrology CenterMansoura UniversityMansouraEgypt
  11. 11.Laboratory of PathologyNCI, NIHBethesdaUSA
  12. 12.Center for Interventional Oncology, NCI and Radiology Imaging Sciences, Clinical CenterNIHBethesdaUSA
  13. 13.Urologic Oncology BranchNCI, NIHBethesdaUSA

Personalised recommendations