Abstract
Purpose
To evaluate the diagnostic role of a dedicated AI software in detecting anomalous breast findings on mammography and tomosynthesis images in the clinical setting, stand-alone and as aid of four readers.
Methods
A total of 210 patients with complete clinical and radiologic records were retrospectively analyzed. Pathology was used as the reference standard for patients undergoing surgery or biopsy, and a 1-year follow-up was used to confirm no change in the remaining patients.
The image evaluation was performed by four readers with different levels of experience (a junior and three senior breast radiologists) using a 5-point Likert scale moving from 1 (definitively no cancer) to 5 (definitively cancer).
The positivity of mammograms was assessed on the presence of any breast lesion (masses, architectural distortions, asymmetries, calcifications), including malignant and benign ones. A multi-reader multi-case analysis was performed. A p value < 0.05 was considered statistically significant.
Results
The stand-alone AI system achieved an accuracy of 71% (69% sensitivity and 73% specificity), which is overall lower than the value achieved by readers without AI. However, with the aid of AI, a significant increase of accuracy (p value = 0.004) and specificity (p value = 0.04) was achieved for the less experienced radiologist and a senior one.
Conclusion
The use of AI software as a second reader for breast lesions assessment could play a crucial role in the clinical setting, by increasing sensitivity and specificity, especially for less experienced radiologists.
Similar content being viewed by others
References
Dembrower K, Wåhlin E, Liu Y, Salim M, Smith K, Lindholm P, Eklund M (2020) Fredrik Strand “Effect of artificial intelligence-based triaging of breast cancer screening mammograms on cancer detection and radiologist workload: a retrospective simulation study. Lancet Digital Health 2:e468–e474
Geras KJ, Mann RM, Moy L (2019) Artificial intelligence for mammography and digital breast tomosynthesis: current concepts and future perspectives. Radiology 293:246–259
Litjens G, Kooi T, Bejnordi BE et al (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88
Le EPV, Wang Y, Huang Y, Hickman S, Gilbert FJ (2019) Artificial intelligence in breast imaging. Clin Radiol 74:357–366
Chan H-P, Samala RK, Hadjiiski LM (2020) CAD and AI for breast cancer-recent development and challenges. Br J Radiol 93:20190580
Bazzocchi M, Mazzarella F, Del Frate C, Girometti R, Zuiani C (2007) CAD Systems for mammography: a real opportunity? A review of the literature. Radiol med 112:329–353
Gur D, Sumkin JH (2006) CAD in screening mammography, AJR Women’s Imaging Commentary. AJR 187:1474
Azavedo E, Zackrisson S, Mejarè I, Arnlind MH (2012) Is single reading with computer-aided detection (CAD) as good as double reading in mammography screening? A systematic review. BMC Med Imag 12:22
Dembrower K, Wåhlin E, Liu Y et al (2020) Effect of artificial intelligence based triaging of breast cancer screening mammograms on cancer detection and radiologist workload: a retrospective simulation study. Lancet Digit Health 2:e468–e474
Kyono T, Gilbert FJ, van der Schaar M (2020) Improving workflow efficiency for mammography using machine learning. J Am Coll Radiol 17:56–63
Raya-Povedano JL, Romero-Martín S, Elías-Cabot E, Gubern-Mérida A, Rodríguez-Ruiz A, Álvarez-Benito M (2021) AI-based strategies to reduce workload in breast cancer screening with mammography and tomosynthesis: a retrospective evaluation. Radiology 1:203555
Yala A, Schuster T, Miles R, Barzilay R, Lehman C (2019) A Deep learning model to triage screening mammograms: a simulation study. Radiology 293:38–46
Rodríguez-Ruiz Al et al (2019) Detection of breast cancer with mammography: effect of an artificial intelligence support system. Radiology 290(2):305–314
Rodriguez-Ruiz A, Lang K, Gubern-Merida A, Broeders M, Gennaro G, Clauser P, Helbich TH, Chevalier M, Tan T, Mertelmeier T, Wallis MG, Andersson I, Zackrisson S, Mann RM, Sechopoulos I (2019) Stand-alone artificial intelligence for breast cancer detection in mammography: comparison with 101 radiologists. JNCI J Natl Cancer Inst 111(9):djy222
Lång K, Dustler M, Dahlblom V, Åkesson A, Andersson I, Zackrisson S (2021) Identifying normal mammograms in a large screening population using artificial intelligence. Europ Radiol 31:1687–1692
Leibig C, Brehmer M, Bunk S, Byng D, Pinker K, Umutlu L (2022) Combining the strengths of radiologists and AI for breast cancer screening: a retrospective analysis. Lancet Digit Health 4:e507–e519
Schaffter T, Buist DS, Lee CI et al (2020) Evaluation of combined artificial intelligence and radiologist assessment to interpret screening mammograms. JAMA Netw Open 3:e200265
Wu N, Phang J, Park J et al (2019) Deep neural networks improve radiologists’ performance in breast cancer screening. IEEE Transact Med Imag 39:1184–1194
Balta C, Rodriguez-Ruiz A, Mieskes C, Karssemeijer N, Heywang-Köbrunner S (2020) Going from double to single reading for screening exams labeled as likely normal by AI: What is the impact?: SPIE 11513 15th International Workshop on Breast Imaging (IWBI2020); May 22, (115130D)
McKinney SM, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H, Back T, Chesus M, Corrado GS, Darzi A, Etemadi M, Garcia-Vicente F et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577(2):89
Shoshan Y, Bakalo R, Gilboa-Solomon F, Ratner V, Barkan E, Ozery-Flato M, Amit M, Khapun D, Ambinder EB, Oluyemi ET, Panigrahi B, DiCarlo PA, Rosen-Zvi M, Mullen LA (2022) Artificial intelligence for reducing workload in breast cancer screening with digital breast tomosynthesis. Radiology 303:69–77
Taylor-Philips S, Freeman K (2022) Artificial intelligence to complement rather than replace radiologists in breast screening. The Lancet Digit Health 4(7):E478–E479
Vicini S et al (2022) A narrative review on current imaging applications of artificial intelligence and radiomics in oncology: focus on the three most common cancers. Radiol Med (Torino) 127(8):819–836
Houssami N, Kirkpatrick-Jones G, Noguchi N, Lee CI (2019) Artificial Intelligence (AI) for the early detection of breast cancer: a scoping review to assess AI’s potential in breast screening practice”e. Expert Rev Med Dev 16:351–362
Lehman CD, Arao RF, Sprague BL et al (2017) National performance benchmarks for modern screening digital mammography: update from the Breast Cancer Surveillance Consortium. Radiology 283:49–58
American College of Radiology (2013) Breast imaging reporting and data system, 5th ed. Reston: American College of Radiology
Eng J. ROC analysis: web-based calculator for ROC curves. Baltimore: Johns Hopkins University [updated 2022 February 17]. Available from: http://www.jrocfit.org
McKinney SM, Sieniek M, Godbole V et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94
Kim H-E, Kim HH, Han B-K, Kim KH, Han K, Nam H, Lee EH, Kim E-K (2020) Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective multi-reader study. Lancet Digit Health 2:138–148
Guermazi A, Tannoury C, Kompel AJ, Murakami AM et al (2021) Improving radiographic fracture recognition performance and efficiency using artificial intelligence. Radiology 000:1–10
Funding
The authors declare that no funds, grants, or other support were received during the preparation of this manuscript. Found information: We didn’t receive any funds for this paper. Manuscript Type: original research. Retrospective diagnostic accuracy study (Multi-reader multi-case analysis).
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by EB, AR, CDS, FZ and EO. The first draft of the manuscript was written by EB, GF, and SM and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript. Statistical analysis was performed by GF.
Corresponding author
Ethics declarations
Competing interests
The authors have no relevant financial or non-financial interests to disclose.
Consent to participate
Written informed consent was waived because of retrospective study design.
Consent to publish
The authors affirm that human research participant provided informed consent for publication of the images in Fig. 4.
Ethical approval
“This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of Provincia di Verona e Rovigo (Date January 2023/No Nr of protocol 2022-ZX).” All procedures performed in studies involving human participants were in accordance with the ethical standard of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standard.
Informed consent
Informed consent was obtained by all patients enrolled for this prospective study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
IRB approved study (Nr of protocol 2022-ZX) in January 2023.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Bassi, E., Russo, A., Oliboni, E. et al. The role of an artificial intelligence software in clinical senology: a mammography multi-reader study. Radiol med 129, 202–210 (2024). https://doi.org/10.1007/s11547-023-01751-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11547-023-01751-1