Skip to main content

Advertisement

SpringerLink
Log in

The role of an artificial intelligence software in clinical senology: a mammography multi-reader study

  • Breast Radiology
  • Published:
La radiologia medica Aims and scope Submit manuscript

Abstract

Purpose

To evaluate the diagnostic role of a dedicated AI software in detecting anomalous breast findings on mammography and tomosynthesis images in the clinical setting, stand-alone and as aid of four readers.

Methods

A total of 210 patients with complete clinical and radiologic records were retrospectively analyzed. Pathology was used as the reference standard for patients undergoing surgery or biopsy, and a 1-year follow-up was used to confirm no change in the remaining patients.

The image evaluation was performed by four readers with different levels of experience (a junior and three senior breast radiologists) using a 5-point Likert scale moving from 1 (definitively no cancer) to 5 (definitively cancer).

The positivity of mammograms was assessed on the presence of any breast lesion (masses, architectural distortions, asymmetries, calcifications), including malignant and benign ones. A multi-reader multi-case analysis was performed. A p value < 0.05 was considered statistically significant.

Results

The stand-alone AI system achieved an accuracy of 71% (69% sensitivity and 73% specificity), which is overall lower than the value achieved by readers without AI. However, with the aid of AI, a significant increase of accuracy (p value  = 0.004) and specificity (p value  = 0.04) was achieved for the less experienced radiologist and a senior one.

Conclusion

The use of AI software as a second reader for breast lesions assessment could play a crucial role in the clinical setting, by increasing sensitivity and specificity, especially for less experienced radiologists.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Dembrower K, Wåhlin E, Liu Y, Salim M, Smith K, Lindholm P, Eklund M (2020) Fredrik Strand “Effect of artificial intelligence-based triaging of breast cancer screening mammograms on cancer detection and radiologist workload: a retrospective simulation study. Lancet Digital Health 2:e468–e474

    Article  PubMed  Google Scholar 

  2. Geras KJ, Mann RM, Moy L (2019) Artificial intelligence for mammography and digital breast tomosynthesis: current concepts and future perspectives. Radiology 293:246–259

    Article  PubMed  Google Scholar 

  3. Litjens G, Kooi T, Bejnordi BE et al (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88

    Article  PubMed  Google Scholar 

  4. Le EPV, Wang Y, Huang Y, Hickman S, Gilbert FJ (2019) Artificial intelligence in breast imaging. Clin Radiol 74:357–366

    Article  CAS  PubMed  Google Scholar 

  5. Chan H-P, Samala RK, Hadjiiski LM (2020) CAD and AI for breast cancer-recent development and challenges. Br J Radiol 93:20190580

    Article  PubMed  Google Scholar 

  6. Bazzocchi M, Mazzarella F, Del Frate C, Girometti R, Zuiani C (2007) CAD Systems for mammography: a real opportunity? A review of the literature. Radiol med 112:329–353

    Article  CAS  PubMed  Google Scholar 

  7. Gur D, Sumkin JH (2006) CAD in screening mammography, AJR Women’s Imaging Commentary. AJR 187:1474

  8. Azavedo E, Zackrisson S, Mejarè I, Arnlind MH (2012) Is single reading with computer-aided detection (CAD) as good as double reading in mammography screening? A systematic review. BMC Med Imag 12:22

    Article  Google Scholar 

  9. Dembrower K, Wåhlin E, Liu Y et al (2020) Effect of artificial intelligence based triaging of breast cancer screening mammograms on cancer detection and radiologist workload: a retrospective simulation study. Lancet Digit Health 2:e468–e474

    Article  PubMed  Google Scholar 

  10. Kyono T, Gilbert FJ, van der Schaar M (2020) Improving workflow efficiency for mammography using machine learning. J Am Coll Radiol 17:56–63

    Article  PubMed  Google Scholar 

  11. Raya-Povedano JL, Romero-Martín S, Elías-Cabot E, Gubern-Mérida A, Rodríguez-Ruiz A, Álvarez-Benito M (2021) AI-based strategies to reduce workload in breast cancer screening with mammography and tomosynthesis: a retrospective evaluation. Radiology 1:203555

    Google Scholar 

  12. Yala A, Schuster T, Miles R, Barzilay R, Lehman C (2019) A Deep learning model to triage screening mammograms: a simulation study. Radiology 293:38–46

    Article  PubMed  Google Scholar 

  13. Rodríguez-Ruiz Al et al (2019) Detection of breast cancer with mammography: effect of an artificial intelligence support system. Radiology 290(2):305–314

    Article  PubMed  Google Scholar 

  14. Rodriguez-Ruiz A, Lang K, Gubern-Merida A, Broeders M, Gennaro G, Clauser P, Helbich TH, Chevalier M, Tan T, Mertelmeier T, Wallis MG, Andersson I, Zackrisson S, Mann RM, Sechopoulos I (2019) Stand-alone artificial intelligence for breast cancer detection in mammography: comparison with 101 radiologists. JNCI J Natl Cancer Inst 111(9):djy222

    Article  Google Scholar 

  15. Lång K, Dustler M, Dahlblom V, Åkesson A, Andersson I, Zackrisson S (2021) Identifying normal mammograms in a large screening population using artificial intelligence. Europ Radiol 31:1687–1692

    Article  Google Scholar 

  16. Leibig C, Brehmer M, Bunk S, Byng D, Pinker K, Umutlu L (2022) Combining the strengths of radiologists and AI for breast cancer screening: a retrospective analysis. Lancet Digit Health 4:e507–e519

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Schaffter T, Buist DS, Lee CI et al (2020) Evaluation of combined artificial intelligence and radiologist assessment to interpret screening mammograms. JAMA Netw Open 3:e200265

    Article  PubMed  PubMed Central  Google Scholar 

  18. Wu N, Phang J, Park J et al (2019) Deep neural networks improve radiologists’ performance in breast cancer screening. IEEE Transact Med Imag 39:1184–1194

    Article  Google Scholar 

  19. Balta C, Rodriguez-Ruiz A, Mieskes C, Karssemeijer N, Heywang-Köbrunner S (2020) Going from double to single reading for screening exams labeled as likely normal by AI: What is the impact?: SPIE 11513 15th International Workshop on Breast Imaging (IWBI2020); May 22, (115130D)

  20. McKinney SM, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H, Back T, Chesus M, Corrado GS, Darzi A, Etemadi M, Garcia-Vicente F et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577(2):89

    Article  CAS  PubMed  ADS  Google Scholar 

  21. Shoshan Y, Bakalo R, Gilboa-Solomon F, Ratner V, Barkan E, Ozery-Flato M, Amit M, Khapun D, Ambinder EB, Oluyemi ET, Panigrahi B, DiCarlo PA, Rosen-Zvi M, Mullen LA (2022) Artificial intelligence for reducing workload in breast cancer screening with digital breast tomosynthesis. Radiology 303:69–77

    Article  PubMed  Google Scholar 

  22. Taylor-Philips S, Freeman K (2022) Artificial intelligence to complement rather than replace radiologists in breast screening. The Lancet Digit Health 4(7):E478–E479

    Article  Google Scholar 

  23. Vicini S et al (2022) A narrative review on current imaging applications of artificial intelligence and radiomics in oncology: focus on the three most common cancers. Radiol Med (Torino) 127(8):819–836

    Article  PubMed  Google Scholar 

  24. Houssami N, Kirkpatrick-Jones G, Noguchi N, Lee CI (2019) Artificial Intelligence (AI) for the early detection of breast cancer: a scoping review to assess AI’s potential in breast screening practice”e. Expert Rev Med Dev 16:351–362

    Article  CAS  Google Scholar 

  25. Lehman CD, Arao RF, Sprague BL et al (2017) National performance benchmarks for modern screening digital mammography: update from the Breast Cancer Surveillance Consortium. Radiology 283:49–58

    Article  PubMed  Google Scholar 

  26. American College of Radiology (2013) Breast imaging reporting and data system, 5th ed. Reston: American College of Radiology

  27. Eng J. ROC analysis: web-based calculator for ROC curves. Baltimore: Johns Hopkins University [updated 2022 February 17]. Available from: http://www.jrocfit.org

  28. McKinney SM, Sieniek M, Godbole V et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94

    Article  CAS  PubMed  ADS  Google Scholar 

  29. Kim H-E, Kim HH, Han B-K, Kim KH, Han K, Nam H, Lee EH, Kim E-K (2020) Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective multi-reader study. Lancet Digit Health 2:138–148

    Article  Google Scholar 

  30. Guermazi A, Tannoury C, Kompel AJ, Murakami AM et al (2021) Improving radiographic fracture recognition performance and efficiency using artificial intelligence. Radiology 000:1–10

    Google Scholar 

Download references

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript. Found information: We didn’t receive any funds for this paper. Manuscript Type: original research. Retrospective diagnostic accuracy study (Multi-reader multi-case analysis).

Author information

Authors and Affiliations

  1. Department of Radiology, Verona University Hospital, Verona, Italy

    Enrica Bassi

  2. Department of Radiology, IRCCS Sacro Cuore Hospital, Via Don A. Sempreboni 10, 37024, Negrar (VR), Italy

    Anna Russo, Eugenio Oliboni, Federico Zamboni, Cecilia De Santis & Giovanni Foti

  3. Department of Radiology, Verona University Hospital, Verona, Italy

    Giancarlo Mansueto

  4. Department of Radiology, Azienda Ospedaliera Universitaria Integrata, Verona, Italy

    Stefania Montemezzi

Authors
  1. Enrica Bassi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna Russo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eugenio Oliboni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Federico Zamboni
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cecilia De Santis
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Giancarlo Mansueto
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Stefania Montemezzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Giovanni Foti
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by EB, AR, CDS, FZ and EO. The first draft of the manuscript was written by EB, GF, and SM and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript. Statistical analysis was performed by GF.

Corresponding author

Correspondence to Giovanni Foti.

Ethics declarations

Competing interests

The authors have no relevant financial or non-financial interests to disclose.

Consent to participate

Written informed consent was waived because of retrospective study design.

Consent to publish

The authors affirm that human research participant provided informed consent for publication of the images in Fig. 4.

Ethical approval

“This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of Provincia di Verona e Rovigo (Date January 2023/No Nr of protocol 2022-ZX).” All procedures performed in studies involving human participants were in accordance with the ethical standard of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standard.

Informed consent

Informed consent was obtained by all patients enrolled for this prospective study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

IRB approved study (Nr of protocol 2022-ZX) in January 2023.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bassi, E., Russo, A., Oliboni, E. et al. The role of an artificial intelligence software in clinical senology: a mammography multi-reader study. Radiol med 129, 202–210 (2024). https://doi.org/10.1007/s11547-023-01751-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11547-023-01751-1

Keywords

Search

Navigation