Abstract
When reading imaging studies, radiologists often compare the acquired images to one or more prior studies of the patient. Machine learning algorithms that assist in identifying abnormalities in medical images usually do not analyze prior images. This work describes a deep-learning classification framework for mammography studies, which incorporates prior image information using four approaches: (1) late fusion of prediction scores; (2) early fusion of input layers; (3) feature fusion combining a convolutional neural network (CNN) and gradient boosting trees; and (4) feature fusion using CNN and long-short term memory (LSTM) architecture. We demonstrate the advantages and limitations of each approach and compare their performance in identifying biopsy-proven malignancies in mammography screening studies. On an evaluation cohort of 439 patients, adding prior studies to the analysis improved the diagnostic performance of the classification framework. The CNN-LSTM architecture achieved the highest area under the ROC curve of 0.88, with sensitivity and specificity of 0.87 and 0.78, respectively. The methods that were trained using information from prior studies achieved better results than the baseline classifier, with up to 45% reduction in false-positive rate at the same sensitivity. The major advantage of the CNN-LSTM approach is in its flexibility and scalability; it allows to use the same network to classify sequences of multiple priors with variable length. The study demonstrates that longitudinal analysis of images can potentially improve the ability of machine learning algorithms to accurately and reliably interpret imaging studies, thus providing value to the radiology community.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Austin, C., Rijken, T., et al.: The role of deep learning in breast screening. Curr. Breast Cancer Rep. 11, 17–22 (2019)
Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. J. Assoc. Phys. India 42(8), 785–794 (2016)
Geras, K.J., Wolfson, S., et al.: High-resolution breast cancer screening with multi-view deep convolutional neural networks, pp. 1–9 (2017)
Hayward, J.H., Ray, K.M., et al.: Improving screening mammography outcomes through comparison with multiple prior mammograms. AJR Am. J. Roentgenol. 207(4), 918–924 (2016)
Hochreiter, S., Urgen Schmidhuber, J.: LTSM. Neural Comput. 9(8), 1735–1780 (1997)
Kooi, T., Karssemeijer, N.: Classifying symmetrical differences and temporal change in mammography using deep neural networks, pp. 1–18 (2017)
Kyono, T., Gilbert, F.J., van der Schaar, M.: MAMMO: a deep learning solution for facilitating radiologist-machine collaboration in breast cancer diagnosis (2018)
Lehman, C.D., Arao, R.F., et al.: National performance benchmarks for modern screening digital mammography: update from the breast cancer surveillance consortium. Radiology 283(1), 49–58 (2017)
McNemar, Q.: Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12(2), 153–157 (1947)
Myers, E.R., Moorman, P., et al.: Benefits and harms of breast cancer screening. JAMA 314(15), 1615 (2015)
Perek, S., Hazan, A., Barkan, E., Akselrod-Ballin, A.: Siamese network for dual-view mammography mass matching. In: Stoyanov, D., et al. (eds.) RAMBO/BIA/TIA -2018. LNCS, vol. 11040, pp. 55–63. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00946-5_6
Ribli, D., Horváth, A., et al.: Detecting and classifying lesions in mammograms with Deep Learning. Sci. Rep. 8(1), 4165 (2018)
Roelofs, A.A.J., Karssemeijer, N., et al.: Importance of comparison of current and prior mammograms in breast cancer screening. Radiology 242(1), 70–77 (2007)
Santeramo, R., Withey, S., Montana, G.: Longitudinal detection of radiological abnormalities with time-modulated LSTM. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 326–333. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_37
Sickles, E.A., d’Orsi, C.J., Bassett, L.W., et al.: ACR BI-RADS® mammography. In: ACR BI-RADS® Atlas, Breast Imaging Reporting and Data System. American College of Radiology, Reston (2013)
Szegedy, C., Ioffe, S., et al.: Inception-v4, Inception-ResNet and the impact of residual connections on learning (2016)
Wu, N., Phang, J., et al.: Deep neural networks improve radiologists’ performance in breast cancer screening (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Perek, S., Ness, L., Amit, M., Barkan, E., Amit, G. (2019). Learning from Longitudinal Mammography Studies. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11769. Springer, Cham. https://doi.org/10.1007/978-3-030-32226-7_79
Download citation
DOI: https://doi.org/10.1007/978-3-030-32226-7_79
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32225-0
Online ISBN: 978-3-030-32226-7
eBook Packages: Computer ScienceComputer Science (R0)