Abstract
The accurate classification of 3D medical images is a challenging task for current deep learning methods. Deep learning models struggle to extract features when the data size is small and the data dimension is large. To solve this problem, we develop a spatial-frequency non-local convolutional LSTM network for 3D image classification. Compared to traditional networks, the proposed model has the ability to extract features from both the spatial and frequency domains, which allows the frequency-domain features to contribute to the classification. Furthermore, the non-local blocks in our architecture enable it to capture the long-range dependencies directly in the feature space. Finally, to simplify the classification task and improve the performance, we utilize a two-stage framework that localizes lesions in the first step, and classifies them in the second. We evaluate our method on a challenging and important clinical task, i.e, the differentiation of papillary renal cell carcinoma (pRCC) into subtype 1 and subtype 2. To the best of our knowledge, this is the first time that the advantage of synthesizing spatial- and frequency-domain features by deep learning networks for medical image classification has been demonstrated. Experimental results demonstrate that the proposed method achieves competitive and often superior performance compared to state-of-the-art networks and three clinical experts.
Y. Zhao and Y. Liu—Contributed equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gupta, K., et al.: Epidemiologic and socioeconomic burden of metastatic renal cell carcinoma (mRCC): a literature review. Cancer Treat. Rev. 34(3), 193–205 (2008)
Hazlett, H.C., et al.: Early brain development in infants at high risk for autism spectrum disorder. Nature 542(7641), 348 (2017)
He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, G., et al.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Klatte, T., et al.: Cytogenetic and molecular tumor profiling for type 1 and type 2 papillary renal cell carcinoma. Clin. Cancer Res. 15(4), 1162–1169 (2009)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)
Novikov, A.A., et al.: Deep sequential segmentation of organs in volumetric medical scans. IEEE Trans. Med. Imaging 36, 1359–1371 (2018)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Roth, H.R., et al.: Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans. Med. Imaging 35(5), 1170–1181 (2016)
Schlemper, J., et al.: Cardiac MR segmentation from undersampled k-space using deep latent representation learning. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 259–267. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_30
Suk, H.I., et al.: Latent feature representation with stacked auto-encoder for AD/MCI diagnosis. Brain Struct. Func. 220(2), 841–859 (2015)
Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Jimenez-del Toro, O., et al.: Cloud-based evaluation of anatomical structure segmentation and landmark detection algorithms: visceral anatomy benchmarks. IEEE Trans. Med. Imaging 35(11), 2459–2475 (2016)
Varghese, B.A., et al.: Differentiation of predominantly solid enhancing lipid-poor renal cell masses by use of contrast-enhanced CT: evaluating the role of texture in tumor subtyping. Am. J. Roentgenol. 211(6), W288–W296 (2018)
Wang, X., et al.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)
Zhu, B., et al.: Image reconstruction by domain-transform manifold learning. Nature 555(7697), 487 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, Y. et al. (2019). Spatial-Frequency Non-local Convolutional LSTM Network for pRCC Classification. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11769. Springer, Cham. https://doi.org/10.1007/978-3-030-32226-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-32226-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32225-0
Online ISBN: 978-3-030-32226-7
eBook Packages: Computer ScienceComputer Science (R0)
