Temporal-Spatial Feature Learning of Dynamic Contrast Enhanced-MR Images via 3D Convolutional Neural Networks

  • Xibin Jia
  • Yujie Xiao
  • Dawei YangEmail author
  • Zhenghan Yang
  • Xiaopei Wang
  • Yunfeng Liu
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 875)


Dynamic contrast-enhanced magnetic resonance imaging provide not only the information on the morphological features of the lesions, but also the changes of the lesion’s blood perfusion. In this paper, we propose a tensor-based temporal data representation (TTD) model and a multi-channel fusion 3D convolutional neural network (MCF-3D CNN) to extract the temporal and spatial features of dynamic contrast enhanced-MR images (DCE-MR images). To evaluate the performance of the proposed methods, we established a DCE-MR image dataset for non-invasively assessing the differentiation of Hepatocellular carcinoma (HCC). The TTD model achieves the accuracy of 73.96% for non-invasive assessment of HCC differentiation via MCF-3D CNN. Meanwhile, the 3D CNN with TTD achieves accuracy, sensitivity and specificity of 95.17%, 96.33%, and 94.00%, respectively, in discriminating the HCC and cirrhosis. Compared with the normal data representation method, the proposed TTD method is more conducive for 3D CNN to extract temporal-spatial features of DCE-MR images.


3D convolutional neural networks Tensor-based temporal data representation model Dynamic contrast enhanced-MR images Temporal-spatial features Hepatocellular carcinoma Non-invasive assessment 



This work is supported in part by the grants from Beijing Natural Science Foundation (No.7184199), Capital’s Funds for Health Improvement and Research (No. 2018-2-2023), Research Foundation of Beijing Friendship Hospital, Capital Medical University (No. yyqdkt2017-25).


  1. 1.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  2. 2.
    Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). Scholar
  3. 3.
    He, K., Gkioxari, G., Dollár, P., Girshick, R.B.: Mask R-CNN. CoRR, abs/1703.06870 (2017)Google Scholar
  4. 4.
    Ji, S., Xu, W., Yang, M., Yu, K.: 3D convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35, 221–231 (2013)CrossRefGoogle Scholar
  5. 5.
    Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60 (2017)CrossRefGoogle Scholar
  6. 6.
    Dou, Q., et al.: Automatic detection of cerebral microbleeds from mr images via 3D convolutional neural networks. IEEE Trans. Med. Imaging 35, 1182–1195 (2016)CrossRefGoogle Scholar
  7. 7.
    Lemaître, G., Martí, R., Freixenet, J., Vilanova, J.C., Walker, P.M., Meriaudeau, F.: Computer-aided detection and diagnosis for prostate cancer based on mono and multi-parametric MRI: a review. Comput. Biol. Med. 60, 8 (2015)CrossRefGoogle Scholar
  8. 8.
    Kamnitsas, K., et al.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61 (2017)CrossRefGoogle Scholar
  9. 9.
    Yang, J.D., Roberts, L.R.: Hepatocellular carcinoma: a global view. Nat. Rev. Gastroenterol. Hepatol. 7, 448 (2010)CrossRefGoogle Scholar
  10. 10.
    Sherman, M.: Hepatocellular carcinoma: screening and staging. Clin. Liver Dis. 15, 323–334 (2011)CrossRefGoogle Scholar
  11. 11.
    Regimbeau, J.M., et al.: Risk factors for early death due to recurrence after liver resection for hepatocellular carcinoma: Results of a multicenter study †. J. Surg. Oncol. 85, 36–41 (2004)CrossRefGoogle Scholar
  12. 12.
    Jiang, T., Zou, Y., Jie-Hua, X.U., Peng, L.R., Shan, H., Radiology, D.O.: Magnetic resonance imaging in predicting histopathological differentiation of hepatocellular carcinoma. J. Sun Yat-Sen Univ. Sci. 37, 903–911 (2016)Google Scholar
  13. 13.
    Fabijańska, A.: A novel approach for quantification of time-intensity curves in a DCE-MRI image series with an application to prostate cancer. Comput. Biol. Med. 73, 119–130 (2016)CrossRefGoogle Scholar
  14. 14.
    Li, J., Fan, M., Zhang, J., Li, L.: Discriminating between benign and malignant breast tumors using 3D convolutional neural network in dynamic contrast enhanced-MR images. In: SPIE Medical Imaging, p. 1013808 (2017)Google Scholar
  15. 15.
    Zhou, W., et al.: Malignancy characterization of hepatocellular carcinomas based on texture analysis of contrast-enhanced MR images. J. Magn. Reson. Imaging JMRI 45, 1476–1484 (2017)CrossRefGoogle Scholar
  16. 16.
    Lecun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436 (2015)CrossRefGoogle Scholar
  17. 17.
    Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: International Conference on Artificial Intelligence and Statistics, pp. 315–323 (2011)Google Scholar
  18. 18.
    Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: International Conference for Learning Representations (2015)Google Scholar
  19. 19.
    Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Chen, H., Yu, L., Dou, Q., Shi, L., Mok, V.C.T., Heng, P.A.: Automatic detection of cerebral microbleeds via deep learning based 3D feature representation. In: IEEE International Symposium on Biomedical Imaging, pp. 764–767 (2015)Google Scholar
  21. 21.
    Buda, M., Maki, A., Mazurowski, M.A.: A systematic study of the class imbalance problem in convolutional neural networks. CoRR, abs/1710.05381 (2017)Google Scholar
  22. 22.
    Shen, L., Lin, Z., Huang, Q.: Relay backpropagation for effective learning of deep convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 467–482. Springer, Cham (2016). Scholar
  23. 23.
    Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous distributed systems. arXiv: Distributed, Parallel, and Cluster Computing (2016)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • Xibin Jia
    • 1
  • Yujie Xiao
    • 1
  • Dawei Yang
    • 2
    • 3
    Email author
  • Zhenghan Yang
    • 2
  • Xiaopei Wang
    • 2
  • Yunfeng Liu
    • 1
  1. 1.Faculty of Information TechnologyBeijing University of TechnologyBeijingChina
  2. 2.Department of Radiology, Beijing Friendship HospitalCapital Medical UniversityBeijingChina
  3. 3.Beijing Key Laboratory of Translational Medicine on Liver CirrhosisBeijingChina

Personalised recommendations