Volumetric Attention for 3D Medical Image Segmentation and Detection

  • Xudong WangEmail author
  • Shizhong Han
  • Yunqiang Chen
  • Dashan Gao
  • Nuno Vasconcelos
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11769)


A volumetric attention (VA) module for 3D medical image segmentation and detection is proposed. VA attention is inspired by recent advances in video processing, enables 2.5D networks to leverage context information along the z direction, and allows the use of pretrained 2D detection models when training data is limited, as is often the case for medical applications. Its integration in the Mask R-CNN is shown to enable state-of-the-art performance on the Liver Tumor Segmentation (LiTS) Challenge, outperforming the previous challenge winner by 3.9 points and achieving top performance on the LiTS leader board at the time of paper submission. Detection experiments on the DeepLesion dataset also show that the addition of VA to existing object detectors enables a 69.1 sensitivity at 0.5 false positive per image, outperforming the best published results by 6.6 points.


Volumetric Attention 3D images LiTS DeepLesion 


  1. 1.
    Bilic, P., et al.: The liver tumor segmentation benchmark (LITS). arXiv:1901.04056 (2019)
  2. 2.
    Chen, K., et al.: MMDetection (2018).
  3. 3.
    Chlebus, G., et al.: Neural network-based automatic liver tumor segmentation with random forest-based candidate filtering. arXiv:1706.00842 (2017)
  4. 4.
    Christ, P.F., et al.: Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 415–423. Springer, Cham (2016). Scholar
  5. 5.
    Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). Scholar
  6. 6.
    Dai, J., et al.: R-FCN: object detection via region-based fully convolutional networks. In: NIPS (2016)Google Scholar
  7. 7.
    Dai, J., et al.: Deformable convolutional networks. In: ICCV (2017)Google Scholar
  8. 8.
    Girshick, R., et al.: Fast R-CNN. In: ICCV (2015)Google Scholar
  9. 9.
    Han, X.: Automatic liver lesion segmentation using a deep convolutional neural network method. arXiv:1704.07239 (2017)
  10. 10.
    He, K., et al.: Mask R-CNN. In: ICCV (2017)Google Scholar
  11. 11.
    He, K., et al.: Rethinking imagenet pre-training. arXiv:1811.08883 (2018)
  12. 12.
    Hu, J., et al.: Squeeze-and-excitation networks. In: CVPR (2018)Google Scholar
  13. 13.
    Li, X., et al.: H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans. Med. Imaging 37(12), 2663–2674 (2018)CrossRefGoogle Scholar
  14. 14.
    Lin, T.Y., et al.: Feature pyramid networks for object detection. In: CVPR (2017)Google Scholar
  15. 15.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  16. 16.
    Vorontsov, E., et al.: Liver lesion segmentation informed by joint liver segmentation. In: ISBI (2018)Google Scholar
  17. 17.
    Wang, X., et al.: Non-local neural networks. In: CVPR (2018)Google Scholar
  18. 18.
    Yan, K., Bagheri, M., Summers, R.M.: 3D context enhanced region-based convolutional neural network for end-to-end lesion detection. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 511–519. Springer, Cham (2018). Scholar
  19. 19.
    Yan, K., et al.: DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. J. Med. Imaging 5(3), 036501 (2018)CrossRefGoogle Scholar
  20. 20.
    Yuan, Y.: Hierarchical convolutional-deconvolutional neural networks for automatic liver and tumor segmentation. arXiv:1710.04540 (2017)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Xudong Wang
    • 1
    • 2
    Email author
  • Shizhong Han
    • 1
  • Yunqiang Chen
    • 1
  • Dashan Gao
    • 1
  • Nuno Vasconcelos
    • 2
  1. 1.12 Sigma TechnologiesSan DiegoUSA
  2. 2.Department of Electrical and Computer EngineeringUniversity of CaliforniaSan DiegoUSA

Personalised recommendations