Image Segmentation with Pyramid Dilated Convolution Based on ResNet and U-Net

  • Qiao Zhang
  • Zhipeng Cui
  • Xiaoguang Niu
  • Shijie Geng
  • Yu QiaoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10635)


Various deep convolutional neural networks (CNNs) have been applied in the task of medical image segmentation. A lot of CNNs have been proved to get better performance than the traditional algorithms. Deep residual network (ResNet) has drastically improved the performance by a trainable deep structure. In this paper, we proposed a new end-to-end network based on ResNet and U-Net. Our CNN effectively combine the features from shallow and deep layers through multi-path information confusion. In order to exploit global context features and enlarge receptive field in deep layer without losing resolution, We designed a new structure called pyramid dilated convolution. Different from traditional networks of CNNs, our network replaces the pooling layer with convolutional layer which can reduce information loss to some extent. We also introduce the LeakyReLU instead of ReLU along the downsampling path to increase the expressiveness of our model. Experiment shows that our proposed method can successfully extract features for medical image segmentation.


Deep learning Semantic image segmentation Convolutional neural network Medical image Ultrasound Nerve Segmentation 



This research is partly supported by NSFC (No: 61375048).


  1. 1.
    Brebisson, A.D., Mountana, G.: Deep neural networks for anatomical brain segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2015)Google Scholar
  2. 2.
    Zhang, W., Li, R., Deng, H., Wang, L.: Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage 108, 214–224 (2015)CrossRefGoogle Scholar
  3. 3.
    Li, Q., Cai, T., Wang, X., Zhou, Y., Feng, D.: Medical image classification with convolutional neural network. In: the 13th International Conference on Control Automation Robotics & Vision (ICARCV). IEEE (2014)Google Scholar
  4. 4.
    Ciresan, D.C., Meier, U., Masci, J., Gambardella, L.M., Schmidhuber, J.: Flexible, high performance convolutional neural networks for image classification. In: Twenty-Second International Joint Conference on Artificial Intelligence (2011)Google Scholar
  5. 5.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  6. 6.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. ArXiv preprint arXiv:1409.1556 (2014)
  7. 7.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the Institute of Electrical and Electronics Engineers Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  8. 8.
    Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions, arXiv preprint arXiv:1511.07122 (2015)
  9. 9.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_28 CrossRefGoogle Scholar
  10. 10.
    Xu, B., Wang, N., Chen, T., Li, M.: Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.00853 (2015)
  11. 11.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010) (2010)Google Scholar
  12. 12.
    He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). doi: 10.1007/978-3-319-46493-0_38 CrossRefGoogle Scholar
  13. 13.
    LeCun, Y., Boser, B.E., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W.E., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)CrossRefGoogle Scholar
  14. 14.
    Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Object detectors emerge in deep scene cnns. arXiv preprint arXiv:1412.6856 (2014)
  15. 15.
    Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Qiao Zhang
    • 1
    • 2
  • Zhipeng Cui
    • 1
  • Xiaoguang Niu
    • 1
  • Shijie Geng
    • 1
  • Yu Qiao
    • 1
    Email author
  1. 1.Intelligence Learning Laboratory, Institute of Image Processing and Pattern Recognition, Department of AutomationShanghai Jiao Tong UniversityShanghaiChina
  2. 2.The Fu Foundation School of Engineering and Applied ScienceColumbia UniversityNew YorkUSA

Personalised recommendations