Graph-FCN for Image Semantic Segmentation

  • Yi Lu
  • Yaran ChenEmail author
  • Dongbin Zhao
  • Jianxin Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11554)


Semantic segmentation with deep learning has achieved great progress in classifying the pixels in the image. However, the local location information is usually ignored in the high-level feature extraction by the deep learning, which is important for image semantic segmentation. To avoid this problem, we propose a graph model initialized by a fully convolutional network (FCN) named Graph-FCN for image semantic segmentation. Firstly, the image grid data is extended to graph structure data by a convolutional network, which transforms the semantic segmentation problem into a graph node classification problem. Then we apply graph convolutional network to solve this graph node classification problem. As far as we know, it is the first time that we apply the graph convolutional network in image semantic segmentation. Our method achieves competitive performance in mean intersection over union (mIOU) on the VOC dataset (about 1.34% improvement), compared to the original FCN model.


Graph neural network Graph convolutional network Semantic segmentation 


  1. 1.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 2481–2495 (2017)Google Scholar
  2. 2.
    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected CRFs. In: International Conference on Learning Representations, San Diego (2015)Google Scholar
  3. 3.
    Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 (2017)
  4. 4.
    Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: European Conference on Computer Vision, Munich, pp. 833–851 (2018)Google Scholar
  5. 5.
    Chen, Y., Zhao, D., Lv, L., Li, C.: A visual attention based convolutional neural network for image classification. In: 12th World Congress on Intelligent Control and Automation, Guilin, pp. 764–769 (2016)Google Scholar
  6. 6.
    Chen, Y., Zhao, D.: Multi-task learning with cartesian product-based multi-objective combination for dangerous object detection. In: Cong, F., Leung, A., Wei, Q. (eds.) ISNN 2017. LNCS, vol. 10261, pp. 28–35. Springer, Cham (2017). Scholar
  7. 7.
    Chen, Y., Zhao, D., Lv, L., Zhang, Q.: Multi-task learning for dangerous object detection in autonomous driving. Inf. Sci. 432, 559–571 (2018)Google Scholar
  8. 8.
    Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Neural Information Processing Systems, Barcelona, pp. 3844–3852 (2016)Google Scholar
  9. 9.
    Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: International Conference on Machine Learning, Sydney, pp. 1263–1272 (2017)Google Scholar
  10. 10.
    Gori, M., Monfardini, G., Scarselli, F.: A new model for learning in graph domains. In: IEEE International Joint Conference on Neural Networks, pp. 729–734 (2005)Google Scholar
  11. 11.
    Huang, K.Q., Ren, W.Q., Tan, T.N.: A review on image object classification and detection. Chin. J. Comput. 37, 1225–1240 (2014)Google Scholar
  12. 12.
    Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations, Toulon (2017)Google Scholar
  13. 13.
    Krahenbuhl, P., Koltun, V.: Efficient inference in fully connected CRFs with Gaussian edge potentials. In: Neural Information Processing Systems, Granada, pp. 109–117 (2011)Google Scholar
  14. 14.
    Li, Q., Han, Z., Wu, X.M.: Deeper insights into graph convolutional networks for semi-supervised learning. In: Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, pp. 3538–3545 (2018)Google Scholar
  15. 15.
    Liu, Y., Wang, R., Shan, S., Chen, X.: Structure inference net: object detection using scene-level context and instance-level relationships. In: IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, pp. 6985–6994 (2018)Google Scholar
  16. 16.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, Boston, pp. 3431–3440 (2015)Google Scholar
  17. 17.
    Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20, 61–80 (2009)Google Scholar
  18. 18.
    Wang, X., Girshick, R.B., Gupta, A., He, K.: Non-local neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, pp. 7794–7803 (2018)Google Scholar
  19. 19.
    Zhao, D., Chen, Y., Lv, L.: Deep reinforcement learning with visual attention for vehicle classification. IEEE Trans. Cogn. Dev. Syst. 9, 356–367 (2017)Google Scholar
  20. 20.
    Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., Torr, P.H.S.: Conditional random fields as recurrent neural networks. In: IEEE International Conference on Computer Vision, Santiago, pp. 1529–1537 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yi Lu
    • 1
    • 2
  • Yaran Chen
    • 1
    • 2
    Email author
  • Dongbin Zhao
    • 1
    • 2
  • Jianxin Chen
    • 3
  1. 1.State Key Laboratory of Management and Control for Complex Systems Institute of AutomationChinese Academy of SciencesBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina
  3. 3.Beijing University of Chinese MedicineBeijingChina

Personalised recommendations