Advertisement

Domain Adaptation for Semantic Segmentation with Conditional Random Field

  • Yuze Sun
  • Xiaofu WuEmail author
  • Quan Zhou
  • Suofei Zhang
Chapter
Part of the Studies in Computational Intelligence book series (SCI, volume 810)

Abstract

Fully-convolutional neural networks (CNNs) for semantic segmentation dramatically improve performance using end-to-end learning on whole images in a supervised manner. The success of CNNs for semantic segmentation depends heavily on the pixel-level ground truth, which is labor-intensive in general. To partially solve this problem, domain adaptation techniques have been adapted to the two similar tasks for semantic segmentation, one of which is fully-labelled, while the other is unlabelled. Based on the adversarial learning method for domain adaptation in the context of semantic segmentation (AdaptSegNet), this paper proposes to employ the conditional random field (CRF) to refine the output of the segmentation network before domain adaptation. The proposed system fully integrates CRF model with CNNs, making it possible to train the whole system end-to-end with the usual backpropagation algorithm. Extensive experiments demonstrate the effectiveness of our framework under various domain adaptation settings, including synthetic-to-real scenarios.

Keywords

Semantic segmentation Adaptation domain CRF 

Notes

Acknowledgements

This work was partly supported the National Natural Science Foundation of China (Grant No. 61881240048, 61701252, 61876093, BK20181393), and HIRP Open 2018 Project of Huawei.

References

  1. 1.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of CVPR (2012)Google Scholar
  2. 2.
    Serikawaa, S., Huimin, L.: Underwater image dehazing using joint trilateral filter. Comput. Electr. Eng. 40(1), 41–50 (2014)CrossRefGoogle Scholar
  3. 3.
    Lu, H., Li, Y., Mu, S., Wang, D., Kim, H., Serikawa, S.: Motor anomaly detection for unmanned aerial vehicles using reinforcement learning. IEEE Internet Things J. (2017).  https://doi.org/10.1109/JIOT.2017.2737479
  4. 4.
    Lu, H., Li, Y., Chen, M., Kim, H., Serikawa, S.: Brain intelligence: go beyond artificial intelligence. Mob. Netw. Appl. 1–8 (2017)Google Scholar
  5. 5.
    Lu, H., Li, B., Zhu, J., Li, Y., Li, Y., Xu, X., He, L., Li, X., Li, J., Serikawa, S.: Wound intensity correction and segmentation with convolutional neural networks. Concurr. Comput. Pract. Exp. (2017).  https://doi.org/10.1002/cpe.3927
  6. 6.
    Lu, H., Li, Y., Uemura, T., Kim, H., Serikawa, S.: Low illumination underwater light field images reconstruction using deep convolutional neural networks. Future Gen. Comput. Syst. (2018).  https://doi.org/10.1016/j.future.2018.01.001
  7. 7.
    Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. CoRR arXiv:1606.00915
  8. 8.
    Richter, S.R., Vineet, V., Roth, S., Koltun, V.: Playing for data: ground truth from computer games. In: Proceedings of ECCV (2016)Google Scholar
  9. 9.
    Ros, G., Sellart, L., Materzynska, J., Vazquez, D., Lopez, A.: The SYNTHIA dataset: a large collection of synthetic images for semantic segmentation of urban scenes. In: Proceedings of CVPR (2016)Google Scholar
  10. 10.
    Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of CVPR (2016)Google Scholar
  11. 11.
    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Proceedings of NIPS (2014)Google Scholar
  12. 12.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of ICLR (2015)Google Scholar
  13. 13.
    Hong, S., Noh, H., Han, B.: Decoupled deep neural network for semi-supervised semantic segmentation. In: Proceedings of NIPS (2015)Google Scholar
  14. 14.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of CVPR (2016)Google Scholar
  15. 15.
    Hoffman, J., Tzeng, E., Park, T., Zhu, J.-Y., Isola, P., Saenko, K., Efros, A.A., Darrell, T.: CyCADA: cycle-consistent adversarial domain adaptation. CoRR (2017). arXiv:1711.03213
  16. 16.
    Khoreva, A., Benenson, R., Hosang, J., Hein, M., Schiele, B.: Simple does it: weakly supervised instance and semantic segmentation. In: Proceedings of CVPR (2017)Google Scholar
  17. 17.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of ICML (2015)Google Scholar
  18. 18.
    Hoffman, J., Wang, D., Yu, F., Darrell, T.: FCNs in the wild: pixel-level adversarial and constraint-based adaptation. CoRR (2016). arXiv:1612.02649

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.National Engineering Research Center of Communications and NetworkingNanjing University of Posts & TelecommunicationsNanjingPeople’s Republic of China

Personalised recommendations