Boundary Loss for Remote Sensing Imagery Semantic Segmentation

  • Alexey Bokhovkin
  • Evgeny BurnaevEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11555)


In response to the growing importance of geospatial data, its analysis including semantic segmentation becomes an increasingly popular task in computer vision today. Convolutional neural networks are powerful visual models that yield hierarchies of features and practitioners widely use them to process remote sensing data. When performing remote sensing image segmentation, multiple instances of one class with precisely defined boundaries are often the case, and it is crucial to extract those boundaries accurately. The accuracy of segments boundaries delineation influences the quality of the whole segmented areas explicitly. However, widely-used segmentation loss functions such as BCE, IoU loss or Dice loss do not penalize misalignment of boundaries sufficiently. In this paper, we propose a novel loss function, namely a differentiable surrogate of a metric accounting accuracy of boundary detection. We can use the loss function with any neural network for binary segmentation. We performed validation of our loss function with various modifications of UNet on a synthetic dataset, as well as using real-world data (ISPRS Potsdam, INRIA AIL). Trained with the proposed loss function, models outperform baseline methods in terms of IoU score.


Semantic segmentation Deep learning Aerial imagery CNN Loss function Building detection Computer vision 


  1. 1.
    Alam, F.I., Zhou, J., Liew, A.W., Jia, X., Chanussot, J., Gao, Y.: Conditional random field and deep feature learning for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 57(3), 1612–1628 (2019). Scholar
  2. 2.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 2481–2495 (2016)Google Scholar
  3. 3.
    Berman, M., Triki, A.R., Blaschko, M.B.: The Lovasz-Softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4413–4421 (2018)Google Scholar
  4. 4.
    Burnaev, E., Cichocki, A., Osin, V.: Fast multispectral deep fusion networks. Bull. Pol. Acad. Sci.: Techn. Sci. 66(4), 875–880 (2018)Google Scholar
  5. 5.
    Burnaev, E., Erofeev, P., Papanov, A.: Influence of resampling on accuracy of imbalanced classification. In: Eighth International Conference on Machine Vision. Proceedings of SPIE, 8 December 2015, vol. 9875, p. 987525 (2015).
  6. 6.
    Chen, L., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018). Scholar
  7. 7.
    Chollet, F., et al.: Keras (2015).
  8. 8.
    Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  9. 9.
    Csurka, G., Larlus, D., Perronnin, F.: What is a good evaluation measure for semantic segmentation? IEEE PAMI 26, 1–11 (2004)Google Scholar
  10. 10.
    Deng, J., Dong, W., Socher, R., Li, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)Google Scholar
  11. 11.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL visual object classes challenge 2012 (VOC 2012) results.
  12. 12.
    Fernandez-Moral, E., Martins, R., Wolf, D., Rives, P.: A new metric for evaluating semantic segmentation: leveraging global and contour accuracy. In: Workshop on Planning, Perception and Navigation for Intelligent Vehicles, PPNIV 2017, Vancouver, Canada, September 2017.
  13. 13.
    Ignatiev, V., Trekin, A., Lobachev, V., Potapov, G., Burnaev, E.: Targeted change detection in remote sensing images. In: Eleventh International Conference on Machine Vision (ICMV 2018). Proceedings of SPIE, vol. 11041, p. 110412H (2019).
  14. 14.
    Ishimtsev, V., Bernstein, A., Burnaev, E., Nazarov, I.: Conformal k-NN anomaly detector for univariate data streams. In: Proceedings of 6th Workshop COPA. PRML, vol. 60, pp. 213–227. PMLR (2017)Google Scholar
  15. 15.
    Krizhevsky, A., Sutskever, I., E. Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  16. 16.
    Maggiori, E., Tarabalka, Y., Charpiat, G., Alliez, P.: Can semantic labeling methods generalize to any city? The Inria aerial image labeling benchmark. In: 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 3226–3229 (2017).
  17. 17.
    Mosinska, A., Marquez-Neila, P., Kozinski, M., Fua, P.: Beyond the pixel-wise loss for topology-aware delineation, pp. 3136–3145, June 2018.
  18. 18.
    Nagendar, G., Singh, D., Balasubramanian, V.N., Jawahar, C.V.: Neuro-IoU: learning a surrogate loss for semantic segmentation. In: British Machine Vision Conference 2018, BMVC 2018, Northumbria University, Newcastle, UK, 3–6 September 2018, pp. 278–289 (2018).
  19. 19.
    Notchenko, A., Kapushev, Y., Burnaev, E.: Large-scale shape retrieval with sparse 3D convolutional neural networks. In: van der Aalst, W.M.P., et al. (eds.) AIST 2017. LNCS, vol. 10716, pp. 245–254. Springer, Cham (2018). Scholar
  20. 20.
    Novikov, G., Trekin, A., Potapov, G., Ignatiev, V., Burnaev, E.: Satellite imagery analysis for operational damage assessment in emergency situations. In: Abramowicz, W., Paschke, A. (eds.) BIS 2018. LNBIP, vol. 320, pp. 347–358. Springer, Cham (2018). Scholar
  21. 21.
    Nowozin, S.: Optimal decisions from probabilistic models: the intersection-over-union case. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 548–555 (2014)Google Scholar
  22. 22.
    Pohlen, T., Hermans, A., Mathias, M., Leibe, B.: Full-resolution residual networks for semantic segmentation in street scenes. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3309–3318 (2017)Google Scholar
  23. 23.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  24. 24.
    Rottensteiner, F., et al.: The ISPRS benchmark on urban object classification and 3D building reconstruction. In: Shortis, M., Paparoditis, N., Mallett, C. (eds.) ISPRS 2012 Proceedings of the XXII ISPRS Congress: Imaging a Sustainable Future, 25 August–01 September 2012, Melbourne, Australia, vol. I-7, pp. 293–298, August 2012. Peer Reviewed Annals, International Society for Photogrammetry and Remote Sensing (ISPRS)Google Scholar
  25. 25.
    Safin, A., Burnaev, E.: Conformal kernel expected similarity for anomaly detection in time-series data. Adv. Syst. Sci. Appl. 17(3), 22–33 (2017)Google Scholar
  26. 26.
    Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2017). Scholar
  27. 27.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556, September 2014
  28. 28.
    Smoliakov, D., Korotin, A., Erifeev, P., Papanov, A., Burnaev, E.: Meta-learning for resampling recommendation systems. In: Eleventh International Conference on Machine Vision (ICMV 2018); 110411S (2019). Proceedings of SPIE, vol. 11041 (2019).
  29. 29.
    Sulimowicz, L., Ahmad, I., Aved, A.J.: Superpixel-enhanced pairwise conditional random field for semantic segmentation. 2018 25th IEEE International Conference on Image Processing (ICIP) pp. 271–275 (2018)Google Scholar
  30. 30.
    Szegedy, C., Ioffe, S., Vanhoucke, V.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI. pp. 4278–4284 (2016)Google Scholar
  31. 31.
    Volkhonskiy, D., Burnaev, E., Nouretdinov, I., Gammerman, A., Vovk, V.: Inductive conformal martingales for change-point detection. In: Proceedings of 6th Workshop COPA. PRML, vol. 60, pp. 132–153. PMLR (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.AeronetSkoltechMoscowRussia
  2. 2.ADASESkoltechMoscowRussia

Personalised recommendations