Ship Segmentation and Orientation Estimation Using Keypoints Detection and Voting Mechanism in Remote Sensing Images

  • Mingxian Nie
  • Jinjie Zhang
  • Xuetao ZhangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11555)


Ship detection in remote sensing images is an important and challenging task in civil fields. However, the various types of ships with different scale and ratio and the complex scenarios are the main bottlenecks for ship detection and orientation estimation of the ship. In this paper, we propose a new method based on Mask R-CNN, which can perform ship segmentation and direction estimation on ships at the same time by simultaneously output the binary mask and the bow and sterns keypoints locations. We can achieve keypoints detection of the ship without significantly losing the accuracy of the mask. Finally, we regress the coordinates of the ship’s bow and sterns to four quadrants and use the voting mechanism to determine which quadrant the bow keypoint locates. Then we combine the quadrant of bow keypoint with the minimum bounding box of the mask to determine the final orientation of the ship. Experiments on the datasets have achieved effective performance.


Remote sensing images Ship detection Orientation estimation Mask R-CNN 



This work was supported by the National Science and Technology Major Project of China grant number 2018ZX01008103 and National Key Research and Development Program of China under Grant 2017YFC0803905.


  1. 1.
    Yang, X.: Automatic ship detection in remote sensing images from Google Earth of complex scenes based on multiscale rotation dense feature pyramid networks. Remote Sens. 10(1), 132 (2018)Google Scholar
  2. 2.
    Zhang, R.: S-CNN ship detection from high-resolution remote sensing images. ISPRS - Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. XLI-(B7), 423–430 (2016)Google Scholar
  3. 3.
    Liu, W.: Automated vehicle extraction and speed determination from QuickBird satellite images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 4(1), 75–82 (2011)Google Scholar
  4. 4.
    Tong, X.: Building-damage detection using pre- and post-seismic high-resolution satellite stereo imagery: a case study of the May 2008 Wenchuan earthquake. ISPRS J. Photogramm. Remote Sens. 68, 13–27 (2012)Google Scholar
  5. 5.
    Zhu, C.: A novel hierarchical method of ship detection from spaceborne optical image based on shape and texture features. IEEE Trans. Geosci. Remote Sens. 48(9), 3446–3456 (2010)Google Scholar
  6. 6.
    He, K., Sun, J.: Convolutional neural networks at constrained time cost. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5353–5360. IEEE, Boston (2015)Google Scholar
  7. 7.
    Yu, Y.: Automated ship detection from optical remote sensing images. Key Eng. Mater. 500, 785–791 (2012)Google Scholar
  8. 8.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards realtime object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28, pp. 91–99. Curran Associates Inc., Montreal (2015)Google Scholar
  9. 9.
    Girshick, R.: Fast R-CNN. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1440–1448. IEEE, Santiago (2015)Google Scholar
  10. 10.
    Yang, X.: Position detection and direction prediction for arbitrary-oriented ships via multitask rotation region convolutional neural network. IEEE Access 6, 50839–50849 (2018)Google Scholar
  11. 11.
    Nie, S., Jiang, Z., Zhang, H., Cai, B., Yao, Y.: Inshore ship detection based on mask R-CNN. In: IGARSS 2018, 2018 IEEE International Geoscience and Remote Sensing Symposium, pp. 693–696. IEEE, Valencia (2018)Google Scholar
  12. 12.
    Andriluka, M., Roth, S., Schiele, B.: Pictorial structures revisited: people detection and articulated pose estimation. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1014–1021. IEEE, Miami (2009)Google Scholar
  13. 13.
    Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). Scholar
  14. 14.
    Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., Sun, J.: Cascaded pyramid network for multi-person pose estimation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7103–7112. IEEE, Salt Lake City (2018)Google Scholar
  15. 15.
    He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: 2017 IEEE International Conference on Computer Vision(ICCV), pp. 2980–2988. IEEE, Venice (2017)Google Scholar
  16. 16.
    Lin, T., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944. IEEE, Honolulu (2017)Google Scholar
  17. 17.
    Liu, Z., Yuan, L., Weng, L., Yang, Y.: A high resolution optical satellite image dataset for ship recognition and some new baselines. In: 6th International Conference on Pattern Recognition Applications and Methods, pp. 324–331. SciTePress, Porto (2017)Google Scholar
  18. 18.
    Airbus Ship Detection Challenge. Accessed 1 Jan 2019

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Software EngineeringXi’an Jiaotong UniversityXi’anChina
  2. 2.School of Electronic and Information EngineeringXi’an Jiaotong UniversityXi’anChina

Personalised recommendations