High-Resolution Remote Sensing Image Scene Classification by Merging Multilevel Features of Convolutional Neural Networks

Abstract

The rapid development of remote sensing technologies has yielded a large amount of high-resolution remote sensing (HRRS) data. However, effectively extracting information from these large datasets is a significant challenge. In this study, a state-of-the-art convolutional neural network was extended for HRRS image classification by establishing an integrated structure including three branches of a pruned DenseNet, a decoder, and an encoder (PDDE-Net). PDDE-Net initially combines deep-level features extracted from the pruned DenseNet with middle-level features extracted from the designed encoder. Subsequently, the designed decoder branch extracts the merged features to obtain more feature expressions. In addition, the earlystop strategy is used to prevent overfitting during the training process. Experiments performed on an aerial image dataset demonstrated that the proposed network can achieve favorable overall accuracy and Kappa values of 90% and 0.897, respectively. Further, under the condition of imbalanced data, two merging methods—concatenation and summation—were applied to test the sensitivity of the integrated PDDE-Net to the feature size. The results on the remote sensing image classification benchmark datasets revealed little difference between the two merging methods, both of which exhibited accuracies greater than 95%. Moreover, the integrated PDDE-Net was tested on hyperspectral remote sensing data, and it achieved a high classification accuracy comparable to those of other advanced methods, which also demonstrated the generalizability of the proposed model.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

References

  1. Akbarizadeh, G. (2012). A new statistical-based kurtosis wavelet energy feature for texture recognition of SAR images. IEEE Transactions on Geoscience and Remote Sensing, 50(11), 4358–4368.

    Article  Google Scholar 

  2. Bera, S., & Shrivastava, V. K. (2020). Analysis of various optimizers on deep convolutional neural network model in the application of hyperspectral remote sensing image classification. International Journal of Remote Sensing, 41(7), 2664–2683.

    Article  Google Scholar 

  3. Bian, X. Y., Chen, C., Tian, L., & Du, Q. (2017). Fusing local and global features for high-resolution scene classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(6), 2889–2901.

    Article  Google Scholar 

  4. Boualleg, Y., Farah, M., & Farah, I. R. (2019). Remote sensing scene classification using convolutional features and deep forest classifier. IEEE Geoscience and Remote Sensing Letters, 16(12), 1944–1948.

    Article  Google Scholar 

  5. Chaib, S., Liu, H., Gu, Y. F., & Yao, H. (2017). Deep feature fusion for VHR remote sensing scene classification. IEEE Transactions on Geoscience and Remote Sensing, 55(8), 4775–4784.

    Article  Google Scholar 

  6. Chen, Y., Lin, Z., Zhao, X., Wang, G., & Gu, Y. (2014). Deep learning-basedclassification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6), 2094–2107.

    Article  Google Scholar 

  7. Chen, Z. L., Wang, Y. Y., Han, W., Feng, R., & Chen, J. (2019). An Improved Pretraining Strategy-Based Scene Classification with Deep Learning. IEEE Geoscience and Remote Sensing Letters, 17(5), 844–848.

  8. Cheriyadat, A. M. (2013). Unsupervised feature learning for aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 52(1), 439–451.

    Article  Google Scholar 

  9. Csurka, G. (2004). Visual categorization with Bags of Keypoints. In European Conference on Computer Vision (pp. 1–22).

  10. Fukushima, K. (1980). Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36(4), 193–202.

    Article  Google Scholar 

  11. Ghanbari, M., Clausi, D. A., Xu, L., & Jiang, M. (2019). Contextual classification of sea-ice types using compact polarimetric SAR data. IEEE Transactions on Geoscience and Remote Sensing, 57(10), 7476–7491.

    Article  Google Scholar 

  12. He, C., Shi, Z., Qu, T., Wang, D., & Liao, M. (2019). Lifting scheme-based deep neural network for remote sensing scene classification. Remote Sensing, 11(22), 2648.

    Article  Google Scholar 

  13. Hu, F., Xia, G. S., Wang, Z. F., Huang, X., Zhang, L. P., & Sun, H. (2015). Unsupervised feature learning via spectral clustering of multidimensional patches for remotely sensed scene classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(5), 2015–2030.

    Article  Google Scholar 

  14. Huang, G., Liu, Z., Laurens, V. D. M., Weinberger, K. Q. (2017). Densely Connected Convolutional Networks. In Computer Vision and Pattern Recognition (pp. 2261–2269).

  15. Hurskainen, P., Adhikari, H., Siljander, M., Pellikka, P. K. E., & Hemp, A. (2019). Auxiliary datasets improve accuracy of object-based land use/land cover classification in heterogeneous savanna landscapes. Remote Sensing of Environment, 233, 111354.

    Article  Google Scholar 

  16. Jaiswal, R. K., Saxena, R., & Mukherjee, S. (1999). Application of remote sensing technology for land use/land cover change analysis. Journal of the Indian Society of Remote Sensing, 27(2), 123–128.

    Article  Google Scholar 

  17. Lazebnik, S., Schmid, C., & Ponce, J. (2006). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 2169–2178).

  18. Li, E., Xia, J. S., Du, P. J., Lin, C., & Samat, A. (2017a). Integrating multilayer features of convolutional neural networks for remote sensing scene classification. IEEE Transactions on Geoscience and Remote Sensing, 55(10), 5653–5665.

    Article  Google Scholar 

  19. Li, H., Tao, C., Wu, Z., Chen, J., Gong, J., Deng, M. (2017b). RSI-CB: A large scale remote sensing image classification benchmark via crowdsource data. arXiv: Computer Vision and Pattern Recognition.

  20. Liu, B. D., Meng, J., Xie, W., Shao, S., Li, Y., & Wang, Y. (2019). Weighted spatial pyramid matching collaborative representation for remote-sensing-image scene classification. Remote Sensing, 11(5), 518.

    Article  Google Scholar 

  21. Mou, L., Ghamisi, P., & Zhu, X. X. (2018). Unsupervised spectral-spatial feature learning via deep residual Conv–Deconv network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 56(1), 391–406.

    Article  Google Scholar 

  22. Nogueira, K., Penatti, O. A. B., & Santos, J. A. D. (2017). Towards better exploiting convolutional neural networks for remote sensing scene classification. Pattern Recognition, 61, 539–556.

    Article  Google Scholar 

  23. Rodarmel, C., & Shan, J. (2002). Principal component analysis for hyperspectral image classification. Surveying and Land Information Systems, 62(2), 115–122.

    Google Scholar 

  24. Teng, W., Wang, N., Shi, H., Liu, Y., & Wang, J. (2020). Classifier-constrained deep adversarial domain adaptation for cross-domain semisupervised classification in remote sensing images. IEEE Geoscience and Remote Sensing Letters, 17(5), 789–793.

    Article  Google Scholar 

  25. Tian, Z., Wang, L., Zhan, R., Hu, J., & Zhang, J. (2018). Classification via weighted kernel CNN: Application to SAR target recognition. International journal of remote sensing, 39(23), 9249–9268.

    Article  Google Scholar 

  26. Xia, G., Hu, J., Hu, F., Shi, B., Bai, X., Zhong, Y., et al. (2017). AID: A benchmark data set for performance evaluation of aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7), 3965–3981.

    Article  Google Scholar 

  27. Yang, W., Yin, X., & Xia, G. (2015). Learning high-level features for satellite image classification with limited labeled samples. IEEE Transactions on Geoscience and Remote Sensing, 53(8), 4472–4482.

    Article  Google Scholar 

  28. Zhang, M., Gong, M., He, H., Zhu, S. (2020). Symmetric all convolutional neural-network-based unsupervised feature extraction for hyperspectral images classification. IEEE Transactions on Cybernetics, in press.

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Xiaoxia Zhang.

Ethics declarations

Conflicts of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Guo, Y. & Zhang, X. High-Resolution Remote Sensing Image Scene Classification by Merging Multilevel Features of Convolutional Neural Networks. J Indian Soc Remote Sens (2021). https://doi.org/10.1007/s12524-021-01310-z

Download citation

Keywords

  • Convolutional neural network
  • High-resolution remote sensing image
  • Hyperspectral remote sensing data
  • Scene classification
  • Feature extraction
  • Integrated structure