Scene Classification of High-Resolution Remote Sensing Image Using Transfer Learning with Multi-model Feature Extraction Framework
Abstract
The remote sensing image is full of scene information. Traditional classification methods are based on the artificial extraction feature, can not effectively express the high-level semantic information, and it requires a lot of high-quality training labeled data. However, the labeled data is usually scarce, and difficult to obtain. Transfer learning is a machine learning method that uses existing knowledge to solve those problems different but related. It can effectively solve the learning problem with only a small number of labeled sample data in the target field. ImageNet and remote sensing images have similar characteristics in image texture, lines, color, structure and space. In this paper, we propose a scene classification method of high spatial resolution remote sensing images using transfer learning with multi-model feature extraction network. It designs a combination of multiple pretrained CNN models to extract the features of remote sensing images, and integrates the features into one-dimensional feature vector. This forms a deep feature extraction framework that enriches feature expression and facilitates the capture of remote sensing image features. After feature extraction, a dropout layer and a fully connected layer are used, followed by a classifier. This method achieves a maximum accuracy of 97.38% on the UC Merced dataset and a maximum of 93.97% accuracy on the AID dataset, which is significantly better than the existing method and improves the classification accuracy.
Keywords
Remote sensing image Convolution neural network Scene classification Multi-model integration Feature extraction and fusion Transfer learningReferences
- 1.Cheng, G., Han, J., Lu, X.: Remote sensing image scene classification: benchmark and state of the art. Proc. IEEE 105(10), 1865–1883 (2017)CrossRefGoogle Scholar
- 2.Vailaya, A., Figueiredo, M.A., Jain, A.K., Zhang, H.J.: Image classification for content-based indexing. IEEE Trans. Image Process. 10(1), 117–130 (2001)CrossRefGoogle Scholar
- 3.Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2169–2178 (2006)Google Scholar
- 4.Yang, Y., Newsam, S.: Spatial pyramid co-occurrence for image classification. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1465–1472, November 2011Google Scholar
- 5.Gueguen, L.: Classifying compound structures in satellite images: a compressed representation for fast queries. IEEE Trans. Geosci. Remote Sens. 53(4), 1803–1818 (2015)CrossRefGoogle Scholar
- 6.Cheriyadat, A.M.: Unsupervised feature learning for aerial scene classification. IEEE Trans. Geosci. Remote Sens. 52(1), 439–451 (2014)CrossRefGoogle Scholar
- 7.Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
- 8.Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
- 9.Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)Google Scholar
- 10.Chollet, F. Xception: deep learning with depthwise separable convolutions. arXiv preprint(2016)Google Scholar
- 11.He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
- 12.Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, No. 2, p. 3, July 2017Google Scholar
- 13.Uba, N.K.: Land Use and Land Cover Classification Using Deep Learning Techniques. Arizona State University, Tempe (2016)Google Scholar
- 14.Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)CrossRefGoogle Scholar
- 15.Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)Google Scholar
- 16.Marmanis, D., Datcu, M., Esch, T., Stilla, U.: Deep learning earth observation classification using ImageNet pretrained networks. IEEE Geosci. Remote Sens. Lett. 13(1), 105–109 (2016)CrossRefGoogle Scholar
- 17.Donahue, J., et al.: Decaf: a deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning, pp. 647–655, January 2014Google Scholar
- 18.Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)
- 19.Xia, G.S., et al.: AID: a benchmark data set for performance evaluation of aerial scene classification. IEEE Trans. Geosci. Remote Sens. 55(7), 3965–3981 (2017)CrossRefGoogle Scholar
- 20.Zhang, F., Du, B., Zhang, L.: Saliency-guided unsupervised feature learning for scene classification. IEEE Trans. Geosci. Remote Sens. 53(4), 2175–2184 (2015)CrossRefGoogle Scholar
- 21.Hu, F., Xia, G.S., Wang, Z., Huang, X., Zhang, L., Sun, H.: Unsupervised feature learning via spectral clustering of multidimensional patches for remotely sensed scene classification. IEEE J. Sel. Top. Appl. Earth Obser. Remote Sens. 8(5) (2015)Google Scholar