In this article, we present an approach to land-use and land-cover (LULC) mapping from multispectral satellite images using deep learning methods. The terms satellite image classification and map production, although used interchangeably have specific meanings in the field of remote sensing. Satellite image classification describes assignment of global labels to entire scenes, whereas LULC map production involves producing maps by assigning a class to each pixel. We show that by classifying each pixel in a satellite image into a number of LULC categories we are able to successfully produce LULC maps. This process of LULC mapping is achieved using deep neural networks pre-trained on the ImageNet large-scale visual recognition competition datasets and fine-tuned on our target dataset, which consists of Landsat 5/7 multispectral satellite images taken of the Province of Manitoba in Canada. This approach resulted in 88% global accuracy. Performance was further improved by considering the state-of-the-art generative adversarial architecture and context module integrated with the original networks. The result is an automated deep learning framework that can produce highly accurate LULC maps images significantly faster than current semi-automated methods. The contribution of this article includes extensive experimentation of different FCN architectures with extensions on a unique dataset, high classification accuracy of 90.46%, and a thorough analysis and accuracy assessment of our results.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
A government agency mandated to create land-use/land-class maps of the province of Manitoba.
Treitz P, Rogan J (2004) Remote sensing for mapping and monitoring land-cover and land-use change an introduction. Prog Plan 61(4):269–279
Lu D, Weng Q (2007) A survey of image classification methods and techniques for improving classification performance. Int J Remote Sens 28(5):823–870
Mera D, Fernández-Delgado M, Cotos JM, Viqueira JR, Barro S (2017) Comparison of a massive and diverse collection of ensembles and other classifiers for oil spill detection in sar satellite images. Neural Comput Appl 28(1):1101–1117
Schmidhuber J (2015) Deep learning in neural networks: An overview. Neural Netw 61:85–117
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis (IJCV) 115(3):211–252
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition 07-12-June-2015, pp 3431–3440
Storie CD, Henry CJ (2018) Deep learning neural networks for land use land cover mapping. In: Proceedings of the 38th IEEE international geoscience and remote sensing symposium, p 4 (in press)
Henry CJ, Storie CD, Palaniappan M, Alhassan V, Swamy M, Aleshinloye D, Curtis A, Kim D (2019) Automated lulc map production using deep neural networks. Int J Remote Sens 40(11):4416–4440
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015a) Going deeper with convolutions. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–9
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778
Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:151107122
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680
Thoma M (2016) A survey of semantic segmentation. arXiv preprint arXiv:160206541
Dumoulin V, Visin F (2016) A guide to convolution arithmetic for deep learning. arXiv preprint arXiv:160307285
Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: Advances in neural information processing systems 27, Curran Associates, Inc., pp 3320–3328
Zheng S, Jayasumana S, Romera-Paredes B, Vineet V, Su Z, Du D, Huang C, Torr PH (2015) Conditional random fields as recurrent neural networks. In: Proceedings of the IEEE international conference on computer vision, IEEE Computer Society, pp 1529–1537
Krähenbühl P, Koltun V (2011) Efficient inference in fully connected CRFs with gaussian edge potentials. In: Advances in neural information processing systems 24, Curran Associates, Inc., pp 109–117
Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018b) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848
Chen LC, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:170605587
Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, Berlin, pp 234–241
Fu G, Liu C, Zhou R, Sun T, Zhang Q (2017) Classification for high resolution remote sensing imagery using a fully convolutional network. Remote Sens 9(5):498
Yang C, Rottensteiner F, Heipke C (2018) Classification of land cover and land use based on convolutional neural networks. ISPRS Ann Photogramm Remote Sens Spat Inf Sci 4(3):251–258
Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495
Paisitkriangkrai S, Sherrah J, Janney P, van den Hengel A (2016) Semantic labeling of aerial and satellite imagery. IEEE J Sel Top Appl Earth Obs Remote Sens 9(7):2868–2881
Zhong P, Gong Z, Schönlieb C (2016) A dbn-crf for spectral-spatial classification of hyperspectral data. In: 23rd international conference on pattern recognition, ICPR 2016, Cancún, Mexico, December 4–8, 2016, pp 1219–1224
Alam FI, Zhou J, Liew AW, Jia X, Chanussot J, Gao Y (2017) Conditional random field and deep feature learning for hyperspectral image segmentation. arXiv preprint arXiv:171104483
Castelluccio M, Poggi G, Sansone C, Verdoliva L (2015) Land use classification in remote sensing images by convolutional neural networks. arXiv preprint arXiv:150800092
Basu S, Ganguly S, Mukhopadhyay S, DiBiano R, Karki M, Nemani R (2015) DeepSat. In: Proceedings of the 23rd SIGSPATIAL international conference on advances in geographic information systems–GIS ’15, pp 1–10
Carreira-Perpiñán M, Hinton GE (2005) On contrastive divergence learning. In: Artificial intelligence and statistics, p 17
Marmanis D, Datcu M, Esch T, Stilla U (2016) Deep learning earth observation classification using ImageNet pretrained networks. IEEE Geosci Remote Sens Lett 13(1):105–109
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2015b) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826
Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2016) Inception-v4, inception-resNet and the impact of residual connections on learning. In: AAAI, vol 4, p 12
Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. arXiv preprint arXiv:161107004
Luc P, Couprie C, Chintala S, Verbeek J (2016) Semantic segmentation using adversarial networks. arXiv preprint arXiv:161108408
Yifang B, Gong P, Gini C (2015) Global land cover mapping using earth observation satellite data: recent progresses and challenges. ISPRS J Photogramm Remote Sens (Print) 103(1):1–6
Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: Large-scale machine learning on heterogeneous systems. Software available from tensorflow.org
Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980
Ruder S (2016) An overview of gradient descent optimization algorithms. arXiv preprint arXiv:160904747
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J MachLearn Res 15:1929–1958
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:150203167
Huang G, Liu Z, Van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 2261–2269
Chen L, Zhu Y, Papandreou G, Schroff F, Adam H (2018a) Encoder-decoder with atrous separable convolution for semantic image segmentation. arXiv preprint arXiv:180202611
This research has been supported by through a Queen Elizabeth II Diamond Jubilee scholarship and by the Natural Sciences and Engineering Research Council of Canada (NSERC) discovery Grants 194376 and 418413. Moreover, this work would not have been possible without the support and cooperation of GeoManitoba.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Alhassan, V., Henry, C., Ramanna, S. et al. A deep learning framework for land-use/land-cover mapping and analysis using multispectral satellite imagery. Neural Comput & Applic 32, 8529–8544 (2020). https://doi.org/10.1007/s00521-019-04349-9
- Deep learning
- Land use
- Land cover
- Deep neural networks
- Satellite images