Land Cover/Land Use Mapping of LISS IV Imagery Using Object-Based Convolutional Neural Network with Deep Features
- 112 Downloads
Abstract
The paper proposes a new method for classifying the LISS IV satellite images using deep learning method. Deep learning method is to automatically extract many features without any human intervention. The classification accuracy through deep learning is still improved by including object-based segmentation. The object-based deep feature learning method using CNN is used to accurately classify the remotely sensed images. The method is designed with the technique of extracting the deep features and using it for object-based classification. The proposed system extracts deep features using pre-defined filter values, thus increasing the overall performance of the process compared to randomly initialized filter values. The object-based classification method can preserve edge information in complex satellite images. To improve the classification accuracy and to reduce complexity, object-based deep learning technique is used. The proposed object-based deep learning approach is used to drastically increase the classification accuracy. Here, the remotely sensed images were used to classify the urban areas of Ahmadabad and Madurai cities. Experimental results show a better performance with the object-based classification.
Keywords
Convolutional neural network (CNN) Deep learning LISS IV image Image classification Multi-resolution segmentationIntroduction
Classifying different areas of remote sensing image has a wide variety of applications in fields such as land cover mapping and detection, water resource detection, agricultural usage, wetland mapping, geological information and urban and regional planning. However, the classification of remote sensing images remains a tedious task due to its complexity. Feature extraction plays an important role in classification. When features are chosen manually with human intervention, the efficiency of the classification process decreases. So, in order to improve the efficiency, we adopt an automatic feature learning method such as deep learning.
Deep learning is one of the excellent methods in artificial intelligence to learn discriminative features without human intervention. The work by LeCun et al. (2015) shows how deep learning is applied for classification. Unlike low-level feature representations, deeply learned features are generally more robust, and it has great effectiveness in image classification, such as face recognition (Sun et al. 2014) and scene classification (Li et al. 2010; Zhang et al. 2016a, b). In the remote sensing field, several researches are done for image classification using deep learning models, such as stacked autoencoder (SAE) and convolutional neural network (CNN). But, the original SAE extracts only one-dimensional spectral features which probably is not sufficient for high-resolution image classification. Therefore, another work is proposed (Chen et al. 2014b), which improved the SAE model by introducing spatial features for the efficient classification of hyperspectral images.
The CNN algorithm is popular for high-resolution image classification due to its effectiveness in spatial feature exploration (Zhao and Du 2016a, b; Zhao et al. 2015; Yue et al. 2015; Chen et al. 2014a). However, deep features are usually learned from local images patches and are pixel based, which sometimes may lead to misinterpretations. Also, deep feature extraction does not provide clear information about boundary and edges. So, to overcome the difficulty, the object-based classification is used instead of pixel-based classification. The object-based approach can be used to efficiently delineate and classify high-resolution imagery (Duveiller et al. 2008). Object-based classification method considers each image segments as building blocks for the further image analysis.
Now, let us look about the earlier works carried out for the classification of LISS IV images. The wavelet packet transform for texture analysis is presented for LISS IV image (Rajesh et al. 2011). Here, the statistical and co-occurrence features of the input patterns are first extracted and those features are used for classification. The Daubechies (DB2) wavelet filter is used for decomposition, and Mahalanobis distance classifier is used as the classifier. Since the methodology results in many features, some of which are found to be not useful, the best among the wavelet packet statistical and wavelet packet co-occurrence textural feature sets is selected using genetic algorithm (Rajesh et al. 2013). It outperforms the other feature reduction techniques like principal component analysis and linear discriminant analysis. Here, multilayer perceptron (MLP) layer is used for classification. The classification outperforms when the MLP for classification is replaced by adaptive neuro-fuzzy inference system (ANFIS) trained with backpropagation gradient descent method in combination with the least squares method (Rajesh and Arivazhagan 2014).
Our proposed system replaces the so-far carried-out works for classification of LISS IV image with a deep learning approach. CNN is used as a feature extractor, and deep features are obtained from LISS IV image. In order to still increase the efficiency, the obtained deep features are combined with the object-based textural features.
The organization of this paper is as follows: “Study Area and Data Used” section presents the study data and area. “Convolutional Neural Network (CNN)” section gives a simple introduction to convolutional neural network (CNN). “Proposed Method” section describes briefly about the proposed work and implementation. “Experimental Study and Results” section discusses the experimental study and results obtained. The conclusions are presented in “Conclusion” section.
Study Area and Data Used
Ground truth of Ahmadabad city
Ahmadabad LISS IV image (300 × 300)
Madurai city (size 400 × 400)
Satellite/sensor: | IRS P6/LISS IV |
---|---|
Resolution: | 5.8 m |
Band 2 (green): | 0.52–0.59 µm |
Band 3 (red): | 0.62–0.68 µm |
Band 4 (near-infrared): | 0.77–0.86 µm |
Convolutional Neural Network (CNN)
Architecture of convolutional neural network
In our proposed system, the CNN is used to extract high-level features automatically from the image with the help of manually initialized filters. The building blocks for CNN include convolutional layer, pooling layer, ReLU layer and fully connected layer. Usually, the CNN randomly initializes the filters and performs convolution operation. But the proposed system uses filters such as mean filter, Gaussian filter, Sobel filter, disk filter, log filter, Prewitt filter, Laplacian filter as initial filters. The CNN has two important concepts: Locally connected network and parameter sharing. If we use the fully connected network, then there will be large number of parameters required. So locally connected networks are used through which parameter requirement is reduced. Also, by using parameter sharing, instead of using new parameters every time the already used parameters are utilized.
Proposed Method
Flowchart for object-based image classification combined with deep features
Multi-resolution Segmentation
Object-based classification method improves the efficiency of classifying remotely sensed images. Segmentation groups the similar regions, and also it preserves the boundary information. The proposed system utilizes multi-resolution segmentation algorithm (Li et al. 2014) to segment image object. Scale, shape and compactness are important in segmenting image object. Scale, shape and compactness determine the size, shape and color of the image object, respectively. Objects of interest appear on different scales in an image. Therefore, the scale of resulting image objects should be freely adaptable to fit to the scale of task. Here, each pixel is considered as one image object. The image objects are merged based on local homogeneity, which describes the similarity of adjacent image objects, and larger image objects are obtained. The main components of multi-resolution segmentation are decision heuristics and definition of homogeneity of image objects. The decision heuristics determines the objects going to be merged at each stage, and definition of homogeneity of image objects is to compute the degree of fitting for a pair of image objects.
Extraction of Texture Feature
These features are extracted, and thus, the feature set is formed. The obtained feature set represents the object-based texture feature since the segmented image is given as input for feature extraction.
Deep Feature Extraction
Illustration of convolutional neural network
-
Depth It refers to the number of filters used for the convolution operation.
-
Stride It refers to the number of pixels sliding the weight matrix over the input matrix.
The final pooling, h, represents the deep feature, and g represents a point-wise nonlinear tanh activation function. Thus, the final deep features are obtained.
The extracted texture features (from Eqs. 1 to 10) such as mean, standard deviation, root mean square, energy, cluster shade, cluster prominence, correlation and deep features are combined and used for classification.
Object-Based Classification with Deep Features
The trained CNN is then used to classify the test image. Thus, the different portions of the remotely sensed image are classified and labeled efficiently.
Experimental Study and Results
The proposed method is implemented using Matlab R2018b. The section presents the performance analysis of the proposed method to classify the remotely sensed LISS IV images. Two satellite images captured by Indian Remote Sensing satellite IRS-P6 are taken for analysis. These images are subjected to geometric correction with the help of ground control points (GCPS). As mentioned above, we combine deep feature and object-based classification method for accurate high-resolution image interpretation.
Performance of Proposed Method to Ahmadabad LISS IV Image
Classification accuracy of Ahmadabad dataset
CNNs | WB | VEG | UB | LC |
---|---|---|---|---|
One-layer | 89 | 70 | 84 | 78 |
Two-layer | 89 | 78 | 88 | 82 |
Three-layer | 91 | 84 | 88 | 86 |
Four-layer | 92 | 90 | 92 | 90 |
Performance of different configuration CNNs—Ahmadabad dataset
CNNs | Precision (%) | Recall (%) | Accuracy (%) | F1 (%) |
---|---|---|---|---|
One-layer | 80 | 82 | 80.5 | 81 |
Two-layer | 86 | 86 | 84.5 | 86 |
Three-layer | 88 | 87 | 87 | 87 |
Four-layer | 90 | 90 | 90.5 | 90 |
Classified output of Ahmadabad using CNN
Confusion matrix for Ahmadabad city image
Performance of Proposed Method to Madurai LISS IV Image
Classification accuracy of Madurai dataset
CNNs | UB | VEG | WB | WL | SL |
---|---|---|---|---|---|
One-layer | 91.1 | 88.8 | 100 | 80 | 71.3 |
Two-layer | 93 | 93 | 100 | 80 | 77 |
Three-layer | 95 | 93 | 100 | 86 | 86 |
Four-layer | 95 | 93 | 100 | 91 | 84 |
Performance of different configuration CNNs—Madurai dataset
CNNs | Precision (%) | Recall (%) | Accuracy (%) | F1 (%) |
---|---|---|---|---|
One-layer | 86 | 87 | 86 | 87 |
Two-layer | 90 | 87 | 88 | 89 |
Three-layer | 93 | 91 | 89 | 92 |
Four-layer | 94 | 92 | 92 | 93 |
Classified output of Madurai using CNN
Ground control points of Madurai city
Name of the area | Latitude | Longitude | Elevation (Feet) | Accuracy (Feet) | Features |
---|---|---|---|---|---|
Kudhal nagar tank | 9°57′03.9″ | 78°06′14.8″ | 138 | 24 | Vegetation |
Sellur tank | 9°56′26.4″ | 78°07′06.3″ | 148 | 27 | Water body |
Malar nagar | 9°57′25.2″ | 78°06′52.3″ | 140 | 25 | Waste land |
Alangulam tank | 9°57′25.6″ | 78°07′14.4″ | 138 | 24 | Water body |
Reserve line | 9°56′49.8″ | 78°07′56.1″ | 140 | 26 | Urban |
Burma colony | 9°51′20.5″ | 78°06′07.1″ | 148 | 21 | Urban |
Perungudi | 9°50′33.9″ | 78°05′33.2″ | 134 | 24 | Waste land |
Ring road 1 | 9°51′23.4″ | 78°07′08.5″ | 127 | 22 | Waste land |
Ring road 2 | 9°51′31.7″ | 78°07′15.0″ | 125 | 22 | Waste land |
Chinthamani | 9°53′14.8″ | 78°08′37.8″ | 133 | 26 | Urban |
Melamadai | 9°55′38.9″ | 78°09′06.7″ | 135 | 25 | Waste land |
Anna nagar | 9°55′04.4″ | 78°08′55.3″ | 140 | 29 | Urban |
Vandiyur stop | 9°54′35.3″ | 78°09′36.5″ | 122 | 26 | Urban |
Vandiyur extend | 9°54′24.0″ | 78°09′47.3″ | 126 | 24 | Waste land |
Vaigai river | 9°54′45.3″ | 78°09′20.1″ | 122 | 23 | Water body |
Confusion matrix for Madurai image
a Object-based deep learning technique, b DB2, c DB2 with GA and ANN, d DB2 with GA and ANFIS
Classification results of different algorithms
Overall | Producer | User | Kappa | |
---|---|---|---|---|
DB2 with ANN | 83.62 | 84.00 | 83.10 | 0.80 |
DB2 with ANFIS | 86.01 | 85.19 | 84.28 | 0.80 |
DB2 | 87.60 | 89.57 | 82.02 | 0.82 |
Object-based deep learning | 92 | 96.4 | 94.6 | 0.85 |
Conclusion
The system utilized the CNN for the effective classification of high-resolution LISS IV images using object-based strategy. Feature extraction is one of the toughest tasks in the analysis of the images. The challenge was efficiently handled by choosing CNN for automatic feature learning with the help of pre-defined filter values. We propose an effective way to classify high-resolution LISS IV images by combining deep features with image objects. The deep features are evaluated by testing the CNN framework with five different layer configurations for the classification of images. The proposed approach significantly reduces the complexity of the feature set and also improves the classifier performance when compared to earlier approaches. The results obtained by the proposed method on Ahmadabad City remotely sensed image and Madurai city remotely sensed image demonstrate that the proposed method outperforms other studied methods.
Notes
References
- Chen, X., Xiang, S., Liu, C. L., & Pan, C. H. (2014a). Vehicle detection in satellite images by hybrid deep convolutional neural networks. IEEE Geoscience and Remote Sensing Letters,11(10), 1797–1801.CrossRefGoogle Scholar
- Chen, Y., Lin, Z., Zhao, X., Wang, G., & Gu, Y. (2014b). Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,7(6), 2094–2107.CrossRefGoogle Scholar
- Duveiller, G., Defourny, P., Desclée, B., & Mayaux, P. (2008). Deforestation in Central Africa: Estimates at regional, national and landscape levels by advanced processing of systematically-distributed landsat extracts. Remote Sensing of Environment,112(5), 1969–1981.CrossRefGoogle Scholar
- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. Cambridge: MIT Press.Google Scholar
- LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature,521(7553), 436–444.CrossRefGoogle Scholar
- Li, H., Tang, Y., Liu, Q., Ding, H., Jing, L., Lin, Q. (2014). A novel multi-resolution segmentation algorithm for high resolution remote sensing imagery based on minimum spanning tree and minimum heterogeneity criterion. In Proceedings of IEEE international geoscience remote sensing symposium (vol. 3, pp. 2850–2854).Google Scholar
- Li, L.-J., Su, H., Fei-Fei, L., & Xing, E. P. (2010). Object bank: A high-level image representation for scene classification and semantic feature sparsification. In Proceedings of the neural information processing systems conference (pp. 1378–1386).Google Scholar
- Rajesh, S., & Arivazhagan, S. (2014). adaptive neuro-fuzzy inference system based land cover/land use mapping of LISS IV imagery using wavelet packet transform. Journal of the Indian Society of Remote Sensing,42(2), 267–277.CrossRefGoogle Scholar
- Rajesh, S., Arivazhagan, S., Pradeep Moses, K., & Abisekaraj, R. (2011). Land cover/land use mapping using different wavelet packet transforms for LISS IV Madurai imagery. Journal of the Indian Society of Remote Sensing,40(2), 313–324.CrossRefGoogle Scholar
- Rajesh, S., Arivazhagan, S., Pradeep Moses, K., & Abisekaraj, R. (2013). Genetic algorithm based feature subset selection for land cover/land use mapping using wavelet packet transform. Journal of the Indian Society of Remote Sensing,41(2), 237–248.CrossRefGoogle Scholar
- Sun, Y., Chen, Y., Wang, X., & Tang, X. (2014). Deep learning face representation by joint identification- verification. In Proceedings of advances neural information processing system conference (pp. 1988–1996).Google Scholar
- Yue, J., Zhao, W., Mao, S., & Liu, H. (2015). Spectral–spatial classification of hyperspectral images using deep convolutional neural networks. Remote Sensing Letters,6(6), 468–477.CrossRefGoogle Scholar
- Zhang, F., Du, B., & Zhang, L. (2016a). Scene classification via a gradient boosting random convolutional network framework. IEEE Transaction on Geoscience and Remote Sensing,54(3), 1793–1802.CrossRefGoogle Scholar
- Zhang, F., Du, B., Zhang, L., & Xu, M. (2016b). Weakly supervised learning based on coupled convolutional neural networks for aircraft detection. IEEE Transaction Geoscience and Remote Sensing,54(9), 5553–5563.CrossRefGoogle Scholar
- Zhao, W., & Du, S. (2016a). Spectral-spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach. IEEE Transaction on Geoscience and Remote Sensing,54(8), 4544–4554.CrossRefGoogle Scholar
- Zhao, W., & Du, S. (2016b). Learning multiscale and deep representations for classifying remotely sensed imagery. ISPRS Journal Photogrammetry Remote Sensing,113, 155–165.CrossRefGoogle Scholar
- Zhao, W., Guo, Z., Yue, J., Zhang, X., & Luo, L. (2015). On combining multiscale deep learning features for the classification of hyperspectral remote sensing imagery. International Journal of Remote Sensing,36(13), 3368–3379. https://doi.org/10.1080/2150704X.2015.1062157.CrossRefGoogle Scholar
- Zhao, W., Shihong, D., & Emery, W. J. (2017). Object-based convolutional neural network for high-resolution imagery classification. IEEE Journal of Selected topics in Applied Earth observation and Remote Sensing,10(7), 3386–3396.CrossRefGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.