Advertisement

Pixel-Based Classification of Hyperspectral Images Using Convolutional Neural Networks

  • Syed Aamer Hussain
  • Ali TahirEmail author
  • Junaid Aziz Khan
  • Ahmad Salman
Original Article
  • 100 Downloads

Abstract

The recent progress in geographical information systems, remote sensing (RS) and data analytics enables us to acquire and process large amount of Earth observation data. Convolutional neural networks (CNN) are being used frequently in classification of multi-dimensional images with high accuracy. In this paper, we test CNNs for the classification of hyperspectral RS data. Our proposed CNN is a multi-layered neural network architecture, which is tailored to classify objects based on pixel-wise spatial information using spectral bands of hyperspectral imagery (HSI). We use benchmark satellite imagery in four different HSI datasets for classification using the proposed architecture. Our results are compared with support vector machine (SVM) and extreme learning machine (ELM) algorithms, which are frequently used techniques of machine learning in RS data classification. Moreover, we also provide a comparison with the state-of-the-art CNN approaches, which have been used for HSI classification. Our results show improvements of up to 6% on average over SVM and ELM while up to 4% improvement is observed in comparison with two recently proposed CNN architectures for HSI classification accuracy. On the other hand, the processing time of our proposed CNN is also significantly lower.

Keywords

Hyperspectral data Machine learning Convolutional neural networks 

Zusammenfassung

Pixelweise Klassifizierung von Hyperspektralszenen mit Convolutional Neural Networks. Der Fortschritt bei Geoinformationssystemen, Fernerkundung und Datenanalyse erlaubt uns die Gewinnung und Verarbeitung von umfangreichen Erdbeobachtungdaten. Convolutional Neural Networks (CNN) werden oft zur Klassifizierung von multidimensionalen hoch aufgelösten Bilddaten verwendet. In diesem Artikel untersuchen wir die Eignung von CNNs für die Klassifizierung von hyperspektralen Fernerkundungsdaten. Das von uns vorgeschlagene CNN besitzt die Struktur eines neuronalen Netzwerks mit mehreren Ebenen zur Objekt-Klassifizierung auf der Grundlage einer pixelweisen Auswertung der hyperspektralen Bilddaten. Zur Verifizierung unserer Klassifizierungsmethode benutzen wir vier verschiedene Datensätze, aufgenommen von Satellitenplattformen. Die Ergebnisse werden mit denen der Methoden Support Vector Machine (SVM) und Extreme Learning Machine (ELM), die beide bei automatischen Klassifizierungsverfahren der Fernerkundung weit verbreitet sind, verglichen. Darüber hinaus liefern wir einen Vergleich zu aktuellen Ansätzen der CNN. Unsere Ergebnisse zeigen eine Verbesserung der Klassifizierungsgenauigkeit von 6% gegenüber SVM und ELM sowie eine Verbesserung von 4% gegenüber kürzlich veröffentlichen CNN-Architekturen. Darüber hinaus ist unser Ansatz deutlich schneller.

References

  1. Abraham A (2005) Artificial neural networks. In: Sydenham PH, Thorn R (eds) Handbook of measuring system design.  https://doi.org/10.1002/0471497398.mm421
  2. Ba J, Mnih V, Kavukcuoglu K (2014) Multiple object recognition with visual attention. arXiv:1412.7755
  3. Belgiu M, Drgu L (2016) Random forest in remote sensing: a review of applications and future directions. ISPRS J Photogramm Remote Sens 114:24–31CrossRefGoogle Scholar
  4. Bengio Y (2009) Learning deep architectures for AI. Found Trends Mach Learn 2(1):1–127CrossRefGoogle Scholar
  5. Blaschke T (2010) Object based image analysis for remote sensing. ISPRS J Photogramm Remote Sens 65:2–16CrossRefGoogle Scholar
  6. Bottou L (2012) Stochastic gradient descent tricks. In: Montavon G, Orr GB, Müller KR (eds) Neural networks: tricks of the trade. Lecture Notes in Computer Science. Springer, Berlin, vol 7700. pp 421–436Google Scholar
  7. Dan C, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image classification. arXiv:1202.2745
  8. Cao X, Zhou F, Xu L, Meng D, Xu Z, Paisley J (2017) Hyperspectral image segmentation with markov random fields and a convolutional neural network. arXiv:1705.00727
  9. Chetlur S, Woolley C, Vandermersch P, Cohen J, Tran J, Catanzaro B et al (2014) CuDNN: efficient primitives for deep learning. arXiv:1410.0759
  10. Dahl GE, Sainath TN, Hinton GE (2013) Improving deep neural networks for LVCSR using rectified linear units and dropout. In: Acoustics, speech and signal processing (ICASSP), 2013 IEEE international conference on, IEEE, pp 8609-8613Google Scholar
  11. Guidici D, Clark ML (2017) One-dimensional convolutional neural network land-cover classification of multi-seasonal hyperspectral imagery in the San Francisco Bay Area, California. Remote Sens 9:629CrossRefGoogle Scholar
  12. Ham J, Chen Y, Crawford MM, Ghosh J (2005) Investigation of the random forest framework for classification of hyperspectral data. IEEE Trans Geosci Remote Sens 43(3):492–501CrossRefGoogle Scholar
  13. Hu W, Huang Y, Wei L, Zhang F, Li H (2015) Deep convolutional neural networks for hyperspectral image classification. J Sens 2015:258619.  https://doi.org/10.1155/2015/258619 CrossRefGoogle Scholar
  14. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501CrossRefGoogle Scholar
  15. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R et al (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on multimedia. Florida, USAGoogle Scholar
  16. Kampffmeyer M, Salberg A-B, Jenssen R (2016) Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 1–9Google Scholar
  17. Kasturi R, Goldgof D, Soundararajan P, Manohar V, Garofolo J, Bowers R et al (2009) Framework for performance evaluation of face, text, and vehicle detection and tracking in video: data, metrics, and protocol. IEEE Trans Pattern Anal Mach Intell 31:319–336CrossRefGoogle Scholar
  18. Kim Y (2014) Convolutional neural networks for sentence classification. arXiv:1408.5882
  19. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Advances in neural information processing systemsGoogle Scholar
  20. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105Google Scholar
  21. Landgrebe D (2002) Hyperspectral image data analysis. IEEE Signal Process Mag 19:17–28CrossRefGoogle Scholar
  22. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444CrossRefGoogle Scholar
  23. LeCun Y, Huang FJ, Bottou L (2004) Learning methods for generic object recognition with invariance to pose and lighting. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition. DC, USA, WashingtonGoogle Scholar
  24. LeCun Y (2015) Learning methods for generic object recognition with invariance to pose and lighting. In: Computer vision and pattern recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, vol 2. pp II–104). IEEE.Google Scholar
  25. Li J, Bioucas Dias JM, Plaza A (2013) Spectral spatial classification of hyperspectral data using loopy belief propagation and active learning. IEEE Trans Geosci Remote Sens 51:844–856CrossRefGoogle Scholar
  26. Li W, Fu H, Yu L, Cracknell A (2016) Deep learning based oil palm tree detection and counting for high-resolution remote sensing images. Remote Sens 9:22CrossRefGoogle Scholar
  27. Li X, Wu T, Liu K, Li Y, Zhang L (2016) Evaluation of the Chinese fine spatial resolution hyperspectral satellite TianGong-1 in urban land-cover classification. Remote Sens 8:438CrossRefGoogle Scholar
  28. Li W, Wu G, Zhang F, Du Q (2017) Hyperspectral image classification using deep pixel-pair features. IEEE Trans Geosci Remote Sens 55:844–853CrossRefGoogle Scholar
  29. Lyu MR, Song J, Cai M (2005) A comprehensive method for multilingual video text detection, localization, and extraction. IEEE Trans Circuits Syst Video Technol 15:243–255CrossRefGoogle Scholar
  30. Ma Y, Wang L, Liu D, Liu P, Wang J, Tao J (2012) Generic parallel programming for massive remote sensing data processing. In: IEEE international conference on cluster computing, Beijing, ChinaGoogle Scholar
  31. Makantasis K, Karantzalos K, Doulamis A, Doulamis N (2015) Deep supervised learning for hyperspectral data classification through convolutional neural networks. In: Geoscience and remote sensing symposium (IGARSS), 2015 IEEE international. Italy, MilanGoogle Scholar
  32. Marmanis D, Wegner JD, Galliani S, Schindler K, Datcu M, Stilla U (2016) Semantic segmentation of aerial images with an ensemble of CNSS. ISPRS Ann Photogramm Remote Sens Spat Inf Sci 3:473–480CrossRefGoogle Scholar
  33. Melgani F, Bruzzone L (2004) Classification of hyperspectral remote sensing images with support vector machines. IEEE Trans Geosci Remote Sens 42:1778–1790CrossRefGoogle Scholar
  34. Mhaskar H, Liao Q, Poggio TA (2017) When and why are deep networks better than shallow ones? In: AAAI, pp 2343–2349Google Scholar
  35. Plaza AJ (2009) Special issue on architectures and techniques for real-time processing of remotely sensed images. J Real Time Image Process 4:191–193CrossRefGoogle Scholar
  36. Saprykin O, Fedoseev A, Mikheeva T (2016) Recognition of urban transport infrastructure objects via hyperspectral images. In: VEHITS, pp 203–208Google Scholar
  37. Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681CrossRefGoogle Scholar
  38. Wu F-Y, Yan S-Y, Smith JS, Zhang B-L (2017) Traffic scene recognition based on deep CNN and VLAD spatial pyramids. arXiv:1707.07411
  39. Zhang T (2001) An introduction to support vector machines and other kernel-based learning methods. AI Mag 22(2):103Google Scholar

Copyright information

© Deutsche Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation (DGPF) e.V. 2019

Authors and Affiliations

  1. 1.Institute of Geographical Information SystemsNational University of Sciences and TechnologyIslamabadPakistan
  2. 2.School of Electrical Engineering and Computer ScienceNational University of Sciences and TechnologyIslamabadPakistan

Personalised recommendations