Advertisement

Earth Science [Big] Data Analytics

Chapter

Abstract

Tremendous research has been done and is still in progress in the domain of earth science. With the advent of Big Data and availability of datasets on Earth science, the study of Earth sciences has reached new dimensions. The diversity and high dimensional remote sensing data have provided with complex data sets capable of giving insights and intelligence that was not possible in last decades. With Computing progress made in ingesting and inferring data from myriad of sources including high resolution cameras mounted on satellites and sensors giving access to unconventional Big Data and also with the GPU computing and Data science advances we are today able to leverage machine learning and deep learning in extensively complex datasets derived from remote sensing about Earth Sciences. Our focus is to analyze what exactly does big data mean in earth science applications and how can big data provide added value in this context. Furthermore, this chapter demonstrates various big data tools which can be mapped with various techniques to be used for experimenting earth science datasets, processed, and exploited for different earth science applications. In order to illustrate the aforementioned aspects, instances are presented in order to demonstrate the use of big data in remote sensing. Firstly, this chapter presents earth science studies, application areas/research fields and a brief insight on earth science data. Then various techniques implemented in this domain are elaborated viz. classification, clustering, regression, deep learning, pattern recognition, machine learning, earth data analysis and processing. Later this chapter introduces big data analytics and various tools/platforms in big data viz. BigInsights, GIS, Jupyter notebook, Matlab, Python. Finally, it is shown how these tools are mapped to Earth science datasets using ArcGIS to illustrate with experimental instances the inferences and patterns generated.

Keywords

Earth data analysis and processing Geosciences GIS Big data analytics Python MATLAB Jupyter notebook BigInsights ArcGIS 

References

  1. 1.
    Karpatne A, Liess S (2015) A guide to earth science data: summary and research challenges. IEEE Comput Sci Eng 14–18CrossRefGoogle Scholar
  2. 2.
    Kempler S, Mathews T (2016) Earth science data analytics tools, techniques and more. In: ESIP Summer Meeting. ESIP CommonsGoogle Scholar
  3. 3.
  4. 4.
    Torahi AA, Rai SC (2011) Land cover classification and forest change analysis, using satellite imagery—a case study in Dehdez Area of Zagros Mountain in Iran. J Geogr Inf Syst 3:1–11Google Scholar
  5. 5.
    Steinbach M, Tan P-N, Boriah S, Kumar V, Klooster S, Potter C (2006) The application of clustering to earth science data: progress and challenges. In: Proceedings of the 2nd NASA data mining workshopGoogle Scholar
  6. 6.
    Clustering. Japan Association of Remote Sensing. Available at http://wtlab.iis.u-tokyo.ac.jp/~wataru/lecture/rsgis/rsnote/cp11/cp11-3.htm
  7. 7.
    Freitas AA (2008) A review of evolutionary algorithms for data mining. In: Maimon O, Rokach L (eds) Soft computing for knowledge discovery and data mining. Springer, New York, pp 79–111. ISSN 978-0-387-69935-6,  https://doi.org/10.1007/978-0-387-69935-6_4CrossRefGoogle Scholar
  8. 8.
    Engelbrecht AP (2007) Computational intelligence: an introduction, 2nd edn. Wiley, SussexCrossRefGoogle Scholar
  9. 9.
    Freitas AA (2003) A survey of evolutionary algorithms for data mining and knowledge. In: Ghosh A, Tsutsui S (eds) Advances in evolutionary computing: theory and applications. Springer, New York, pp 819–846CrossRefGoogle Scholar
  10. 10.
    Cuddy SJ, Glover PWJ (2002) The application of fuzzy logic and genetic algorithms to reservoir characterization and modeling. In: Wong P, Aminzadeh F, Nikravesh M (eds) Soft Computing for reservoir characterization and modeling. Springer, Berlin, pp 219–242. ISSN 14349922CrossRefGoogle Scholar
  11. 11.
  12. 12.
    Patra P (2011) Remote sensing and geographical information system (GIS). Assoc Geogr Stud 1–28Google Scholar
  13. 13.
  14. 14.
  15. 15.
    Introduction to InfoSphere BigInsights. IBM Knowledge Center, available at https://www.ibm.com/support/knowledgecenter/SSERCR_1.0.0/com.ibm.swg.im.infosphere.biginsights.product.doc/doc/c0057605.htmlGoogle Scholar
  16. 16.
    Lin JW-B (2012) Why Python is the next wave in earth sciences computing. Bull Am Meteor Soc 93(12):1823–1824.  https://doi.org/10.1175/BAMS-D-12-00148.1CrossRefGoogle Scholar
  17. 17.
    Groenendijk M (2017) Mapping all the things with Python. IBM Watson Data Lab. Available at https://medium.com/ibm-watson-data-lab/mapping-all-the-things-with-python-1228187dc665
  18. 18.
  19. 19.
    Dean J, Ghemawat S (2008) MapReduce: simplified data processing on large clusters. Commun ACM 51:107–113CrossRefGoogle Scholar
  20. 20.
    Li Z, Yang C, Jin B, Yu M, Liu K, Sun M, Zhan M (2015) Enabling big geoscience data analytics with a cloud-based, MapReduce-enabled and service-oriented workflow framework. PLoS ONE 10(3):e0116781.  https://doi.org/10.1371/journal.pone.0116781CrossRefGoogle Scholar
  21. 21.
    Kamal Sarwar, Ripon SH, Dey N, Ashour AS, Santhi V (2016) A MapReduce approach to diminish imbalance parameters for big deoxyribonucleic acid dataset. Comput Methods Programs Biomed 131:191–206CrossRefGoogle Scholar
  22. 22.
    MapReduce for Gridding LIDAR Data. In: Applications and Limitations of MapReduce. Available at http://mapreduce-specifics.wikispaces.asu.edu
  23. 23.
    Ericson G, Franks L, Rorer B (2017) How to choose algorithms for Microsoft Azure Machine Learning. Available at https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-algorithm-choice
  24. 24.
    Ferguson M (2012) Architecting a big data platform for analytics. Intelligent Business Strategies LimitedGoogle Scholar
  25. 25.
    Dutt V, Chaudhry V, Khan I (2012) Pattern recognition: an overview. Am J Intell Syst 2(1):23–27.  https://doi.org/10.5923/j.ajis.20120201.04CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  1. 1.IBM India Pvt. Ltd.BangaloreIndia
  2. 2.NIT JalandharJalandharIndia

Personalised recommendations