A Literature Review on Hadoop Ecosystem and Various Techniques of Big Data Optimization

  • Vikash Kumar Singh
  • Manish Taram
  • Vinni Agrawal
  • Bhartee Singh Baghel
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 38)

Abstract

We are living in twenty-first century, and this century means for its faster work, accurate analysis, highly processed data, and speed. This is the epoch of “Big data.” Big data is a term that describes huge mass of structured and unstructured data that is unable to be processed by traditional data processing systems. Big data stands for storage of large amount of data to extract the valuable content with its characteristics 5-Vs, i.e., Volume, Variety, Velocity, Veracity, and Value. But before the arrival of Hadoop, procuring and depository of data was an issue. Hadoop takes its first step in the Data Science Market in 2005. It was created by Doug Cutting and Mike Cafarella. Hadoop is a software framework that allows users to depot data and run their applications on Hadoop clusters. Its best part is its open-source framework.

Keywords

Hadoop Big data MapReduce Pig Hive Sqoop 

References

  1. 1.
    Bagriyanik S, Karahoca A (2016) Big Data in software engineering: a systematic literature review. Glob J Inf Technol 6(1):107–116Google Scholar
  2. 2.
    Tsai CW, Lai CF, Chao1 HC, Vasilakos AV (2015) Big Data analytics: a survey, of Big Data 2:21.  https://doi.org/10.1186/s40537-015-0030-3
  3. 3.
    Saltz JS, Shamshurin I (2016) Big Data team process methodologies: a literature review and the identification of key factors for a project’s success. In: 2016 IEEE International Conference on Big Data (Big Data)Google Scholar
  4. 4.
    Nelson B, Olovsson T Security and privacy for Big Data: a systematic literature review. In: 2016 IEEE International Conference on Big Data (Big Data)Google Scholar
  5. 5.
    Kumari S A review paper on Big Data and Hadoop. Int J Recent Adv Eng Technol (IJRAET) 4(1):2347–2812 (For National Conference on Recent Innovations in Science, Technology & Management (NCRISTM) ISSN (Online))Google Scholar
  6. 6.
    Ularu EG, Puican FC, Apostu A, Velicanu M (2012) Perspectives on Big Data and Big Data analytics. Database Sys J III(4)Google Scholar
  7. 7.
    Anjali PP, Binu A (2014) A comparative survey based on processing network traffic data using Hadoop Pig and typical map-reduce. Int J Comput Sci Eng Surv (IJCSES) 5(1)Google Scholar
  8. 8.
    Assunção MD, Calheiros RN, Bianchi S, Netto MA, Buyya R (2015) Big Data computing and clouds: trends and future directions. J Parallel Distrib Comput 79–80:3–15 (Elsevier)CrossRefGoogle Scholar
  9. 9.
    Mukherjee S, Shaw R Big Data—concepts, applications, challenges and future scope. Int J Adv Res Comput Commun Eng 5(2)Google Scholar
  10. 10.
    Sreedhar C, Kasiviswanath N, Reddy PC (2017) Clustering large datasets using K-means modified inter and intra clustering (KMI2C) in Hadoop. J Big Data (Springer)Google Scholar
  11. 11.
    Taylor R (2010) An overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics Author. In: Pacific Northwest National Laboratory Bioinformatics Open Source Conference 2010 Richland, WACrossRefGoogle Scholar
  12. 12.
    Lu H, Hai-Shan C, Ting-Ting H (2012) Research on Hadoop cloud computing model and its applications. In: 2012 third international conference on networking and distributed computingGoogle Scholar
  13. 13.
    Dhavapriya M, Yasodha N (2016) Big data analytics: challenges and solutions using Hadoop, map reduce and big table. Int J Comput Sci Trends Technol (IJCST) 4(1) Jan–Feb 2016Google Scholar
  14. 14.
    Wang L, Taoc J, Ranjan R, Marten H, Streit A, Chene J, Chena D (2013) G-Hadoop: MapReduce across distributed data centers for data-intensive computing. Future Gener Comput Sys 29:739–750, ElsevierCrossRefGoogle Scholar
  15. 15.
    Dean J, Ghemawat S (2004) MapReduce: simplifed data processing on large clusters. research.google.com/archive/mapreduceGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • Vikash Kumar Singh
    • 1
  • Manish Taram
    • 1
  • Vinni Agrawal
    • 1
  • Bhartee Singh Baghel
    • 1
  1. 1.IGNTUAmarkantakIndia

Personalised recommendations