Abstract
Big Data, as we all know, is becoming a new technological trend in the industries, in science and even businesses. Indefinite data scalability allows organizations to process huge amounts of data in parallel, assisting dramatically decrease the amount of time it takes to manage several amount of work, optimize hardware resource usage and permit the extreme quantity of data per node to be handled. Optimization is to done to attain the finest strategy relative to a set of selected constraints which include maximizing factors such as efficiency, productivity, reliability, strength, and utilization. When the current system becomes insufficient, instead of upgrading it by adding more components to the existing structure you just add more computers to a cluster. This research discusses a hierarchical architecture of Hadoop Nodes namely Name nodes and Data nodes and mainly focuses on the optimization of Data Node by distributing some of its work load to Name Node.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Yadav, K., Pandey, M., Rautaray, S.S.: Feedback analysis using big data tools. In: International Conference on ICT in Business Industry & Government (ICTBIG). IEEE (2016)
Chakraborty, S. et al.: A proposal for high availability of HDFS architecture based on threshold limit and saturation limit of the namenode (2017)
Jena, B. et al.: Name node performance enlarging by aggregator based HADOOP framework. In: 2017 International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)(I-SMAC). IEEE (2017)
Shvachko, K., et al.: The hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST). IEEE (2010)
Jahani, Eaman, Cafarella, Michael J., Ré, Christopher: Automatic optimization for MapReduce programs. Proc. VLDB Endow. 4(6), 385–396 (2011)
Lee, K.-H. et al.: Parallel data processing with MapReduce: a survey. ACM sIGMoD Record 40(4), 11–20 (2012)
White, T.: Hadoop: The Definitive Guide. O’Reilly Media, Inc. (2012)
Kanaujia, P.K.M., Pandey, M., Rautaray, S.S.: Real time financial analysis using big data technologies. In: 2017 International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)(I-SMAC). IEEE (2017)
Borthakur, Dhruba: The hadoop distributed file system: architecture and design. Hadoop Proj. Website 11(2007), 21 (2007)
Jena, B. et al.: A survey work on optimization techniques utilizing map reduce framework. Hadoop Cluster. Int. J. Intell. Syst. Appl. 9(4), 61 (2017)
Feng, D., Zhu, L., Zhang, L.: Review of hadoop performance optimization. In: 2016 2nd IEEE International Conference on Computer and Communications (ICCC). IEEE (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Roy, C., Barua, K., Agarwal, S., Pandey, M., Rautaray, S.S. (2019). Horizontal Scaling Enhancement for Optimized Big Data Processing. In: Abraham, A., Dutta, P., Mandal, J., Bhattacharya, A., Dutta, S. (eds) Emerging Technologies in Data Mining and Information Security. Advances in Intelligent Systems and Computing, vol 755. Springer, Singapore. https://doi.org/10.1007/978-981-13-1951-8_58
Download citation
DOI: https://doi.org/10.1007/978-981-13-1951-8_58
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-1950-1
Online ISBN: 978-981-13-1951-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)