Abstract
The replication system attracts many researchers to rethink of the tactics of data placement in the various devices, namely RAM and HDD/SSD. The advantages of a replication system are (a) parallelism, (b) data availability, (c) fault tolerance, (d) data recovery, and (e) failover. However, the replication system poses some overheads, namely, communication and synchronization cost. In this paper, we show an in-memory data replication system using the Dr. Hadoop framework. It adapts Dr. Hadoop framework for in-memory replication system. This, in turn, provides very high availability, scalability, and fault-tolerant nature for Dr. Hadoop metadata server. The paper delivers various theorems that are presented in different sections of the paper and are proved using theoretical analysis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
D. Dev, R. Patgiri, Performance evaluation of HDFs in big data management, in 2014 International Conference on High Performance Computing and Applications (ICHPCA) (2014), pp. 1–7
D. Dev, R. Patgiri, HAR+: archive and metadata distribution! why not both? in 2015 International Conference on Computer Communication and Informatics (ICCCI) (2015), pp. 1–6
D. Dev, R. Patgiri, Dr. Hadoop: an infinite scalable metadata management for Hadoop—how the baby elephant becomes immortal. Front. Inf. Technol. Electron. Eng. 17(1), 15–31 (2016)
D. Dev, R. Patgiri, A survey of different technologies and recent challenges of big data, in Proceedings of 3rd International Conference on Advanced Computing, Networking and Informatics, ed. by A. Nagar, D.P. Mohapatra, N. Chaki (Springer, New Delhi, India, 2016), pp. 537–548
D. Dev, R. Patgiri, A survey of different technologies and recent challenges of big data, in Proceedings of 3rd International Conference on Advanced Computing, Networking and Informatics (2015), pp. 537–548
B. Fitzpatrick, Distributed caching with memcached. Linux J. 2004(124), 5 (2004)
A. Kemper, T. Neumann, Hyper: a hybrid OLTP & OLAP main memory database system based on virtual memory snapshots, in 2011 IEEE 27th International Conference on Data Engineering (2011), pp. 195–206
R. Nishtala, H. Fugal, S. Grimm, M. Kwiatkowski, H. Lee, H.C. Li, R. McElroy, M. Paleczny, D. Peek, P. Saab et al., Scaling memcache at facebook. in NSDI, vol. 13 (2013), pp. 385–398
R. Patgiri, Mds: in-depth insight, in 2016 International Conference on Information Technology (ICIT) (2016), pp. 193–199
R. Patgiri, Issues and challenges in big data: a survey, in Distributed Computing and Internet Technology, ed. by A. Negi, R. Bhatnagar, L. Parida (Springer, Cham, 2018), pp. 295–300
R. Patgiri, A. Ahmed, Big data: the v’s of the game changer paradigm, in 2016 IEEE 18th International Conference on High Performance Computing and Communications; IEEE 14th International Conference on Smart City; IEEE 2nd International Conference on Data Science and Systems (HPCC/SmartCity/DSS) (2016), pp. 17–24
R. Patgiri, D. Dev, A. Ahmed, dMDS: uncover the hidden issues of metadata server design, in Progress in Intelligent Computing Techniques: Theory, Practice, and Applications, ed. by P.K. Sa, M.N. Sahoo, M. Murugappan, Y. Wu, B. Majhi (Springer, Singapore, 2018), pp. 531–541
M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M.J. Franklin, S. Shenker, I. Stoica, Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing, in Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation (USENIX Association, Berkeley, CA, USA, 2012), pp. 2–2
M. Zaharia, M. Chowdhury, M.J. Franklin, S. Shenker, I. Stoica, Spark: cluster computing with working sets. HotCloud 10(10–10), 95 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Patgiri, R., Nayak, S., Dev, D., Borgohain, S.K. (2020). Dr. Hadoop Cures In-Memory Data Replication System. In: Elçi, A., Sa, P., Modi, C., Olague, G., Sahoo, M., Bakshi, S. (eds) Smart Computing Paradigms: New Progresses and Challenges. Advances in Intelligent Systems and Computing, vol 767. Springer, Singapore. https://doi.org/10.1007/978-981-13-9680-9_19
Download citation
DOI: https://doi.org/10.1007/978-981-13-9680-9_19
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-9679-3
Online ISBN: 978-981-13-9680-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)