Abstract
The Compute Aggregate model used to model Map Reduce does not allow for dynamic node reordering once a job has started, assumes homogenous nodes and a balanced tree layout. We introduce heterogeneous nodes into the tree structure, thereby causing unbalanced trees. Finally, we present a new programming abstraction to allow for dynamic tree balancing.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Communications of the ACM 51, 107–113 (2008)
Konstantin, S., Hairong, K., Sanjay, R., Robert, C.: The Hadoop distributed file system. In: Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST) (2010)
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., et al.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, San Jose, CA (2012)
Culhane, W., Kogan, K., Jayalath, C., Eugster, P.: Optimal communication structures for big data aggregation. In: 2015 IEEE Conference on Computer Communications (INFOCOM), pp. 1643–1651 (2015)
Cheng, Y.C., Robertazzi, T.G.: Distributed computation for a tree network with communication delays. IEEE Transactions on Aerospace and Electronic Systems 26, 511–516 (1990)
Hyoung Joong, K., Gyu-in, J., Jang Gyu, L.: Optimal load distribution for tree network processors. IEEE Transactions on Aerospace and Electronic Systems 32, 607–612 (1996)
Morozov, D., Weber, G.: Distributed merge trees. SIGPLAN Not. 48, 93–102 (2013)
Acknowledgements
This work is supported by the Ministry of Science, Technology and Innovation Malaysia [Grant No.: FP067-2015A].
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sheng, C.Y., Keong, P.K. (2017). Beyond Map-Reduce: LATNODE – A New Programming Paradigm for Big Data Systems. In: Kim, K., Joukov, N. (eds) Information Science and Applications 2017. ICISA 2017. Lecture Notes in Electrical Engineering, vol 424. Springer, Singapore. https://doi.org/10.1007/978-981-10-4154-9_9
Download citation
DOI: https://doi.org/10.1007/978-981-10-4154-9_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-4153-2
Online ISBN: 978-981-10-4154-9
eBook Packages: EngineeringEngineering (R0)