Abstract
Cloud MapReduce, as an implementation of MapReduce framework on Cloud for big data analysis, is facing the unknown job makespan and long wait time problem, which have seriously affected the service quality. The Inefficient virtual machine allocation is one critical causing factor. Based on the M/M/1 model, a new queuing equation is built to ensure the virtual machine with the high efficiency. By jointing queuing equation and objectives function, a two variables equation group is designed to compute the desired virtual machine number for different jobs. According to the desired virtual machine number of each job, we developed a queuing-oriented job optimizing scheduling algorithm, called QTJS, to optimal job scheduling and enhance the resource utilization in Cloud MapReduce. Extensive experiments show that our QTJS algorithm consumes less job execution time and performs better efficiency than other three algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
J. Dean and S. Ghemawat, “MapReduce: Simplified data processing on large clusters,” in Proc. 6th Symp. Oper. Syst. Design Implementation (OSDI), Dec. 2004, pp. 137–150.
Thabet M, Boufaida M, Kordon F. An approach for developing an interoperability mechanism between cloud providers [J]. International Journal of Space-Based and Situated Computing 27, 2014, 4(2): 88-99.
Afrati F N, Ullman J D. Optimizing joins in a map-reduce environment[C]//Proceedings of the 13th International Conference on Extending Database Technology. ACM, 2010: 99-110.
Verma A, Cherkasova L, Campbell R H. Orchestrating an ensemble of MapReduce jobs for minimizing their makespan[J]. Dependable and Secure Computing, IEEE Transactions on, 2013, 10(5): 314-327.
Xilong Q, Peng X. An energy-efficient virtual machine scheduler based on CPU sharereclaiming policy [J]. International Journal of Grid and Utility Computing, 2015, 6(2): 113-120.
Hacker T J, Romero F, Nielsen J J. Secure live migration of parallel applications using container-based virtual machines[J]. International Journal of Space-Based and Situated Computing 1, 2012, 2(1): 45-57.
Hashem I A T, Yaqoob I, Anuar N B, et al. The rise of “big data” on cloud computing: review and open research issues [J]. Information Systems, 2015, 47: 98-115.
Manoharan M, Selvarajan S. An efficient methodology to improve service negotiation in cloud environment [J]. International Journal of Grid and Utility Computing, 2015, 6(3-4): 150-158.
Dahiphale D, Karve R, Vasilakos A V, et al. An advanced MapReduce: cloud MapReduce, enhancements and applications [J]. Network and Service Management, IEEE Transactions on, 2014, 11(1): 101-115.
Liu H, Orban D. Cloud MapReduce : a MapReduce implementation on top of a cloud operating system[C]//Proceedings of the 2011 11th IEEE/ACM international symposium on cluster, cloud and grid computing. IEEE Computer Society, 2011: 464-474.
Verma A, Kaushal S. Deadline constraint heuristic-based genetic algorithm for workflow scheduling in cloud [J]. International Journal of Grid and Utility Computing, 2014, 5(2): 96-106.
Verma A, Cho B, Zea N, et al. Breaking the MapReduce stage barrier [J]. Cluster computing, 2013, 16(1): 191-206.
Saez J C, Pousa A, Castro F, et al. ACFS: a completely fair scheduler for asymmetric single-isa multicore systems[C]//Proceedings of the 30th Annual ACM Symposium on Applied Computing. ACM, 2015: 2027-2032.
Shengjun X, Delong W, Suhong S. PCSP: A Preemptive Capacity Scheduler Policy for Scheduling Hadoop Jobs[J]. International Journal of Grid and Distributed Computing, 2015, 8(5): 33-46.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
He, TQ., Cai, LJ., Deng, ZY., Meng, T., Wang, X. (2017). Queuing-Oriented Job Optimizing Scheduling In Cloud Mapreduce. In: Xhafa, F., Barolli, L., Amato, F. (eds) Advances on P2P, Parallel, Grid, Cloud and Internet Computing. 3PGCIC 2016. Lecture Notes on Data Engineering and Communications Technologies, vol 1. Springer, Cham. https://doi.org/10.1007/978-3-319-49109-7_41
Download citation
DOI: https://doi.org/10.1007/978-3-319-49109-7_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-49108-0
Online ISBN: 978-3-319-49109-7
eBook Packages: EngineeringEngineering (R0)