Advertisement

MapReduce machine covering problem on a small number of machines

  • Yiwei JiangEmail author
  • Ping Zhou
  • Wei Zhou
Article

Abstract

We study machine covering problem in MapReduce system. Each job consists of two sets of tasks, namely the map tasks and reduce tasks. A job’s reduce tasks can only be processed after all its map tasks are finished. The map tasks are fractional, i.e., they can be arbitrarily split and processed on different machines in parallel. Our goal is to maximize the minimum machine completion time. We consider two variants of the problem, namely the cases involving preemptive reduce tasks and non-preemptive reduce tasks. For preemptive reduce tasks, we present optimal solution algorithms for the problem on two and three machines. For non-preemptive reduce tasks, we provide an approximation algorithm with a tight worse-case ratio of \(\frac{4}{3}\) for the problem on two machines.

Keywords

MapReduce Machine covering Algorithm 

Notes

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China 11571013.

References

  1. Chang H, Kodialam M, Kompella RR et al (2015) Scheduling in MapReduce-like systems for fast completion time. Proce IEEE INFOCOM 2(3):3074–3082Google Scholar
  2. Chen C, Xu Y, Zhu Y, Sun C (2017) Online MapReduce scheduling problem of minimizing the makespan. J Comb Optim 33:590–608MathSciNetCrossRefzbMATHGoogle Scholar
  3. Csirik J, Kellerer H, Woeginger G (1992) The exact LPT-bound for maximizing the minimum completion time. Oper Res Lett 11:281–287MathSciNetCrossRefzbMATHGoogle Scholar
  4. Dean J, Ghemawat S (2004) MapReduce: simplified data processing on large clusters. Proc Oper Syst Des Implement (OSDI) 51(1):107–113Google Scholar
  5. Deuermeyer B, Friesen D, Langston M (1982) Scheduling to maximize the minimum processor finish time in a multiprocessor system. SIAM J Discrete Methods 3:190–196MathSciNetCrossRefzbMATHGoogle Scholar
  6. Jiang Y, Tan Z, He Y (2005) Preemptive machine covering on parallel machines. J Comb Optim 10(4):345–363MathSciNetCrossRefzbMATHGoogle Scholar
  7. Jiang Y, Zhu Y, Wu W, Li D (2017) Makespan minimization for MapReduce systems with different servers. Future Gener Comput Syst 67:13–21CrossRefGoogle Scholar
  8. Jiang Y, Zhou W, Zhou P (2018) An optimal preemptive algorithm for online MapReduce scheduling on two parallel machines. Asia Pac J Oper Res 35(3):185003MathSciNetCrossRefzbMATHGoogle Scholar
  9. Le Y, Liu J, Ergun F, Wang D (2014) Online load balancing for MapReduce with skewed data input. In: INFOCOM’14, 2004–2012. IEEEGoogle Scholar
  10. Luo T, Zhu Y, Wu W, Xu Y, Du D (2017) Online makespan minimization in MapReduce-like systems with complex reduce tasks. Optim Lett 11:271–277MathSciNetCrossRefzbMATHGoogle Scholar
  11. Woeginger G (1997) A polynomial time approximation scheme for maximizing the minimum machine completion time. Oper Res Lett 20:149–154MathSciNetCrossRefzbMATHGoogle Scholar
  12. Zheng Y, Shroff NB, Sinha P (2013) A new analytical technique for designing provably efficient MapReduce schedulers. In: Proceedings of IEEE, INFOCOM, 2013. IEEE, pp 1600–1608Google Scholar
  13. Zhu Y, Jiang Y, Ding L, Teredesai A, Li D, Lee W (2014) Minimizing makespan and total completion time in MapReduce-like systems. In: INFOCOM 14. IEEE, pp 2166–2174Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Management and E-Business, Contemporary Business and Trade Research CenterZhejiang Gongshang UniversityHangzhouChina
  2. 2.College of HumanitiesZhejiang Business CollegeHangzhouChina
  3. 3.Department of MathematicsZhejiang Sci-Tech UniversityHangzhouChina

Personalised recommendations