Skip to main content
SpringerLink
Log in

Enhancing the power of two choices load balancing algorithm using round robin policy

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

This paper proposes a new version of the power of two choices, SQ(d), load balancing algorithm. This new algorithm improves the performance of the classical model based on the power of two choices randomized load balancing. This model considers jobs that arrive at a dispatcher as a Poisson stream of rate \(\lambda n,\) \(\lambda < 1,\) at a set of n servers. Using the power of two choices, the dispatcher chooses some d constant for each job independently and uniformly from the n servers in a random way and sends the job to the server with the fewest number of jobs. This algorithm offers an advantage over the load balancing based on shortest queue discipline, because it provides good performance and reduces the overhead in the servers and the communication network. In this paper, we propose a new version, shortest queue of d with randomization and round robin policies, SQ-RR(d). This new algorithm combines randomization techniques and static local balancing based on a round-robin policy. In this new version, the dispatcher chooses the d servers as follows: one is selected using a round-robin policy, and the \(d-1\) servers are chosen independently and uniformly from the n servers in a random way. Then, the dispatcher sends the job to the server with the fewest number of jobs. We demonstrate with a theoretical approximation of this approach that this new version improves the performance obtained with the classical solution in all situations, including systems at 99% capacity. Furthermore, we provide simulations that demonstrate the theoretical approximation developed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Belardi, R., Mattia, G.P.: Power of random choices made efficient for fog computing. IEEE Trans. Cloud Comput. Early Access (2020). https://doi.org/10.1109/TCC.2020.2968443

    Article  Google Scholar 

  2. Bramson, M., Lu, Y., Prabhakar, B.: Randomized load balancing with general service time distributions. In: SIGMETRICS’10, 14–18 June, New York, USA (2010)

  3. Bramson, M., Lu, Y., Prabhakar, B.: Asymptotic independence of queues under randomized load balancing. Queueing Syst. 71(3), 247–292 (2012)

    Article  MathSciNet  Google Scholar 

  4. Bramson, M., Lu, Y., Prabhakar, B.: Decay of tails at equilibrium for FIFO join the shortest queue networks. Ann. Appl. Probab. 23(5), 1841–1878 (2013)

    Article  MathSciNet  Google Scholar 

  5. Breitgand, D., Cohen, R., Nahir, A., Raz, D.: On cost-aware monitoring for self-adaptive load sharing. IEEE J. Sel. Areas Commun. 28(1), 70–83 (2010)

    Article  Google Scholar 

  6. Casanova, H., Giersch, A., Legrand, A., Quison, M., Sutero, F.: Higher order approximations for the single server queue with splitting, merging and feedback. J. Parallel Distrib. Comput. 10(74), 2899–2917 (2014)

    Article  Google Scholar 

  7. Chen, Y., Sun, M.: TSSA: A two step scheduling algorithm for the event-driven clusters. In: 2018 20th International Conference on Advanced Communication Technology (ICACT), IEEE, pp. 184–189 (2018)

  8. Chen, H., Ye, H.Q.: Asymptotic optimality of balanced routing. Oper. Res. 60(1), 163–179 (2012)

    Article  MathSciNet  Google Scholar 

  9. Crovella, M.E., Taqqu, M.S., Bestavros, A.: Heavy-tailed probability distributions in the World Wide Web. In: A Practical Guide to Heavy Tails, vol. 1, pp. 3–26. Birkhaüser, Boston (1998)

  10. Da Costa, G., Grange, L., De Courchelle, I.: Modeling and generating large-scale Google-like Workload. In: International Workshop on Resilience and/or Energy-Aware Techniques for High-Performance Computing, November 2016, Hangzhou, China (2016)

  11. Dai, D., Chen, Y., Kimpe, D., Ross, R.: Two-choice randomized dynamic I/O scheduler for object storage systems. In: SC’14 Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 635–646 (2014)

  12. Eager, L.D., Lazowska, E.D., Zahorjan, J.: Adaptive load sharing in homogeneous distributed systems. IEEE Trans. Softw. Eng. SE–12(5), 662–675 (1996)

    Article  Google Scholar 

  13. Ephremides, A., Varaiya, P., Walrand, J.: A simple dynamic routing problem. IEEE Trans. Autom. Control 25(4), 690–693 (1980)

    Article  MathSciNet  Google Scholar 

  14. Fanti, G., Jiao, J., Makkuva, A., Oh, S., Rana, R., Viswanath, P.: Barracuda: the power of l-polling in proof-of-stake blockchains. In: Proceedings of the Twentieth ACM International Symposium on Mobile Ad Hoc Networking and Computing, pp. 351–360 (2019)

  15. Girish, M.K., Hu, J.-Q.: Higher order approximations for the single server queue with splitting, merging and feedback. Eur. J. Oper. Res. 124, 447–467 (2000)

    Article  MathSciNet  Google Scholar 

  16. Gupta, V., Harchol-Balter, M., Sigman, K., Whitt, W.: Analysis of join-the-shortest-queue routing for web server farms. Perform. Eval. 64(9–12), 1062–1081 (2007)

    Article  Google Scholar 

  17. Hellemans, T., Van Houdt, B.: On the power-of-d-choices with least loaded server selection. In: Proceedings of the ACM on Measurement and Analysis of Computing Systems, vol. 2(2), pp. 1–22 (2018)

  18. Hordijk, A., Koole, G.: On the optimality of the generalized shortest queue policy. Probab. Eng. Inf. Sci. 4(04), 477–487 (1990)

    Article  Google Scholar 

  19. Hyytia, E., Aalto, S.: Round-robin routing policy. In: ValueTools’13, 10–12 December 2013, Turin, Italy

  20. Izagirre, A., Makowskid, A.M.: Light traffic performance under the power-of-two load balancing strategy: the case of server heterogeneity. In: Conference IFIP Performance 2014, Turin, Italy

  21. Johnson, M.A.: Selecting parameters of phase distributions: combining nonlinear programming, heuristics and Erlang distributions. ORSA J. Comput. 5(1), 69–83 (1993)

    Article  Google Scholar 

  22. Kingman, J.F.C.: The single server queue in heavy traffic. Math. Proc. Camb. Philos. Soc. 57(4), 902–904 (1961)

    Article  MathSciNet  Google Scholar 

  23. Liu, A., Bai, Z., Liu, Z., Li, X., Kim, C., Braverman, V., Jin, X., Stoica, I.: Distcache: provable load balancing for large-scale storage systems with distributed caching. In: 17th $\{$USENIX$\}$ Conference on File and Storage Technologies ($\{$FAST$\}$ 19), pp. 143–157 (2019)

  24. Lu, Y., Xie, Q., Kliot, G., Geller, A., Larus, J.R., Greenberg, A.: Join-Idle-Queue: a novel load balancing algorithm for dynamically scalable web services. Perform. Eval. 68(11), 1056–1071 (2011)

    Article  Google Scholar 

  25. Luczak, J.M., Norris, J.: Strong approximation for the supermarket model. Ann. Appl. Probab. 15(3), 2038–2061 (2005)

    MathSciNet  MATH  Google Scholar 

  26. Lumetta, S., Mitzenmacher, M.: Using the power of two choices to improve bloom filters. Internet Math. 4(1), 17–33 (2009)

    Article  MathSciNet  Google Scholar 

  27. Matsumoto, M., Nishimura, T.: Mersenne Twister: a 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Trans. Model. Comput. Simul. 8, 3–30 (1998)

    Article  Google Scholar 

  28. Mishra, A.K., Hellerstein, J.L., Crime, W., Das, C.R.: Towards characterizing cloud backend workloads: insights from Google compute clusters. ACM SIGMETRICS Perform. Eval. Rev. 37(4), 34–41 (2009)

    Article  Google Scholar 

  29. Mitzenmacher, M.: The power of two choices in randomized load balancing. IEEE Trans. Parallel Distrib. Syst. 12(10), 1094–1104 (2001)

    Article  Google Scholar 

  30. Mitzenmacher, M., Richa, A., Sitaraman, R.: The power of two random choices: a survey of techniques and results. In: Pardalos, P., Rajasekaran, S., Rolim, J. (eds.) Handbook of Randomized Computing, vol. 1, pp. 255–312. Springer, New York (2001)

    Chapter  Google Scholar 

  31. Nasir, M.A.U., Morales, G.F., Garcia-Soriano, D., Kourtellis, N.: The power of both choices: practical load balancing for distributed stream processing engines. In: Proceedings of the 31st International Conference on Data Engineering, ICDE2015, April 2015, Korea

  32. Quian, A., Gao, Y., Ji, M., Peng, H., Jin, Y., Lu, S.: Scheduling, workload-aware, for data analytics upon heterogeneous storage. In: 2009 IEEE International Conference on Parallel and Distributed Processing with Applications, Big Data and Cloud Computing, Sustainable Computing and Communications, Social Computing and Networking (ISPA/BDCloud/SocialCom/SustainCom), pp. 580–587. IEEE (2019)

  33. Reiss, C., Wilkes, J., Joseph, L.: Google Cluster Usage: Format + Schema, Technical Report. Google, Inc., Mountain View (2011). Revised 2012. http://code.google.com/p/googleclusterdata/wiki/TraceVersion2

  34. Schroeder, B., Harchol-Balter, M.: Evaluation of task assignment policies for supercomputing servers: the case for load unbalancing and fairness. Clust. Comput. 7, 151–161 (2004)

    Article  Google Scholar 

  35. Sosa-Sosa, V., Barron, A., Gonzalez, J.L., Carretero, J.: Improving performance and capacity utilization in cloud storage for content delivery and sharing services. IEEE Trans. Cloud Comput. (2020). https://doi.org/10.1109/TCC.2020.2968444

    Article  Google Scholar 

  36. Stavrinides, G.L., Karatza, H.D.: Task group scheduling in distributed systems. In: 2018 International Conference on Computer, Information and Telecommunication Systems (CITS), pp. 1–5. IEEE

  37. The Spark Project, Lightning-Fast Cluster Computing. http://spark.apache.org (2019)

  38. Weber, R.W.: On optimal assignment of customers to parallel servers. J. Appl. Probab. 15, 406–413 (1978)

    Article  MathSciNet  Google Scholar 

  39. Venkataraman, S., Panda, A., Ananthanarayanan, G., Franklin, M.J., Sotica, I.: The power of choice in data-aware cluster scheduling. In: Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation, pp. 301–316. USENIX Association (2014)

  40. Vvedenskaya, N.D., Dobrushin, R.L., Karpelevich, F.I.: Queueing system with selection of the shortest of two queues: an asymptotic approach. Probl. Pereda. Inf. 32(1), 20–34 (1996)

    MATH  Google Scholar 

  41. Winston, W.: Optimality of the shortest line discipline. J. Appl. Probab. 14, 181–189 (1977)

    Article  MathSciNet  Google Scholar 

  42. Wu, S., Chen, H., Jin, H., Ibrahim, S.: Shadow: exploiting the power of choice for efficient shuffling in MapReduce. IEEE Trans. Big Data (2019). https://doi.org/10.1109/TBDATA.2019.2943473

    Article  Google Scholar 

  43. Xu, J., Hajek, B.: The supermarket fame. In: 2012 IEEE International Symposium on Information Theory, pp. 2511–2515 (2012)

Download references

Acknowledgements

This work was partially supported by the Project “CABAHLA-CM: Convergencia Big data-Hpc: de los sensores a las Aplicaciones” S2018/TCS-4423 from Madrid Regional Government.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Universidad Carlos III de Madrid, Leganes, Madrid, Spain

    Felix Garcia-Carballeira, Alejandro Calderon & Jesus Carretero

Authors
  1. Felix Garcia-Carballeira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alejandro Calderon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jesus Carretero
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Felix Garcia-Carballeira.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Garcia-Carballeira, F., Calderon, A. & Carretero, J. Enhancing the power of two choices load balancing algorithm using round robin policy. Cluster Comput 24, 611–624 (2021). https://doi.org/10.1007/s10586-020-03139-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-020-03139-6

Keywords

Search

Navigation