Skip to main content
SpringerLink
Log in

Queue-Waiting-Time Based Load Balancing Algorithm for Fine-Grain Microservices

  • Conference paper
  • First Online:
Services Computing – SCC 2018 (SCC 2018)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10969))

Included in the following conference series:

Abstract

For fine-grain microservices, queue-waiting-time is defined as load index of a server for the first time. However, as internet traffic is bursty, queue-waiting-time is not merely calculated by adding up the normal service time of queued requests due to resource contention. Moreover, normal service time changes over time especially for database-driven web applications. Therefore, an adaptive load balancing algorithm is required. This paper focuses on load balancing algorithms under differentiated requests and heterogeneous servers. In order to solve the tuning problem in load balancing, an online learning algorithm of time-weight (OLTW) is designed, which can learn the time-weight of request adaptively. Based on OLTW, a shortest queue-waiting-time load balancing algorithm (SQLB) is then proposed, in this algorithm, an incoming request is dispatched to the server with shortest queue-waiting-time. The experimental results show that 80% prediction values of OLTW have relative error of less than 25%, and SQLB outperforms the classical load balancing algorithms in terms of throughput, mean response time and deadline drop rate.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Lewis, J., Fowler, M.: Microservices (2014). http://martinfowler.com/articles/microservices.html

  2. Jenkins, J., Shipman, G.: A case study in computational caching microservices for HPC. In: IEEE Parallel and Distributed Processing Symposium Workshops, pp. 1309–1316 (2017)

    Google Scholar 

  3. Taibi, D., Lenarduzzi, V.: Processes, motivations, and issues for migrating to microservices architectures: an empirical investigation. IEEE Cloud Comput. 4(5), 22–32 (2017)

    Article  Google Scholar 

  4. Ask Jeeves. http://www.ask.com

  5. Teoma search. http://www.teoma.com

  6. Zhou, J., Zhang, C.: Request-aware scheduling for busy internet services. In: IEEE International Conference on Computer Communications, pp. 1–11 (2006)

    Google Scholar 

  7. Gilly, K., Juiz, C.: An up-to-date survey in web load balancing. World Wide Web-Internet Web Inf. Syst. 14(2), 105–131 (2011)

    Article  Google Scholar 

  8. Shen, K., Yang, T.: Cluster load balancing for fine-grain network services. In: IEEE Parallel and Distributed Processing Symposium (2002)

    Google Scholar 

  9. Zhang, W., Wang, H., Yu, B.: A request distribution algorithm for web server cluster. J. Netw. 6, 1760–1766 (2011)

    Google Scholar 

  10. Casalicchio, E., Tucci, S.: Static and dynamic scheduling algorithms for scalable web server farm. In: IEEE Parallel and Distributed Processing, pp. 369–376 (2001)

    Google Scholar 

  11. Zhang, L., Li, X.P.: A Content-based dynamic load-balancing algorithm for heterogeneous web server cluster. Comput. Sci. Inf. Syst. 7, 153–162 (2010)

    Article  Google Scholar 

  12. Sharifian, S., Motamedi, S.A.: An approximation-based load-balancing algorithm with admission control for cluster web servers with dynamic workloads. J. Supercomput. 53, 440–463 (2010)

    Article  Google Scholar 

  13. Singh, H., Kumar, S.: WSQ: web server queueing algorithm for dynamic load balancing. Wirel. Pers. Commun. 80, 229–245 (2015)

    Article  Google Scholar 

  14. Acme Air sample and benchmark. https://github.com/acmeair/acmeair

  15. Ueda, T., Nakaike, T., Ohara, M.: Workload characterization for microservices. In: IEEE International Symposium on Workload Characterization, pp. 1–10 (2016)

    Google Scholar 

  16. Ogasawara, T.: Workload characterization of server-side JavaScript. In: IEEE International Symposium on Workload Characterization, pp. 13–21 (2014)

    Google Scholar 

  17. Park, G., et al.: Adaptive load balancing mechanism for server cluster. In: Gavrilova, M.L., et al. (eds.) ICCSA 2006. LNCS, vol. 3983, pp. 549–557. Springer, Heidelberg (2006). https://doi.org/10.1007/11751632_60

    Chapter  Google Scholar 

  18. Awada U.: Improving resource efficiency of container-instance clusters on clouds. In: IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 929–934 (2017)

    Google Scholar 

  19. Paxson, V.: Fast approximation of self-similar network traffic. ACM SIGCOMM Comput. Commun. Rev. 27, 5–18 (1997)

    Article  Google Scholar 

  20. Lee, J.Y., Kim, S.: Bandwidth optimization for internet traffic in generalized processor sharing servers. IEEE Trans. Parallel Distrib. Syst. 16(4), 324–334 (2005)

    Article  Google Scholar 

  21. Jin, X.: Performance analysis of priority scheduling mechanisms under heterogeneous network traffic. J. Comput. Syst. Sci. 73(8), 1207–1220 (2007)

    Article  MathSciNet  Google Scholar 

  22. Silva, R.A.C.D., Fonseca, N.L.S.D.: Topology-aware virtual machine placement in data centers. J. Grid Comput. 14, 75–90 (2016)

    Article  Google Scholar 

  23. Tars, Tencent corporation (2017). https://github.com/Tencent/Tars

Download references

Author information

Authors and Affiliations

  1. University of Chinese Academy of Sciences, Beijing, 100049, China

    XiaoDong Liu & YongHao Song

  2. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China

    XiaoDong Liu, Yan Jin, YongHao Song & XiaoFang Zhao

Authors
  1. XiaoDong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. YongHao Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. XiaoFang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to XiaoDong Liu .

Editor information

Editors and Affiliations

  1. University of São Paulo, São Paulo, Brazil

    João Eduardo Ferreira

  2. City, University of London, London, United Kingdom

    George Spanoudakis

  3. Wuhan University, Wuhan, China

    Yutao Ma

  4. Kingdee International Software Group Co., Shenzhen, China

    Liang-Jie Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, X., Jin, Y., Song, Y., Zhao, X. (2018). Queue-Waiting-Time Based Load Balancing Algorithm for Fine-Grain Microservices. In: Ferreira, J., Spanoudakis, G., Ma, Y., Zhang, LJ. (eds) Services Computing – SCC 2018. SCC 2018. Lecture Notes in Computer Science(), vol 10969. Springer, Cham. https://doi.org/10.1007/978-3-319-94376-3_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-94376-3_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-94375-6

  • Online ISBN: 978-3-319-94376-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation