Skip to main content

A load balancing and optimization strategy (LBOS) using reinforcement learning in fog computing environment

Abstract

Fog computing (FC) can be considered as a computing paradigm which performs Internet of Things (IoT) applications at the edge of the network. Recently, there is a great growth of data requests and FC which lead to enhance data accessibility and adaptability. However, FC has been exposed to many challenges as load balancing (LB) and adaptation to failure. Many LB strategies have been proposed in cloud computing, but they are still not applied effectively in fog. LB is an important issue to achieve high resource utilization, avoid bottlenecks, avoid overload and low load, and reduce response time. In this paper, a LB and optimization strategy (LBOS) using dynamic resource allocation method based on Reinforcement learning and genetic algorithm is proposed. LBOS monitors the traffic in the network continuously, collects the information about each server load, handles the incoming requests, and distributes them between the available servers equally using dynamic resource allocation method. Hence, it enhances the performance even when it’s the peak time. Accordingly, LBOS is simple and efficient in real-time systems in fog computing such as in the case of healthcare system. LBOS is concerned with designing an IoT-Fog based healthcare system. The proposed IoT-Fog system consists of three layers, namely: (1) IoT layer, (2) fog layer, and (3) cloud layer. Finally, the experiments are carried out and the results show that the proposed solution improves the quality-of-service in the cloud/fog computing environment in terms of the allocation cost and reduce the response time. Comparing the LBOS with the state-of-the-art algorithms, it achieved the best load balancing Level (85.71%). Hence, LBOS is an efficient way to establish the resource utilization and ensure the continuous service.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Alam M, Khan ZA (2017) Issues and challenges of load balancing algorithm in cloud computing environment. Indian J Sci Technol. https://doi.org/10.17485/ijst/2017/v10i25/105688

    Article  Google Scholar 

  2. Aqueveque P, Gutierrez C, Saavedra F, Pino EJ, Morales A, Wiechmann E (2017) Monitoring physio-logical variables of mining workers at high altitude. IEEE Trans Ind Appl 53:2628–2634. https://doi.org/10.1109/TIA.2017.2675360

    Article  Google Scholar 

  3. Arani M, Souri A, Rahmanian A (2019) Resource management approaches in fog computing: a comprehensive review. J Grid Comput. https://doi.org/10.1007/s10723-019-09491-1

    Article  Google Scholar 

  4. Atul G (2014) A comparative study of static and dynamic load balancing algorithms. IJARCSMS 2(12):386–392

    Google Scholar 

  5. Baek J, Kaddoum G, Garg S, Kaur K, Gravel V (2019) Managing fog networks using reinforcement learning based load balancing algorithm. IEEE Wirel Commun Netw Conf. https://doi.org/10.1109/WCNC.2019.8885745

    Article  Google Scholar 

  6. Bahati RM, Bauer MA (2010) Towards adaptive policy-based management. IEEE Netw Oper Manag Sym NOMS 2010:511–518. https://doi.org/10.1109/NOMS.2010.5488472

    Article  Google Scholar 

  7. Barrett E, Howley E, Duggan J (2013) Applying reinforcement learning towards automating resource allocation and application scalability in the cloud. Concur Comput 25(12):1656–1674. https://doi.org/10.1002/cpe.2864

    Article  Google Scholar 

  8. Beloglazov A, Rajkumar B (2012) Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers. Concurrency and computation: practice and experience. Concur Computat 24(13):1397–1420. https://doi.org/10.1002/cpe.1867

    Article  Google Scholar 

  9. Dubey Sh, Dahiya M, Jain S (2019) Implementation of load balancing algorithm with cloud collaboration for logistics. J Eng Appl Sci 14(2):507–515. https://doi.org/10.36478/jeasci.2019.507.515

    Article  Google Scholar 

  10. Duggan M, Flesk K, Duggan J, Howley E, Barrett E (2016) A reinforcement learning approach for dynamic selection of virtual machines in cloud data centres. 2016 Sixth International Conference on Innovative Computing Technology (INTECH): 92−97. https://doi.org/10.1109/intech.2016.7845053

  11. Dutreilh X, Kirgizov S, Melekhova O, Malenfant J, Rivierre N, Truck I (2011) Using reinforcement learning for autonomic resource allocation in clouds: towards a fully automated workflow. In ICAS 2011, The seventh international conference on autonomic and autonomous systems: 67-74

  12. Fan Q, Ansari N (2018) Towards workload balancing in fog computing empowered IoT. IEEE Trans Netw Sci Eng. https://doi.org/10.1109/TNSE.2018.2852762

    Article  Google Scholar 

  13. Farahnakian F, Liljeberg P, Plosila J (2014) Energy-efficient virtual machines consolidation in cloud data centers using reinforcement learning. In parallel, distributed and network-based processing (PDP), 2014 22nd Euromicro International Conference on, IEEE: 500-507. https://doi.org/10.1109/PDP.2014.109

  14. Ghomia EJ (2017) Load-balancing algorithms in cloud computing: a survey. J Netw Comput Appl 88:50–71. https://doi.org/10.1016/j.jnca.2017.04.007

    Article  Google Scholar 

  15. Gubbi SV, Amrutur B (2015) Adaptive pulse width control and sampling for low power pulse oximetry. IEEE Trans Biomed Circuits Syst 9(2):272–283. https://doi.org/10.1109/TBCAS.2014.2326712

    Article  Google Scholar 

  16. Gupta S, Rani S, Dixit A, Dev H (2019) Features exploration of distinct load balancing algorithms in cloud computing environment. Int J Adv Netw Appl 11(1):4177–4183

    Google Scholar 

  17. Heart Foundation (2017) High blood pressure statistics. [Online]. Available: https://www.heartfoundation.org.au/about-us/what-we-do/heart-disease-in-australia/high-blood-pressure-statistics

  18. Hussain Md, Beg MM (2019) Fog computing for internet of things (IoT)-aided smart grid architectures. Big Data Cogn Comput 3:12–23. https://doi.org/10.3390/bdcc3010008

    Article  Google Scholar 

  19. Islam SMR (2015) The internet of things for health care: a comprehensive survey. IEEE Access 3:678–708. https://doi.org/10.1109/ACCESS.2015.2437951

    Article  Google Scholar 

  20. James J, Verma B (2012) Efficient vm load balancing algorithm for a cloud computing environment. Int J Comput Sci Eng (IJCSE) 4(9):1658–1662

    Google Scholar 

  21. Jung G, Mong Sim K (2012) Location-aware dynamic resource allocation model for cloud computing environment. International conference on information and computer applications (ICICA 2012) IPCSIT. 24: 37−41

  22. Kaur R, Luthra P (2014) Load balancing in cloud system using max min and min min algorithm. international journal of computer applications (0975–8887) National conference on emerging trends in computer technology (NCETCT-2014): 31-34. https://archive.ics.uci.edu/ml/datasets/MHEALTH+Dataset

  23. Khan S, Parkinson S, Qin Y (2017) Fog computing security: a review of current applications and security solutions. J Cloud Comp 6:19. https://doi.org/10.1186/s13677-017-0090-3

    Article  Google Scholar 

  24. Kim KI, Ullah S, Verikoukis C (2019) Editorial on special issue on fog computing for healthcare. Peer-to-Peer Netw Appl 12:1214–1215. https://doi.org/10.1007/s12083-019-00790-8

    Article  Google Scholar 

  25. Kong L, Mapetu JPB, Chen Z (2019) Heuristic load balancing based zero imbalance mechanism in cloud computing. J Grid Comput. https://doi.org/10.1007/s10723-019-09486-y

    Article  Google Scholar 

  26. Lesser V, Irwin D, Zink M (2010) Automated negotiation with decommitment for dynamic resource allocation in cloud computing. Conference at University of Massachusetts, Amherst, USA. Proc. of 9th Int. Conf. on Autonomous Agents and MultiagentSystems (AAMAS 2010), van der Hoek, Kaminka, Lespérance, Luck andSen (eds.). International Foundation for Autonomous Agents andMultiagent Systems (www.ifaamas.org): 981−988

  27. Lim S P, Haron H (2013) Performance comparison of genetic algorithm, differential evolution and particle swarm optimization towards benchmark functions. 2013 IEEE conference on open systems (ICOS), Kuching: 41−46. https://doi.org/10.1109/icos.2013.6735045

  28. Matthew B (2019) Reinforcement learning, fast and slow. Trends Cogn Sci 23(5):408–422. https://doi.org/10.1016/j.tics.2019.02.006

    Article  Google Scholar 

  29. Meng F, Chen P (2019) Power allocation in multi-user cellular networks: deep reinforcement learning approaches. Comput Sci Netw Internet Arch. arXiv:1901.07159v1

  30. Milici S, Lorenzo J, Lazaro A, Villarino R, Girbau D (2016) Wireless breathing sensor based on wear-able modulated frequency selective surface. IEEE Sens J 99:1. https://doi.org/10.1109/JSEN.2016.2645766

    Article  Google Scholar 

  31. Milojicic DS, Douglis F, Paindaveine Y, Wheeler R (2000) Process migration. ACM Comput Surv 32(3):241–299. https://doi.org/10.1145/367701.367728

    Article  Google Scholar 

  32. Mousavi S (2017) Dynamic resource allocation in cloud computing. Acta Polytech Hung 14(4):83–104

    MathSciNet  Google Scholar 

  33. Narczyk P, Siwiec K, Pleskacz W A (2016) Precision human body temperature measurement based on ther-mistor sensor. 2016 IEEE 19th International sympo-sium on design and diagnostics of electronic circuits and systems (DDECS): 1−5. https://doi.org/10.1109/ddecs.2016.7482451

  34. Nassar A, Yilmaz Y (2019) Reinforcement learning-based resource allocation in fog RAN for IoT with heterogeneous latency requirements. Comput Sci Netw Internet Arch. arXiv:1806.04582v2

  35. Nguyen P, Cong H (2017) Load balancing algorithm to improve response time on cloud computing. Int J Cloud Comput Serv Archit (IJCCSA) 7(6):1–12. https://doi.org/10.5121/ijccsa.2017.7601

    Article  Google Scholar 

  36. Nogueira V, Carnaz G (2019) An overview of IoT and healthcare. http://hdl.handle.net/10174/19998

  37. Padilha L (2018) Analysis of the use of SDN for load balancing. Leonardo de Carvalho Freitas Padilha Aguilar. https://www.researchgate.net/publication/330668114_Analysis_of_the_use_of_SDN_for_load_balancing

  38. Pate G, Mehta R (2017) A survey on various task scheduling algorithm in cloud computing. Int J Adv Res Comput Eng Technol (IJARCET) 3(3):715–717

    Google Scholar 

  39. Patel R, Patel S (2013) Survey on resource allocation strategies in cloud computing. Int J Eng Res Technol (IJERT) 2(2):2278

    Google Scholar 

  40. Paulsingh S, Sandhya RA, Sahai R, Venugopal KR, Lalit P (2012) Comparative study on load balancing techniques in distributed systems. Int J Inf Technol Knowle Manag 6(1) 53:60

    Google Scholar 

  41. Pawar S, Wagh B (2012) Priority based dynamic resource allocation in cloud computing. Int Symp Cloud Serv Comput Mangalore. https://doi.org/10.1109/ISCOS.2012.14

    Article  Google Scholar 

  42. Prajapati R, Rathod D, Khanna S (2015) Comparison of static and dynamic load balancing in grid computing. Int J Technol Res Eng 2(7):2347–4718

    Google Scholar 

  43. Rodrigo da Rosa R (2019) A survey on global management view: toward combining system monitoring, resource management, and load prediction. J Grid Comput 17(3):473–502. https://doi.org/10.1007/s10723-018-09471-x

    Article  Google Scholar 

  44. Schulz P, Matthe M, Klessig H, Simsek M, Fettweis G, Ansari J, Ashraf SA, Almeroth B, Voigt J, Riedel I (2017) Latency critical iot applications in 5 g: perspective on the design of radio interface and network architecture. IEEE Commun Mag 55(2):70–78. https://doi.org/10.1109/MCOM.2017.1600435CM

    Article  Google Scholar 

  45. Shakshuki EM, Malik H (2019) Special issue on ubiquitous computing in the IoT revolution. J Ambient Intell Hum Comput. https://doi.org/10.1007/s12652-019-01360-9

    Article  Google Scholar 

  46. Singh G, Kaur K (2018) An improved weighted least connection scheduling algorithm for load balancing in web cluster systems. Int Res J Eng Technol (IRJET) 5(3):2395

    Google Scholar 

  47. Sukhpal S, Inderveer C (2016) A survey on resource scheduling in cloud computing: issues and challenges. J Grid Comput 14:217–264. https://doi.org/10.1007/s10723-015-9359-2

    Article  Google Scholar 

  48. Tan Y, Liu W, Qiu Q (2009) Adaptive power management using reinforcement learning. In Proceedings of the 2009 international conference on computer-aided design, ACM: 461-467. https://doi.org/10.1145/1687399.1687486

  49. Tesauro G, Jong N K, Das R, Bennani M N (2006) A hybrid reinforcement learning approach to autonomic resource allocation. Proceedings of the 2006 IEEE International Conference on Autonomic Computing: 65–73. https://doi.org/10.1109/ICAC.2006.1662383

  50. Wei Y, Yu FR, Song M, Han Z (2018) User scheduling and resource allocation in hetnets with hybrid energy supply: an actor-critic reinforcement learning approach. IEEE Trans Wirel Commun 17(1):680–692. https://doi.org/10.1109/TWC.2017.2769644

    Article  Google Scholar 

  51. Yan M, Feng G, Qin S (2017) Multi-RAT access based on multi-agent reinforcement learning. 2017 IEEE Global communications conference. https://doi.org/10.1109/glocom.2017.8254980

  52. Youm DH, Yadav R (2016) Load balancing strategy using round robin algorithm. Asia-Pacific J Convergent Res Interchange 2(3):1–10. https://doi.org/10.21742/apjcri.2016.09.01

    Article  Google Scholar 

  53. Yousefpour A, Fung C, Nguyen T, Kadiyala K (2019) All one needs to know about fog computing and related edge computing paradigms: a complete survey. J Syst Architect 98:289–330. https://doi.org/10.1016/j.sysarc.2019.02.009

    Article  Google Scholar 

  54. Yu J, Buyya R, Tham C K (2005) Cost-based scheduling of scientific workflow application on utility grids. Proc.1st Int.Conf. e-Sci. Grid Comput Sci: 140–147. https://doi.org/10.1109/E-SCIENCE.2005.26

  55. Yuan J, Miao X, Li L, Jiang X (2013) An online energy saving resource optimization methodology for data center. J Softw 8(8):1875–1880. https://doi.org/10.4304/jsw.8.8.1875-1880

    Article  Google Scholar 

  56. Zenko J, Kos M, Kramberger I (2016) Pulse rate variability and blood oxidation content identification using miniature wearable wrist device. International conference on systems, signals and image processing (IWSSIP): 1−4

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Fatma M. Talaat.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Talaat, F.M., Saraya, M.S., Saleh, A.I. et al. A load balancing and optimization strategy (LBOS) using reinforcement learning in fog computing environment. J Ambient Intell Human Comput 11, 4951–4966 (2020). https://doi.org/10.1007/s12652-020-01768-8

Download citation

Keywords

  • Load balancing
  • Resource allocation
  • Real time fog computing
  • Healthcare system
  • Machine learning
  • Reinforcement learning
  • Artificial intelligence
  • Genetic algorithm
  • Mobile HEALTH dataset