Advertisement

Resource and Cost Aware Glowworm Mapreduce Optimization Based Big Data Processing in Geo Distributed Data Center

  • S. NithyananthamEmail author
  • G. Singaravel
Article
  • 9 Downloads

Abstract

Handling large data in geographically distributed information centers with resource and cost optimization is a key challenge. With several approaches being designed, handling a large volume of data in multiple datacenters in an inappropriate manner yet is considered to be a time-consuming process. To address these issues, a Multivariate Metaphor based Metaheuristic Glowworm Swarm Map-Reduce Optimization (MM-MGSMO) technique is presented. Here, with search space and large data volume as input for geo-distributed datacenters, glowworm (i.e. virtual machine) population is initialized. With each glowworm possessing a certain amount of luciferin (i.e. objective function), multiple objective functions (i.e. bandwidth, storage capacity, energy and computation cost) are defined for each virtual machine. Next, the glowworm position is updated according to the neighboring factor by means of probability. Followed by this, MapReduce function identifies the optimal virtual machine and accordingly allocation is performed, therefore improving data allocation efficiency. Besides, the workload is assigned across datacenters, reduction in computation cost and storage capacity is guaranteed. Experimental evaluation of MM-MGSMO approach with existing methods attained improved performances with factors such as data allocation efficiency, false-positive rate, storage capacity compared with other cutting edge technologies such as Joint optimization algorithm and Game theory-based dynamic resource allocation strategy.

Keywords

Big data processing Geo-distributed data center Glowworm Swarm optimization MapReduce function 

Notes

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

References

  1. 1.
    Gu, L., Zeng, D., Li, P., & Guo, S. (2014). Cost minimization for big data processing in geo-distributed data centers. IEEE Transactions on Emerging Topics in Computing,2(3), 314–323.CrossRefGoogle Scholar
  2. 2.
    Yuan, X., Min, G., Yang, L. T., Ding, Y., & Fang, Q. (2017). A game theory-based dynamic resource allocation strategy in geo-distributed datacenter clouds. Future Generation Computer Systems,76, 63–72.CrossRefGoogle Scholar
  3. 3.
    Li, P., Guo, S., Miyazaki, T., Liao, X., Jin, H., Zomaya, A. Y., et al. (2017). Traffic-aware geo-distributed big data analytics with predictable job completion time. IEEE Transactions on Parallel and Distributed Systems,28(6), 1785–1796.CrossRefGoogle Scholar
  4. 4.
    Xiao, W., Bao, W., Zhu, X., & Liu, L. (2017). Cost-aware big data processing across geo-distributed datacenters. IEEE Transactions on Parallel and Distributed Systems,28(11), 3114–3127.CrossRefGoogle Scholar
  5. 5.
    Chalack, V. A., Razavi, S. N., & Gudakahriz, S. J. (2017). Resource allocation in cloud environment using approaches based particle swarm optimization. International Journal of Computer Applications Technology and Research.,6(2), 87–90.CrossRefGoogle Scholar
  6. 6.
    Manasrah, A. M., & Ba Ali, H. (2018). Workflow scheduling using hybrid GA-PSO algorithm in cloud computing. Wireless Communications and Mobile Computing,2018, 1–16.CrossRefGoogle Scholar
  7. 7.
    Hashem, I. A. T., Anuar, N. B., Marjani, M., Gani, A., Sangaiah, A. K., & Sakariyah, A. K. (2018). Multi-objective scheduling of MapReduce jobs in big data processing. Multimedia Tools and Applications,77(8), 9979–9994.CrossRefGoogle Scholar
  8. 8.
    Palmieri, F., Fiore, U., Ricciardi, S., & Castiglione, A. (2016). GRASP-based resource re-optimization for effective big data access in federated clouds. Future Generation Computer Systems,54, 168–179.CrossRefGoogle Scholar
  9. 9.
    Rajesh, M., & Singaravel, G. (2014). I/O workload in virtualized data center using hypervisor. International Journal on Recent and Innovation Trends in Computing and Communication,2(8), 2256–2260.Google Scholar
  10. 10.
    Chen, W., Paik, I., & Li, Z. (2017). Cost-aware streaming workflow allocation on geo-distributed data centers. IEEE Transactions on Computers,66(2), 256–271.MathSciNetzbMATHGoogle Scholar
  11. 11.
    Zeng, X., Garg, S. K., Wen, Z., Strazdins, P., Zomaya, A. Y., & Ranjan, R. (2018). Cost efficient scheduling of MapReduce applications on public clouds. Journal of Computational Science,26, 375–388.CrossRefGoogle Scholar
  12. 12.
    Sun, D., Yan, H., Gao, S., Liu, X., & Buyya, R. (2018). Rethinking elastic online scheduling of big data streaming applications over high-velocity continuous data streams. The Journal of Supercomputing, Springer,74(2), 615–636.CrossRefGoogle Scholar
  13. 13.
    Simic, V., Stojanovic, B., & Ivanovic, M. (2019). Optimizing the performance of optimization in the cloudenvironment–an intelligent auto-scaling approach. Future Generation Computer Systems,101, 909–920.CrossRefGoogle Scholar
  14. 14.
    Gawali, M. B., & Shinde, S. K. (2018). Task scheduling and resource allocation in cloud computing using a heuristic approach. Journal of Cloud Computing,7(4), 1–16.Google Scholar
  15. 15.
    Sharkh, M. A., Shami, A., & Ouda, A. (2017). Optimal and suboptimal resource allocation techniques in cloud computing data centers. Journal of Cloud Computing,6(6), 1–17.Google Scholar
  16. 16.
    Rawas, S., & Zekri, A. (2018). Location-aware energy-efficient workload allocation in geo distributed cloud environment. Journal of Computer Science,14(3), 334–350.CrossRefGoogle Scholar
  17. 17.
    Ziafat, H., & Babamir, S. M. (2018). Optimal selection of VMs for resource task scheduling in geographically distributed clouds using fuzzy c-mean and MOLP. Journal of Software: Practice and Experience,48(10), 1820–1846.Google Scholar
  18. 18.
    Ficco, M., Esposito, C., Palmieri, F., & Castiglion, A. (2018). A coral-reefs and game theory-based approach for optimizing elastic cloud resource allocation. Future Generation Computer Systems,78, 343–352.CrossRefGoogle Scholar
  19. 19.
    Zheng, W., Qin, Y., Bugingo, E., Zhang, D., & Chen, J. (2018). Cost optimization for deadline-aware scheduling of big-data processing jobs on clouds. Future Generation Computer Systems,82, 244–255.CrossRefGoogle Scholar
  20. 20.
    Zhang, N., Yang, X., Zhang, M., Sun, Y., & Lon, K. (2018). A genetic algorithm-based task scheduling for cloud resource crowd-funding model. International Journal of Communication Systems,31(1), 1–10.CrossRefGoogle Scholar
  21. 21.
    Forestiero, A., Mastroianni, C., Meo, M., Papuzzo, G., & Sheikhalishahi, M. (2017). Hierarchical approach for efficient workload management in geo-distributed data centers. IEEE Transactions on Green Communications and Networking,1(1), 97–111.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2020

Authors and Affiliations

  1. 1.Computer Science and EngineeringAL-Amen Engineering CollegeErodeIndia
  2. 2.Department of Information TechnologyK.S.R. College of Engineering (Autonomous)Tiruchengode, Namakkal DistrictIndia

Personalised recommendations