Advertisement

Profit Maximization and Time Minimization Admission Control and Resource Scheduling for Cloud-Based Big Data Analytics-as-a-Service Platforms

  • Yali ZhaoEmail author
  • Rodrigo N. Calheiros
  • Athanasios V. Vasilakos
  • James Bailey
  • Richard O. Sinnott
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11512)

Abstract

Big data analytics typically requires large amounts of resources to process ever-increasing data volumes. This can be time consuming and result in considerable expenses. Analytics-as-a-Service (AaaS) platforms provide a way to tackle expensive resource costs and lengthy data processing times by leveraging automatic resource management with a pay-per-use service delivery model. This paper explores optimization of resource management algorithms for AaaS platforms to automatically and elastically provision cloud resources to execute queries with Service Level Agreement (SLA) guarantees. We present admission control and cloud resource scheduling algorithms that serve multiple objectives including profit maximization for AaaS platform providers and query time minimization for users. Moreover, to enable queries that require timely responses and/or have constrained budgets, we apply data sampling-based admission control and resource scheduling where accuracy can be traded-off for reduced costs and quicker responses when necessary. We conduct extensive experimental evaluations for the algorithm performances compared to state-of-the-art algorithms. Experiment results show that our proposed algorithms perform significantly better in increasing query admission rates, consuming less resources and hence reducing costs, and ultimately provide a more flexible resource management solution for fast, cost-effective, and reliable big data processing.

Keywords

Optimization Service level agreement Analytics-as-a-service Admission control Resource scheduling Data sampling Big data Cloud computing 

References

  1. 1.
    Assunção, M.D., Calheiros, R.N., Bianchi, S., Netto, M.A., Buyya, R.: Big data computing and clouds: trends and future directions. J. Parallel Distrib. Comput. 79, 3–15 (2015)CrossRefGoogle Scholar
  2. 2.
    Hashem, I.A.T., Yaqoob, I., Anuar, N.B., Mokhtar, S., Gani, A., Khan, U.: The rise of ‘big data’ on cloud computing: review and open research issues. Inf. Syst. 47, 98–115 (2014)CrossRefGoogle Scholar
  3. 3.
    Zhao, Y., Calheiros, R.N., Bailey, J., Sinnott, R.: SLA-based profit optimization for resource management of big data analytics-as-a-service platforms in cloud computing environments. In: Proceedings of the IEEE International Conference on Big Data, pp. 432–441 (2016)Google Scholar
  4. 4.
    Chaudhuri, S., Das, G., Narasayya, V.: Optimized stratified sampling for approximate query processing. ACM Trans. Database Syst. (TODS) 32(2), 9 (2007)CrossRefGoogle Scholar
  5. 5.
    Benayoun, R., De Montgolfier, J., Tergny, J., Laritchev, O.: Linear programming with multiple objective functions: step method (STEM). Math. Program. 1, 366–375 (1971)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Isermann, H.: Linear lexicographic optimization. OR Spektrum 4, 223–228 (1982)CrossRefGoogle Scholar
  7. 7.
    Calheiros, R.N., Ranjan, R., Beloglazov, A., De Rose, C.A., Buyya, R.: CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw. Pract. Exp. 41(1), 23–50 (2011)CrossRefGoogle Scholar
  8. 8.
    IBM ILOG CPLEX Optimization Studio. https://www.ibm.com/developerworks/downloads/ws/ilogcplex/. Accessed 03 Dec 2018
  9. 9.
    Amazon EC2. http://aws.amazon.com/ec2/instance-types/. Accessed 03 Dec 2018
  10. 10.
    Big Data Benchmark. https://amplab.cs.berkeley.edu/benchmark/. Accessed 12 Dec 2018
  11. 11.
    Agarwal, S., Mozafari, B., Panda, A., Milner, H., Madden, S., Stoica, I.: BlinkDB: queries with bounded errors and bounded response times on very large data. In: Proceedings of the 8th ACM European Conference on Computer Systems, p. 29 (2013)Google Scholar
  12. 12.
    Zhao, Y., Calheiros, R.N., Gange, G., Ramamohanarao, K., Buyya, R.: SLA-based resource scheduling for big data analytics as a service in cloud computing environments. In: Proceedings of the 44th IEEE International Conference on Parallel Processing, pp. 510–519 (2015)Google Scholar
  13. 13.
    Zhao, Y., Calheiros, R.N., Gange, G., Bailey, J., Sinnott, R.: SLA-based profit optimization for resource scheduling of big data analytics-as-a-service in cloud computing environments. IEEE Trans. Cloud Comput. 1–18 (2018)Google Scholar
  14. 14.
    Zhang, H., Zhao, Y., Pang, C., He, J.: Splitting large medical data sets based on normal distribution in cloud environment. IEEE Trans. Cloud Comput. (99), 1 (2015, in press)Google Scholar
  15. 15.
    Tordini, F., Aldinucci, M., Viviani, P., Merelli, I., Lio, P.: Scientific workflows on clouds with heterogeneous and preemptible instances. In: Proceedings of the International Conference on Parallel Computing, pp. 605–614 (2017)Google Scholar
  16. 16.
    Mian, R., Martin, P., Vazquez-Poletti, J.: Provisioning data analytic workloads in a cloud. Future Gener. Comput. Syst. 29(6), 1452–1458 (2013)CrossRefGoogle Scholar
  17. 17.
    Wang, K., Zhou, X., Li, T., Zhao, D., Lang, M., Raicu, I.: Optimizing load balancing and data-locality with data-aware scheduling. In: Proceedings of the 2014 IEEE International Conference on Big Data, pp. 119–128 (2015)Google Scholar
  18. 18.
    Xia, Q., Xu, Z., Liang, W., Zomaya, A.Y.: Collaboration-and fairness-aware big data management in distributed clouds. IEEE Trans. Parallel Distrib. Syst. 27(7), 1941–1953 (2015)CrossRefGoogle Scholar
  19. 19.
    Garg, S.K., Toosi, A.N., Gopalaiyengar, S.K., Buyya, R.: SLA-based virtual machine management for heterogeneous workloads in a cloud datacenter. J. Netw. Comput. Appl. 45, 108–120 (2014)CrossRefGoogle Scholar
  20. 20.
    Gu, L., Zeng, D., Li, P., Guo, S.: Cost minimization for big data processing in geo-distributed data centers. IEEE Trans. Emerg. Top. Comput. 2(3), 314–323 (2014)CrossRefGoogle Scholar
  21. 21.
    Zheng, W., Qin, Y., Bugingo, E., Zhang, D., Chen, J.: Cost optimization for deadline-aware scheduling of big data processing jobs on clouds. Future Gener. Comput. Syst. 82, 244–255 (2018)CrossRefGoogle Scholar
  22. 22.
    Dai, W., Qiu, L., Wu, A., Qiu, M.: Cloud infrastructure resource allocation for big data applications. IEEE Trans. Big Data 4(3), 313–324 (2018)CrossRefGoogle Scholar
  23. 23.
    Zhou, A.C., He, B., Cheng, X., Lau, C.T.: A declarative optimization engine for resource provisioning of scientific workflows in IaaS clouds. In: Proceedings of the 24th International Symposium on High Performance Parallel Distributed Computing, pp. 223–234 (2015)Google Scholar
  24. 24.
    Mao, M., Humphrey, M.: Auto-scaling to minimize cost and meet application deadlines in cloud workflows. In: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, Seatle, WA, pp. 1–12 (2011)Google Scholar
  25. 25.
    Chen, J., Wang, C., Zhou, B.B., Sun, L., Lee, Y.C., Zomaya, A.Y.: Tradeoffs between profit and customer satisfaction for service provisioning in the cloud. In: Proceedings of the 20th International Symposium on High Performance Distributed Computing, pp. 229–238 (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yali Zhao
    • 1
    Email author
  • Rodrigo N. Calheiros
    • 2
  • Athanasios V. Vasilakos
    • 3
  • James Bailey
    • 1
  • Richard O. Sinnott
    • 1
  1. 1.The University of MelbourneMelbourneAustralia
  2. 2.Western Sydney UniversitySydneyAustralia
  3. 3.Lulea University of TechnologyLuleaSweden

Personalised recommendations