Advertisement

CSI2: Cloud Server Idleness Identification by Advanced Machine Learning in Theories and Practice

  • Jun DuanEmail author
  • Guangcheng Li
  • Neeraj Asthana
  • Sai Zeng
  • Ivan Dell’Era
  • Aman Chanana
  • Chitra Agastya
  • William Pointer
  • Rong Yan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11895)

Abstract

Studies show that virtual machines (VMs) in cloud are easily forgotten with non-productive status. This incurs unnecessary cost for cloud tenants and resource waste for cloud providers. As a solution to this problem, we present our Cloud Server Idleness Identification (CSI2) system. The CSI2 system collects data from the servers in cloud, performs analytics against the dataset to identify the idle servers, then provides suggestions to the owners of the idle servers. Once the confirmation from the owners are received, the idle servers are deleted or archived. We not only design and implement the CSI2 system, but also bring it alive into production environment.

How to accurately identify the idleness in cloud is the challenging part of this problem, because there is a trade-off between the cost saving and the user experience. We build a machine learning model to handle this challenge. In addition to that, we also build an advanced tool based on Bayesian optimization (BO) to help us finely tune the hyperparameters of the models. It turns out that our finely tuned models works accurately, successfully handling the aforementioned conflict, and outperforms its predecessors with a F1 score of 0.89.

Keywords

Classification Machine learning Cloud idleness Bayesian optimization 

References

  1. 1.
  2. 2.
    Stoess, J., Lang, C., Bellosa, F.: Energy management for hypervisor-based virtual machines. In: 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference, ATC 2007, USENIX Association, Berkeley, CA, USA, pp. 1:1–1:14 (2007)Google Scholar
  3. 3.
    Wu, H., et al.: Automatic cloud bursting under fermicloud. In: 2013 International Conference on Parallel and Distributed Systems (ICPADS), pp. 681–686, December 2013Google Scholar
  4. 4.
    Wood, T., Shenoy, P., Venkataramani, A., Yousif, M.: Black-box and gray-box strategies for virtual machine migration. In: Proceedings of the 4th USENIX Conference on Networked Systems Design and Implementation, NSDI 2007, USENIX Association, Berkeley, CA, USA, p. 17 (2007)Google Scholar
  5. 5.
    Breitgand, D., Epstein, A.: Improving consolidation of virtual machines with risk-aware bandwidth oversubscription in compute clouds. In: INFOCOM, 2012 Proceedings IEEE, pp. 2861–2865, March 2012Google Scholar
  6. 6.
    Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of machine learning algorithms. In: NIPS (2012)Google Scholar
  7. 7.
    Shen, Z., Young, C.C., Zeng, S., Murthy, K., Bai, K.: Identifying resources for cloud garbage collection. In: 2016 12th International Conference on Network and Service Management (CNSM), pp. 248–252. IEEE, October 2016Google Scholar
  8. 8.
    Cohen, N., Bremler-Barr, A.: Garbo: Graph-based cloud resource cleanup. In: 2015 ACM Symposium on Cloud Computing (SoCC 2015), Kohala Coast, Hawaii, USA, August 2015Google Scholar
  9. 9.
    Kim, I.K., Zeng, S., Young, C., Hwang, J., Humphrey, M.: iCSI: a cloud garbage VM collector for addressing inactive VMs with machine learning. In: 2017 IEEE International Conference on Cloud Engineering (IC2E), pp. 17–28. IEEE, April 2017Google Scholar
  10. 10.
    Kim, I.K., Zeng, S., Young, C., Hwang, J., Humphrey, M.: A supervised learning model for identifying inactive VMs in private cloud data centers. In: Proceedings of the Industrial Track of the 17th International Middleware Conference, p. 2. ACM, December 2016Google Scholar
  11. 11.
    Zhang, B., Al Dhuraibi, Y., Rouvoy, R., Paraiso, F., Seinturier, L.: CloudGC: recycling idle virtual machines in the cloud. In: 2017 IEEE International Conference on Cloud Engineering (IC2E), pp. 105–115. IEEE, April 2017Google Scholar
  12. 12.
    Devoid, S., Desai, N., Hochstein, L.: Poncho: enabling smart administration of full private clouds. In: LISA, pp. 17–26, November 2013Google Scholar
  13. 13.
    Baek, H., Srivastava, A., Van der Merwe, J.: Cloudvmi: virtual machine introspection as a cloud service. In: 2014 IEEE International Conference on Cloud Engineering (IC2E), pp. 153–158. IEEE, March 2014Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Jun Duan
    • 1
    Email author
  • Guangcheng Li
    • 2
  • Neeraj Asthana
    • 1
  • Sai Zeng
    • 1
  • Ivan Dell’Era
    • 1
  • Aman Chanana
    • 1
  • Chitra Agastya
    • 1
  • William Pointer
    • 1
  • Rong Yan
    • 2
  1. 1.IBM T. J. Watson Research CenterYorktown HeightsUSA
  2. 2.IBM China Research LabBeijingChina

Personalised recommendations