Cluster Computing

, Volume 16, Issue 2, pp 249–264 | Cite as

Performance and energy modeling for live migration of virtual machines



Live migration of virtual machine (VM) provides a significant benefit for virtual server mobility without disrupting service. It is widely used for system management in virtualized data centers. However, migration costs may vary significantly for different workloads due to the variety of VM configurations and workload characteristics. To take into account the migration overhead in migration decision-making, we investigate design methodologies to quantitatively predict the migration performance and energy consumption. We thoroughly analyze the key parameters that affect the migration cost from theory to practice. We construct application-oblivious models for the cost prediction by using learned knowledge about the workloads at the hypervisor (also called VMM) level. This should be the first kind of work to estimate VM live migration cost in terms of both performance and energy in a quantitative approach. We evaluate the models using five representative workloads on a Xen virtualized environment. Experimental results show that the refined model yields higher than 90% prediction accuracy in comparison with measured cost. Model-guided decisions can significantly reduce the migration cost by more than 72.9% at an energy saving of 73.6%.


Virtual machine Live migration Performance model Energy 



This work is supported by National 973 Basic Research Program of China under grant No. 2007CB310900, the China National Natural Science Foundation (NSFC) (Grant No. 60973133), the MoE-Intel Information Technology Special Research Foundation under grant No. MOE-INTEL-10-05, and U.S. NSF under grants CRI-0708232, CNS-0702488, CNS-0914330, CCF-1016966. The preliminary result of this paper was published in HPDC 2011 [26].


  1. 1.
    Akoush, S., Sohan, R., Rice, A., Moore, A.W., Hopper, A.: Predicting the performance of virtual machine migration. In: The 18th Annual IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS’10), Miami, Florida, USA, 17–19 August 2010, pp. 37–46 (2010) Google Scholar
  2. 2.
    Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles (SOSP’03), Lake George, New York, USA, 19–22 October 2003, pp. 164–177 (2003) CrossRefGoogle Scholar
  3. 3.
    Blackburn, S.M., Garner, R., Hoffman, C., Khan, A.M., McKinley, K.S., Bentzur, R., Diwan, A., Feinberg, D., Frampton, D., Guyer, S.Z., Hirzel, M., Hosking, A., Jump, M., Lee, H., Moss, J.E.B., Phansalkar, A., Stefanović, D., VanDrunen, T., von Dincklage, D., Wiedermann, B.: The DaCapo benchmarks: Java benchmarking development and analysis. In: Proceedings of the 21st Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA’06), Portland, OR, USA, 22–26 October 2006 Google Scholar
  4. 4.
    Cardosa, M., Korupolu, M.R., Singh, A.: Shares and utilities based power consolidation in virtualized server environments. In: Proceedings of the 11th IFIP/IEEE International Conference on Symposium on Integrated Network Management (IM’09), New York, NY, USA, 1–5 June 2009, pp. 327–334 (2009) CrossRefGoogle Scholar
  5. 5.
    Chen, Y., Zhang, S., Xu, S., Li, G.: Fundamental tradeoffs on green wireless networks. IEEE Commun. Mag. 49(6), 30–37 (2011) CrossRefGoogle Scholar
  6. 6.
    Choi, J., Govindan, S., Urgaonkar, B., Sivasubramaniam, A.: Profiling prediction, and capping of power consumption in consolidated environments. In: Proceedings of IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS’08), Baltimore, MD, USA, 8–10 Sept 2008, pp. 1–10 (2008) CrossRefGoogle Scholar
  7. 7.
    Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E., Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines. In: Proceedings of Second Symposium Networked Systems Design and Implementation (NSDI’05), pp. 273–286, 2–4 May 2005 Google Scholar
  8. 8.
    Comer, D.: Internetworking with TCP/IP, p. 226. Prentice Hall, Upper Saddle River (2000) Google Scholar
  9. 9.
    Electronic Educational Devices Inc., Watts up pro power meter.
  10. 10.
    Feeney, L.M., Nilsson, M.: Investigating the energy consumption of a wireless network interface in an ad hoc networking environment. In: Proceeding of IEEE Conference on Computer Communications (Infocom’01), pp. 1548–1557, 22–26 April 2001 Google Scholar
  11. 11.
    Fu, S., Xu, C.-Z.: Stochastic modeling and analysis of hybrid mobility in reconfigurable distributed virtual machines. J. Parallel Distrib. Comput. 66(11), 1442–1454 (2006) CrossRefGoogle Scholar
  12. 12.
    Gong, J., Xu, C.-Z.: A gray-box feedback control approach for system-level peak power management. In: Proceedings of 39th International Conference on Parallel Processing (ICPP’10), San Diego, CA, USA, 13–16 September 2010, pp. 555–564 (2010) Google Scholar
  13. 13.
    Gong, J., Xu, C.-Z.: vPnP: automated coordination of power and performance in virtualized datacenters. In: The IEEE International Workshop on Quality of Service (IWQoS’10), Beijing, China, 16–18 June 2010, pp. 1–9 (2010) Google Scholar
  14. 14.
    Hines, M., Gopalan, K.: Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning. In: International Conference on Virtual Execution Environments (VEE’09), Washington DC, USA, 11–13 March 2009, pp. 51–60 (2009) Google Scholar
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
    Hu, L., Jin, H., Liao, X., Xiong, X., Liu, H.: Magnet: A novel scheduling policy for power reduction in cluster with virtual machines. In: Proceeding of 2008 IEEE International Conference on Cluster Computing (Cluster‘08), Tsukuba, Japan, 29 September–1 October 2008, pp. 13–22 (2008) Google Scholar
  21. 21.
    Kangarlou, A., Eugster, P., Xu, D.: VNsnap: taking snapshots of virtual networked environments with minimal downtime. In: Proceedings of IEEE/IFIP International Conference on Dependable Systems and Networks (DSN’09). Estoril, Portugal, 29 June–2 July 2009, pp. 524–533 (2009) CrossRefGoogle Scholar
  22. 22.
    Krishnan, B., Amur, H., Gavrilovska, A., Schwan, K.: VM power metering: feasibility and challenges. In: The Second Green Metrics Workshop, in conjunction with SIGMETRICS’10, New York, NY, USA, 14 June 2010 Google Scholar
  23. 23.
    Kumar, K., Lu, Y.: Cloud computing for mobile users: Can offloading computation save energy? IEEE Comput. 43(4), 51–56 (2010) CrossRefGoogle Scholar
  24. 24.
    Lim, M.Y., Rawson, F., Bletsch, T., Freeh, V.W.: PADD: power aware domain distribution. In: Proceedings of the 29th IEEE International Conference on Distributed Computing Systems (ICDCS’09), Montreal, Quebec, Canada, 22–26 June 2009, pp. 239–247 (2009) CrossRefGoogle Scholar
  25. 25.
    Liu, H., Jin, H., Liao, X., Hu, L., Yu, C.: Live migration of virtual machine based on full system trace and replay. In: Proceedings of the 18th International Symposium on High Performance Distributed Computing (HPDC’09), Munich, Germany, 11–13 June 2009, pp. 101–110 (2009) CrossRefGoogle Scholar
  26. 26.
    Liu, H., Xu, C.-Z., Jin, H., Gong, J., Liao, X.: Performance and energy modeling for live migration of virtual machine. In: Proceedings of the 20th International ACM Symposium on High Performance Distributed Computing (HPDC’11), San Jose, CA, USA, 8–11 June 2011, pp. 171–182 (2011) Google Scholar
  27. 27.
    Miettinen, A.P., Nurminen, J.K.: Energy efficiency of mobile clients in cloud computing. In: The 2nd USENIX Workshop on Hot Topics in Cloud Computing (HotCloud’10), Boston, MA, USA, 22 June 2010 Google Scholar
  28. 28.
    Nagarajan, A.B., Mueller, F., Engelmann, C., Scott, S.L.: Proactive fault tolerance for HPC with Xen virtualization. In: Proceedings of ACM Annual International Conference on Supercomputing (ICS’07), Seattle, Washington, USA, 17–21 June 2007, pp. 23–32 (2007) CrossRefGoogle Scholar
  29. 29.
    Nathuji, R., Schwan, K.: Virtual power: coordinated power management in virtualized enterprise systems. In: Proceedings of ACM Symposium on Operating Systems Principles (SOSP’07), Stevenson, WA, USA, 14–17 October 2007 Google Scholar
  30. 30.
    Nelson, M., Lim, B.H., Hutchins, G.: Fast transparent migration for virtual machines. In: Proceedings of USENIX Annual Technical Conference (USENIX’05), Anaheim, California, USA, 10–15 April 2005, pp. 391–394 (2005) Google Scholar
  31. 31.
    Rodero, I., Jaramillo, J., Quiroz, A., Parashar, M., Guim, F.: Towards energy-aware autonomic provisioning for virtualized environments. In: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing (HPDC’10), Chicago, Illinois, USA, 20–25 June 2010, pp. 320–323 (2010) CrossRefGoogle Scholar
  32. 32.
    Sato, K., Sato, H., Matsuoka, S.: Model-based optimization for data-intensive application on virtual cluster. In: The 9th IEEE/ACM International Conference on Grid Computing (Grid’08), Tsukuba, Japan, pp. 367–368 Google Scholar
  33. 33.
    Sotomayor, B., Keahey, K., Foster, I.: Combining batch execution and leasing using virtual machines. In: Proceedings of the Eighteenth International Symposium on High Performance Distributed Computing (HPDC’08), Boston, MA, USA, 23–27 June 2008, pp. 87–96 (2008) Google Scholar
  34. 34.
    Tarighi, M., Motamedi, S.A., Sharifian, S.: A new model for virtual machine migration in virtualized cluster server based on fuzzy decision making. J. Telecommun. 1(1), 40–51 (2010) Google Scholar
  35. 35.
    Verma, A., Ahuja, P., Neogi, A.: pMapper: power and migration cost aware application placement in virtualized systems. In: Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware (Middleware’08), Leuven, Belgium, 1–5 December 2008, pp. 243–264. Springer, Berlin (2008) Google Scholar
  36. 36.
    Voorsluys, W., Broberg, J., Venugopal, S., Buyya, R.: Cost of virtual machine live migration in clouds: a performance evaluation. In: Proceedings of the 1st International Conference on Cloud Computing. Lecture Notes in Computer Science, Beijing, China, December 2009, pp. 254–265 (2009) Google Scholar
  37. 37.
    Wood, T., Shenoy, P., Venkataramani, A., Yousif, M.: Black-box and gray-box strategies for virtual machine migration. In: Proceedings of 4th USENIX Symposium on Networked Systems Design and Implementation (NSDI’07), Cambridge, MA, USA, 11–13 April 2007, pp. 229–242 (2007) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Haikun Liu
    • 1
  • Hai Jin
    • 1
  • Cheng-Zhong Xu
    • 2
  • Xiaofei Liao
    • 1
  1. 1.Services Computing Technology and System Lab., Cluster and Grid Computing Lab., School of Computer Science and TechnologyHuazhong University of Science and TechnologyWuhanChina
  2. 2.Department of Electrical and Computer EngineeringWayne State UniversityDetroitUSA

Personalised recommendations