Journal of Network and Systems Management

, Volume 21, Issue 4, pp 562–587 | Cite as

Security-Preserving Live Migration of Virtual Machines in the Cloud



Hypervisor-based process protection is a novel approach that provides isolated execution environments for applications running on untrusted commodity operating systems. It is based on off-the-shelf hardware and trusted hypervisors while it meets the requirement of security and trust for many cloud computing models, especially third-party data centers and a multi-tenant public cloud, in which sensitive data are out of the control of the users. However, as the hypervisor extends semantic protection to the process granularity, such a mechanism also breaks the platform independency of virtual machines and thus prohibits live migration of virtual machines, which is another highly desirable feature in the cloud. In this paper, we extend hypervisor-based process protection systems with live migration capabilities by migrating the protection-related metadata maintained in the hypervisor together with virtual machines and protecting sensitive user contents using encryption and hashing. We also propose a security-preserving live migration protocol that addresses several security threats during live migration procedures including timing-related attacks, replay attacks and resumption order attacks. We implement a prototype system base on Xen and Linux. Evaluation results show that performance degradation in terms of both total migration time and downtime are reasonably low compared to the unmodified Xen live migration system.


Privacy protection Live migration Virtual machine Cloud 



This work was funded by Shanghai Science and Technology Development Funds (No. 12QA1401700), China National Natural Science Foundation under grant numbered 61003002 and Fundamental Research Funds for the Central Universities in China.


  1. 1.
    Heiser, J., Nicolett, M.: Assessing the security risks of cloud computing., Jun 2008
  2. 2.
    Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R.H., Konwinski, A., Lee, G., Patterson, D.A., Rabkin, A., Stoica, I., Zaharia, M.: Above the clouds: a Berkeley view of cloud computing. Technical Report UCB/EECS-2009-28, EECS Department, University of California, Berkeley, Feb 2009.
  3. 3.
    Chen, H., Zhang, F., Chen, C., Chen, R., Zang, B., Yew, P., Mao, W.: Tamper-Resistant Execution in an Untrusted Operating System Using A Virtual Machine Monitor. Technical Report 2007-08001, Parallel Processing Institute, Fudan University, Aug 2007Google Scholar
  4. 4.
    Chen, X., Garfinkel, T., Lewis, E.C., Subrahmanyam, P., Waldspurger, C.A., Boneh, D., Dwoskin, J., Ports, D.R.K.: Overshadow: a virtualization-based approach to retrofitting protection in commodity operating systems. In: Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 2–13. ACM, New York (2008)Google Scholar
  5. 5.
    Yang, J. Shin, K.G.: Using hypervisor to provide data secrecy for user applications on a per-page basis. In: Proceedings of ACM International Conference on Virtual Execution Environments, pp. 71–80 (2008)Google Scholar
  6. 6.
    Chen, H., Chen, J., Mao, W., Yan, F.: Daonity–grid security from two levels of virtualization. Inf. Secur. Tech. Rep. 12(3), 123–138 (2007)CrossRefGoogle Scholar
  7. 7.
    Dewan, P., Durham, D., Khosravi, H., Long, M., Nagabhushan, G.: A hypervisor-based system for protecting software runtime memory and persistent storage. In: Proceedings of the Spring Simulation Multiconference. The Society for Computer Simulation, pp. 828–835. International San Diego, CA (2008)Google Scholar
  8. 8.
    Sailer, R., Zhang, X., Jaeger, X., Doorn, L.: Design and implementation of a TCG-based integrity measurement architecture. In: Proceedings of the USENIX Security Symposium, pp. 223–238 (2004)Google Scholar
  9. 9.
    Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E., Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines. In: Proceedings of the Symposium on Networked Systems Design and Implementation, pp. 273–286 (2005)Google Scholar
  10. 10.
    Wood, T., Shenoy, P., Venkataramani, A., Yousif, M.: Black-box and gray-box strategies for virtual machine migration. In: Proceedings of Usenix Conference on Networked Systems Design and Implementation, pp. 229–242 (2007)Google Scholar
  11. 11.
    Nelson, M., Lim, B.H., Hutchins, G.: Fast transparent migration for virtual machines. In: Proceedings of the USENIX Annual Technical Conference, pp. 391–394 (2005)Google Scholar
  12. 12.
    McPhee, W.S.: Operating system integrity in OS/VS2. IBM J. Res. Dev. 13(3), 230 (1974)Google Scholar
  13. 13.
    Bishop, M., Dilger, M.: Checking for race conditions in file accesses. Comput. Syst. 2(2), 131–152 (1996)Google Scholar
  14. 14.
    Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: Proceedings of the ACM Symposium on Operating Systems Principles, pp. 164–177. ACM, New York (2003)Google Scholar
  15. 15.
    Zhang, F., Huang, Y., Wang, H., Chen, H., Zang, B.: PALM: security preserving VM live migration for systems with VMM-enforced protection. In: Proceedings of the 2008 Third Asia-Pacific Trusted Infrastructure Technologies Conference, pp. 9–18. IEEE Computer Society (2008)Google Scholar
  16. 16.
    Bratus, S., D'Cunha, N., Sparks, E., Smith, S.: TOCTOU, Traps, and Trusted Computing. Trusted Computing-Challenges and Applications, pp. 14–32 (2008)Google Scholar
  17. 17.
    Trusted Computing Group. TPM Specification version 1.2, Revision 103, October 2006.
  18. 18.
    Harrison, K., Xu, S.: Protecting cryptographic keys from memory disclosure attacks. In: Proceedings of Annual IEEE/IFIP International Conference on Dependable Systems and Networks, pp. 137–143 (2007)Google Scholar
  19. 19.
    Percival, C.: Cache missing for fun and profit. In: Proceedings of BSDCan, Ottawa, Canada, 2005.
  20. 20.
    Aciiçmez, O.: Yet another microarchitectural attack: exploiting I-Cache. In: Proceedings of ACM Workshop on Computer Security Architecture, pp. 11–18. ACM, New York (2007)Google Scholar
  21. 21.
    Acıiçmez, O., Koç, Ç.K.: Trace-driven cache attacks on AES. Information and Communications Security, pp. 112–121 (2006)Google Scholar
  22. 22.
    Acıiçmez, O., Koç, Ç., Seifert, J.P.: Predicting secret keys via branch prediction. Topics in Cryptology–CT-RSA 2007, pp. 225–242 (2007)Google Scholar
  23. 23.
    Bernstein, D.J.: Cache-timing attacks on AES, 2005.
  24. 24.
    Bonneau, J., Mironov, I.: Cache-collision timing attacks against AES. Cryptographic Hardware and Embedded Systems-CHES 2006, pp. 201–215 (2006)Google Scholar
  25. 25.
    Osvik, D., Shamir, A., Tromer, E.: Cache attacks and countermeasures: the case of AES. Topics in Cryptology–CT-RSA 2006, pp. 1–20 (2006)Google Scholar
  26. 26.
  27. 27.
    Amazon. Amazon web services customer agreement, Oct 2009.
  28. 28.
    Halderman, J.A., Schoen, S.D., Heninger, N., Clarkson, W., Paul, W., Calandrino, J.A., Feldman, A.J., Appelbaum, J., Felten, E.W.: Lest we remember: cold boot attacks on encryption keys. In: Proceedings of the USENIX Security Symposium, pp. 45–60 (2008)Google Scholar
  29. 29.
    Kauer, B.: OSLO: improving the security of trusted computing. In: Proceedings of the USENIX Security Symposium (2007)Google Scholar
  30. 30.
    Kursawe, K., Schellekens, D., Preneel, B.: Analyzing trusted platform communication. In: Proccedings of the CRASH Workshop: CRyptographic Advances in Secure Hardware (2005)Google Scholar
  31. 31.
    Selhorst, M., Stiible, C.: Trusted grub, 2006.
  32. 32.
    Oberheide, J., Cooke, E., Jahanian, F.: Empirical exploitation of live virtual machine migration. In: Proceedings of BlackHat DC Convention (2008)Google Scholar
  33. 33.
    EMC Corp. Daoli trust cloud infrastructure., 2007
  34. 34.
    Sapuntzakis, C.P., Chandra, R., Pfaff, B., Chow, J., Lam, M.S., Rosenblum, M.: Optimizing the migration of virtual computers. In: Proceedings of the 5th ACM Symposium on Operating Systems Design and Implementation (OSDI), pp. 377–390, Boston, MA Dec 2002Google Scholar
  35. 35.
    Whitaker, A., Cox, R.S., Shaw, M., Gribble, S.D.: Constructing services with interposable virtual hardware. In: Proceedings of the Usenix Symposium on Networked Systems Design and Implementation (2004)Google Scholar
  36. 36.
    Hansen, J.G., Henriksen, A.K.: Nomadic operating systems. Master’s thesis, Depaerment of Computer Science, University of Copenhagen, Denmark (2002)Google Scholar
  37. 37.
    Jin, H., Deng, L., Wu, S., Shi, X., Pan, X.: Live virtual machine migration with adaptive memory compression. In: Proceedings of IEEE International Conference on Cluster Computing. IEEE (2009)Google Scholar
  38. 38.
    Liu, H., Jin, H., Liao, X., Hu, L., Yu, C.: Live migration of virtual machine based on full system trace and replay. In: Proceedings of the 18th ACM International Symposium on High Performance Distributed Computing, pp. 101–110. ACM, New York (2009)Google Scholar
  39. 39.
    Liu, P., Yang, Z., Song, X., Zhou, Y., Chen, H., Zang, B.: Heterogeneous live migration of virtual machines. In: Proceedings of International Workshop on Virtualization Technology (2008)Google Scholar
  40. 40.
    Xianqin, C., Han, W., Sumei, W., Xiang, L.: Seamless virtual machine live migration on network security enhanced hypervisor. In Procceding of IEEE International Conference on Broadband Network and Multimedia Technology, pp. 847–853. IEEE (2009)Google Scholar
  41. 41.
    VMware Corp. Virtual Infrastructure 3, 2007.
  42. 42.
    Garfinkel, T., Pfaff, B., Chow, J., Rosenblum, M., Boneh, D.: Terra: a virtual machine-based platform for trusted computing. In: Proceedings of the ACM Symposium on Operating Systems Principles, pp. 193–206 (2003)Google Scholar
  43. 43.
    Peinado, M., Chen, Y., England, P., Manferdelli, J.: NGSCB: a trusted open system. In: Proceedings of ACISP, pp. 86–97 (2004)Google Scholar
  44. 44.
    Kuhlmann, D., Landfermann, R., Ramasamy, H., Schunter, M., Ramunno, G., Vernizzi, D.: An open trusted computing architecture-secure virtual machines enabling user-defined policy enforcement. Technical Report RZ3655, IBM Research, 2006.
  45. 45.
    System Architecture Group. L4Ka::Pistachio Whitepaper. White paper, University of Karlsruhe, Germany (2003)Google Scholar
  46. 46.
    Murray, D.G., Milos, G., Hand, S.: Improving Xen security through disaggregation. In: Proceedings of ACM International Conference on Virtual Execution Environments, pp. 151–160 (2008)Google Scholar
  47. 47.
    Sailer, R., Valdez, E., Jaeger, T., Perez, R., van Doorn, L., Griffin, J.L., Berger, S.: sHype: Secure Hypervisor Approach to Trusted Virtualized Systems. Technical Report RC23511, IBM Research, Feb 2005Google Scholar
  48. 48.
    Intel Corp. Intel Trusted Execution Technology, 2008.
  49. 49.
    Intel. LaGrande Technology Architectural Overview. Technical Report 252491-001, Intel Corporation, Sep. 2003Google Scholar
  50. 50.
    Strongin, G.: Trusted computing using AMD. Inf. Secur. Tech. Rep. 10(2), 120–132 (2005)MathSciNetCrossRefGoogle Scholar
  51. 51.
    Lie, D., Thekkath, C., Mitchell, M., Lincoln, P.: Architectural support for copy and tamper resistant software. In: Proceedings of International Conference on Architectural Support for Programming Languages and Operating Systems (2000)Google Scholar
  52. 52.
    Lie, D., Thekkath, C.A., Horowitz, M.: Implementing an untrusted operating system on trusted hardware. In: Proceedings of ACM Symposium on Operating Systems Principles (2003)Google Scholar
  53. 53.
    Champagne, D., Lee, R.B.: Scalable architectural support for trusted software. In: Proceedings of IEEE International Symposium on High-Performance Computer Architecture, Bangalore, India (2010)Google Scholar
  54. 54.
    Lee, R.B., Kwan, P.C.S., McGregor, J.P., Dwoskin, J., Wang, Z.: Architecture for protecting critical secrets in microprocessors. In: Proceedings of International Symposium on Computer Architecture, pp. 2–13 (2005)Google Scholar
  55. 55.
    Dwoskin, J., Lee, R.B.: Hardware-rooted trust for secure key management and transient trust. In: Proceedings of ACM conference on Computer and Communications Security, pp. 389–400, Alexandria, VA, Oct 2007Google Scholar
  56. 56.
    Suh, G.E., O’Donnell, C.W., Sachdev, I., Devadas, S.: Design and implementation of the aegis single-chip secure processor using physical random functions. In: Proceedings of International Symposium on Computer Architecture, pp. 25–36 (2005)Google Scholar
  57. 57.
    Suh, G.E., Clarke, D., Gassend, B., van Dijk, M., Devadas, S.: AEGIS: architecture for tamper-evident and tamper-resistant processing. In: Proceedings of the Annual international conference on Supercomputing, pp. 160–171. ACM Press, New York, NY (2003)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Parallel Processing InstituteFudan UniversityShanghaiChina
  2. 2.Institute of Parallel and Distributed Systems, School of SoftwareShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations