Quality of Service for I/O Workloads in Multicore Virtualized Servers

Part of the Computer Communications and Networks book series (CCN)


Emerging trend of multicore servers promises to be the panacea for all data-center issues with system virtualization as the enabling technology. System virtualization allows one to create virtual replicas of the physical system, over which independent virtual machines can be created, complete with their own, individual operating systems, software, and applications. This provides total system isolation of the virtual machines. Apart from this, the key driver for virtualization adoption in data-centers will be safe virtual machine performance isolation that can be achieved over a consolidated server with shared resources. This chapter identifies the basic requirements for performance isolation of virtual machines on such servers. The consolidation focus is on enterprise workloads that are a mix of compute and I/O intensive workloads. An analysis of prevalent, popular system virtualization technologies is presented with a view toward application performance isolation. Based on the observed lacunae, an end-to-end system virtualization architecture is proposed and evaluated.


Virtual Machine Device Driver Virtual Machine Monitor Network Interface Card Device Virtualization 



Credits for this work are due to all those unknown reviewers who have meticulously pointed out deficiencies and improvements over several rounds of reviews and also to the summer interns who have enthusiastically carried out the numerous experimental work that helped validate the simulation results.


  1. 1.
    Goldberg, R.P.: Survey of virtual machine research. IEEE Comput. 7(6), 34–45 (1974) CrossRefGoogle Scholar
  2. 2.
    Jones, R.A.: Netperf: a network performance benchmark revision 2.0. Technical Report, Information Networks Division, Hewlett-Packard Company (1993). Available online: http://ci.nii.ac.jp/naid/10000088072/en/. Cited 30 April 2010
  3. 3.
    RADS Carleton Univ.: Layered Queueing Network Solver software package (1995). Available online: http://www.sce.carleton.ca/rads/lqns. Cited 30 April 2010
  4. 4.
    Rolia, J.A., Sevcik, K.C.: The method of layers. IEEE Trans. Softw. Eng. 21(8), 689–700 (1995) CrossRefGoogle Scholar
  5. 5.
    Woodside, C.M., Neilson, J.E., Petriu, D.C., Majumdar, S.: The stochastic rendezvous network model for performance of synchronous client-server-like distributed software. IEEE Trans. Comput. 44(1), 20–34 (1995) MATHCrossRefGoogle Scholar
  6. 6.
    Kaashoek, M.F., et al.: Application performance and flexibility on exokernel systems. In: 16th ACM SOSP, pp. 52–65 (1997) Google Scholar
  7. 7.
    Verghese, B., Gupta, A., Rosenblum, M.: Performance isolation: sharing and isolation in shared-memory multiprocessors. ACM SIGPLAN Not. 19, 181–192 (1998) CrossRefGoogle Scholar
  8. 8.
    Mosberger, D., Jin, T.: httperf: a tool for measuring web server performance. In: ACM Workshop on Internet Server Performance, pp. 59–67 (1998) Google Scholar
  9. 9.
    Sugerman, J., Venkatachalam, G., Lim, B.: Virtualizing I/O devices on VMware workstation’s hosted virtual machine monitor. In: Proceedings of the USENIX Annual Technical Conference, pp. 1–14 (2001) Google Scholar
  10. 10.
    Welsh, M., Culler, D.: Virtualization considered harmful OS design directions for well-conditioned services. In: Hot Topics in OS 8th Workshop, pp. 139–144 (2001) Google Scholar
  11. 11.
    Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: 19th ACM SIGOPS, pp. 164–177 (2003) Google Scholar
  12. 12.
    Fraser, K., Hand, S., Neugebauer, R., Pratt, I., Wareld, A., Williamson, M.: Safe hardware access with the Xen virtual machine monitor. In: 1st Workshop on OASIS (2004) Google Scholar
  13. 13.
    The Globus Resource Specification Language RSL v1.0 (2004). Available online: http://www-fp.globus.org/gram/rsl_spec1.html. Cited 30 April 2010
  14. 14.
    Menon, Santos, J.R., Turner, Y., Janakiraman, G.J., Zwaenepoel, W.: Diagnosing performance overheads in the Xen virtual machine environment. In: Proceedings of the ACM/USENIX Conference on Virtual Execution Environments, pp. 13–23 (2005) Google Scholar
  15. 15.
    Vmware (2005) Vmware ESX Server 2—architecture and performance implications (2005). Available online: http://www.vmware.com/pdf/esx2_performance_implications.pdf. Cited 30 April 2010
  16. 16.
    Raj, H., Schwan, K.: Implementing a scalable selfvirtualizing network interface on a multicore platform. In: Workshop on the Interaction Between Operating Systems and Computer Architecture (2005) Google Scholar
  17. 17.
    Gupta, D., Cherkasova, L., Gardner, R., Vahdat, A.: Enforcing performance isolation across virtual machines in Xen. Lect. Notes Comput. Sci. 4290, 342–362 (2006) CrossRefGoogle Scholar
  18. 18.
    Intel Virtualization Technology for Directed-I/O (2006). Available online: www.intel.com/technology/itj/2006/v10i3/2-io/7-conclusion.htm. Cited 30 April 2010
  19. 19.
    Liu, J., Huang, W., Abali, B., Panda, D.K.: High performance VMMbypass I/O in virtual machines. In: Proceedings of the USENIX Annual Technical Conference, pp. 3–3 (2006) Google Scholar
  20. 20.
    Menon, Cox, A.L., Zwaenepoel, W.: Optimizing network virtualization in Xen. In: Proceedings of the USENIX Annual Technical Conference, pp. 2–2 (2006) Google Scholar
  21. 21.
    PCI-SIG IOV Specification (2006). Available online: http://www.pcisig.com/specifications/iov. Cited 30 April 2010
  22. 22.
    Santos, J.R., Janakiraman, G., Turner, Y., Pratt, I.: Netchannel 2: optimizing network performance. In: Xen Summit Talk (2007) Google Scholar
  23. 23.
    Willmann, P., Shafer, J., Carr, D., Menon, A., Rixner, S., Cox, A.L., Zwaenepoel, W.: Concurrent direct network access for virtual machine monitors. In: Proceedings of the International Symposium on High-Performance Computer Architecture, pp. 306–317 (2007) Google Scholar
  24. 24.
    Nesbit, K.J., Moreto, M., Cazorla, F.J., Ramirez, A., Valero, M., Smith, J.E.: Multicore resource management. IEEE Micro 28(3), 6–16 (2008). Special Issue on Interaction of Computer Architecture and Operating System in the Manycore Era CrossRefGoogle Scholar
  25. 25.
    Neterion (2008). Available online: http://www.neterion.com/. Cited 30 April 2010
  26. 26.
    Netxen (2008). Available online: http://www.netxen.com/. Cited 30 April 2010
  27. 27.
    Rixner, S.: Breaking the performance barrier: shared I/O in virtualization platforms has come a long way but performance concerns remain. ACM Queue 6(1), 36 (2008) CrossRefGoogle Scholar
  28. 28.
    Sun Microsystems: CrossBow Network Virtualization and Resource Control (2008). Available online: http://www.opensolaris.org/os/community/networking/crossbow_sunlabs_ext.pdf. Cited 30 April 2010
  29. 29.
    Lakshmi, J., Nandy, S.K.: Modeling Architecture-OS interactions using layered queuing network models. In: International Conference Proceedings of HPC Asia, pp. 382–389 (2009) Google Scholar
  30. 30.
    Lakshmi, J., Nandy, S.K.: I/O device virtualization in multi-core era, a QoS perspective. In: Workshop on Grids, Clouds and Virtualization, Conference on Grids and Pervasive Computing, pp. 128–135 (2009) Google Scholar
  31. 31.
    Kim, H., Lim, H., Jeong, J., Jo, H., Lee, J.: Task-aware virtual machine scheduling for I/O performance. In: Proceedings of ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, pp. 101–110 (2009) Google Scholar
  32. 32.
    Weng, C., Wang, Z., Li, M., Lu, X.: The hybrid scheduling framework for virtual machine systems. In: Proceedings of ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, pp. 111–120 (2009) Google Scholar
  33. 33.
    Solarflare Communications (2009). Available online: http://www.solarflare.com/. Cited 30 April 2010
  34. 34.
    Linux Advanced routing and Traffic control HowTo. Available online: http://lartc.org/howto/index.html. Cited 30 April 2010
  35. 35.
    Lakshmi, J., Nandy, S.K.: I/O virtualization architecture for security. In: IEEE Proceedings of International Workshop on Virtualization Technology (2010) Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.SERCIndian Institute of ScienceBangaloreIndia

Personalised recommendations