CUDA Virtualization and Remoting for GPGPU Based Acceleration Offloading at the Edge

  • Antonio MentoneEmail author
  • Diana Di LuccioEmail author
  • Luca Landolfi
  • Sokol Kosta
  • Raffaele Montella
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11874)


In the last decade, GPGPU virtualization and remoting have been among the most important research topics in the field of computer science and engineering due to the rising of cloud computing technologies. Public, private, and hybrid infrastructures need such virtualization tools in order to multiplex and better organize the computing resources. With the advent of novel technologies and paradigms, such as edge computing, code offloading in mobile clouds, deep learning techniques, etc., the need for computing power, especially of specialized hardware such as GPUs, has skyrocketed. Although many GPGPU virtualization tools are available nowadays, in this paper we focus on improving GVirtuS, our solution for GPU virtualization. The contributions in this work focus on the CUDA plug-in, in order to provide updated performance enabling the next generation of GPGPU code offloading applications. Moreover, we present a new GVirtuS implementation characterized by a highly modular approach with a full multithread support. We evaluate and discuss the benchmarks of the new implementation comparing and contrasting the results with the pure CUDA and with the previous version of GVirtuS. The new GVirtuS yielded better results when compared with its previous implementation, closing the gap with the pure CUDA performance and trailblazing the path for the next future improvements.


HPC GPGPU Cloud computing Virtualization 


  1. 1.
    Castelló, A., et al.: On the use of remote GPUs and low-power processors for the acceleration of scientific applications. In: The Fourth International Conference on Smart Grids, Green Communications and IT Energy-Aware Technologies (ENERGY), pp. 57–62 (2014)Google Scholar
  2. 2.
    Di Lauro, R., Giannone, F., Ambrosio, L., Montella, R.: Virtualizing general purpose GPUs for high performance cloud computing: an application to a fluid simulator. In: 2012 IEEE 10th International Symposium on Parallel and Distributed Processing with Applications, pp. 863–864. IEEE (2012)Google Scholar
  3. 3.
    Duato, J., Pena, A.J., Silla, F., Fernandez, J.C., Mayo, R., Quintana-Orti, E.S.: Enabling CUDA acceleration within virtual machines using rCUDA. In: 2011 18th International Conference on High Performance Computing, pp. 1–10. IEEE (2011)Google Scholar
  4. 4.
    Duato, J., Pena, A.J., Silla, F., Mayo, R., Quintana-Ortí, E.S.: rCUDA: reducing the number of GPU-based accelerators in high performance clusters. In: 2010 International Conference on High Performance Computing & Simulation, pp. 224–231. IEEE (2010)Google Scholar
  5. 5.
    Foster, I., Zhao, Y., Raicu, I., Lu, S.: Cloud computing and grid computing 360-degree compared. arXiv preprint arXiv:0901.0131 (2008)
  6. 6.
    Giunta, G., Montella, R., Agrillo, G., Coviello, G.: A GPGPU transparent virtualization component for high performance computing clouds. In: D’Ambra, P., Guarracino, M., Talia, D. (eds.) Euro-Par 2010. LNCS, vol. 6271, pp. 379–391. Springer, Heidelberg (2010). Scholar
  7. 7.
    Hong, C.H., Spence, I., Nikolopoulos, D.S.: Gpu virtualization and scheduling methods: a comprehensive survey. ACM Comput. Surv. (CSUR) 50(3), 35 (2017)CrossRefGoogle Scholar
  8. 8.
    Martinez-Noriega, E.J., Kawai, A., Yoshikawa, K., Yasuoka, K., Narumi, T.: CUDA enabled for android tablets through DS-CUDA (2013)Google Scholar
  9. 9.
    Mell, P.: The NIST definition of cloud computing v15. (2009)
  10. 10.
    Montella, R., Coviello, G., Giunta, G., Laccetti, G., Isaila, F., Blas, J.G.: A general-purpose virtualization service for HPC on cloud computing: an application to GPUs. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds.) PPAM 2011. LNCS, vol. 7203, pp. 740–749. Springer, Heidelberg (2012). Scholar
  11. 11.
    Montella, R., Ferraro, C., Kosta, S., Pelliccia, V., Giunta, G.: Enabling Android-based devices to high-end GPGPUs. In: Carretero, J., Garcia-Blas, J., Ko, R.K.L., Mueller, P., Nakano, K. (eds.) ICA3PP 2016. LNCS, vol. 10048, pp. 118–125. Springer, Cham (2016). Scholar
  12. 12.
    Montella, R., et al.: On the virtualization of CUDA based GPU remoting on arm and X86 machines in the GVirtuS framework. Int. J. Parallel Prog. 45(5), 1142–1163 (2017)CrossRefGoogle Scholar
  13. 13.
    Montella, R., et al.: Accelerating Linux and Android applications on low-power devices through remote GPGPU offloading. Concurr. Comput.: Pract. Exp. 29(24), e4286 (2017)., e4286 cpe.4286CrossRefGoogle Scholar
  14. 14.
    Oikawa, M., Kawai, A., Nomura, K., Yasuoka, K., Yoshikawa, K., Narumi, T.: DS-CUDA: a middleware to use many GPUs in the cloud environment, pp. 1207–1214 (2012)Google Scholar
  15. 15.
    Reaño, C., Silla, F.: A performance comparison of CUDA remote GPU virtualization frameworks. In: 2015 IEEE International Conference on Cluster Computing, pp. 488–489. IEEE (2015)Google Scholar
  16. 16.
    Reaño, C., Silla, F.: Reducing the performance gap of remote GPU virtualization with InfiniBand Connect-IB. In: 2016 IEEE Symposium on Computers and Communication (ISCC), pp. 920–925. IEEE (2016)Google Scholar
  17. 17.
    Reaño, C., Silla, F., Shainer, G., Schultz, S.: Local and remote GPUs perform similar with EDR 100G InfiniBand. In: Proceedings of the Industrial Track of the 16th International Middleware Conference, p. 4. ACM (2015)Google Scholar
  18. 18.
    Tange, O., et al.: GNU parallel-the command-line power tool. USENIX Mag. 36(1), 42–47 (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Science and Technologies DepartmentUniversity of Naples “Parthenope”NapoliItaly
  2. 2.Department of Electronic SystemsAalborg University CopenhagenCopenhagenDenmark

Personalised recommendations