Advertisement

Use of Containers for High-Performance Computing

  • F. Medrano-JaimesEmail author
  • J. E. Lozano-Rizk
  • S. Castañeda-Avila
  • R. Rivera-Rodriguez
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 948)

Abstract

The past decade, virtual machines emerged to solve many infrastructure problems and practical use of computing resources. The limitations of this type of technology, are in the sense of resource overload because each virtual machine has a complete copy of an operating system plus different libraries needed to run an application. Containers technology reduces this load by eliminating the hypervisor and the virtual machine for its operation, where each application is executed with the most elementary of a server, plus a shared instance of the operating system that hosts it. Container technology is already an essential part of the IT industry, as it is a simpler and more efficient way to virtualize Micro-Services with workflow’s creations support in development and operations (DevOps). Unlike the use of virtual machines, this solution generates much less overhead in the kernel host and the application, improving performance. In the high-performance computing (HPC) there is a willingness to implement this solution for scientific computing purposes. The most important and standard technology in the industry is Docker, however is not a trivial and direct adoption of this standard for the requirements of scientific computing in a HPC environment. In the present study, a review of research works focused on the use of containers for the HPC will be carried out with the objective of familiarizing the user and system administrator of HPC in the use of this technology, and how scientific research projects can get benefit from this type of technology in terms of mobility of compute and reproducibility of workflows.

Keywords

Containers Virtual machines High-performance computing 

References

  1. 1.
    Adufu, T., Choi, J., Kim, Y.: Is container-based technology a winner for high performance scientific applications? In: 17th Asia-Pacific Network Operations and Management Symposium (APNOMS), pp. 507–510. IEEE (2015)Google Scholar
  2. 2.
    Beserra, D., Moreno, E., Takako, P., et al.: Performance analysis of LXC for HPC environments. In: 9th International Conference on Complex, Intelligent and Software Intensive Systems, pp. 358–363 (2015)Google Scholar
  3. 3.
    Kovács, Á.: Comparison of different Linux containers. In: 40th International Conference on Telecommunications and Signal Processing (TSP), pp. 47–51. IEEE (2017)Google Scholar
  4. 4.
    Bayser, M., Cerqueira, R.: Integrating MPI with docker for HPC. In: IEEE International Conference on Cloud Engineering, pp. 259–265. IEEE Computer Society (2017)Google Scholar
  5. 5.
    Hale, I.J., Li, L., Richardson, C., Wells, G.: Containers for portable, productive, and performant scientific computing. Comput. Sci. Eng. 19, 40–50 (2017)CrossRefGoogle Scholar
  6. 6.
    FEniCS. http://fenicsproject.org. Accessed 21 Feb 2018
  7. 7.
    Younge, A., Pedretti, K., Grant, R., Brightwell, R.: A tale of two systems: using containers to deploy HPC applications on supercomputers and clouds. In: 9th International Conference on Cloud Computing Technology and Science, pp. 74–81. IEEE Computer Society (2017)Google Scholar
  8. 8.
    Priedhorsky, R., Randles, T.: Charliecloud: unprivileged containers for user-defined software stacks in HPC. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, Article no. 36 (2017)Google Scholar
  9. 9.
    Gerhardt, L., Bhimji, W., et al.: Shifter: containers for HPC. In: Journal of Physics: Conference Series (2017)Google Scholar
  10. 10.
    Kurtzer, G., Sochat, S., Bauer, M.: Singularity: scientific containers for mobility of compute. PLoS ONE 12(5), e0177459 (2017).  https://doi.org/10.1371/journal.pone.0177459CrossRefGoogle Scholar
  11. 11.
    Pattinson, D.: Plos One launches reproducibility iniciative. http://blogs.plos.org/everyone/2012/08/14/plos-one-launches-reproducibility-initiative/. Accessed 18 Feb 2018

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • F. Medrano-Jaimes
    • 1
    Email author
  • J. E. Lozano-Rizk
    • 1
  • S. Castañeda-Avila
    • 1
  • R. Rivera-Rodriguez
    • 1
  1. 1.Center for Scientific Research and Higher Education of EnsenadaEnsenadaMexico

Personalised recommendations