Abstract
Nowadays, the requirements of modern software demand a greater computing power; numerous scientific and engineering applications request an increase in data storage capacity, be able to exchange of information at high speeds, as well as a faster data processing and better memory management. The implementation of personal computers interconnected to form a cluster and the use of distributed/parallel file systems are presented as a highly suitable alternative in the solution of complex problems that require these resources as their needs grow. The present work shows the evaluation of OrangeFS as a tool to achieve high performance storage and massive parallel processing. It takes advantage of the capacity of the hard drives included in each node of the cluster through the virtual file system and the network bandwidth, instead of having to add a more expensive type of storage. The tests carried out in a cluster with CentOS show that stripping a large file into small objects and distributed in parallel to the I/O servers provides that upcoming read/write operations runs faster; In addition, the use of the message passing interface in the development and execution of applications allows to increase the parallelism of the data in terms of processing due to the intervention of the multicore processor in each of the clients.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abacus-I Supercomputer. http://www.abacus.cinvestav.mx/caracteristicas. Accessed 03 Dec 2017
Carns, P.H., Ligon III, W.B., Ross, R.B., Thakur, R.: PVFS: a parallel file system for Linux clusters. In: Proceedings of the Extreme Linux Track: 4th Annual Linux Showcase and Conference (2000)
Dickens, P.M., Logan, J.: A high performance implementation of MPI-IO for a Lustre file system environment. Concurrency Comput.: Pract. Exper. 22, 1433–1449 (2010). https://doi.org/10.1002/cpe.1491
Riahi, H., et al.: J. Phys.: Conf. Ser. 396, 042050 (2012). https://doi.org/10.1088/1742-6596/396/4/042050
Hua, X., Wu, H., Li, Z., Ren, S.: Enhancing throughput of the Hadoop Distributed File System for interaction-intensive tasks. J. Parallel Distrib. Comput. 74(8), 2770–2779 (2014). https://doi.org/10.1016/j.jpdc.2014.03.010. http://www.elsevier.com/inca/publications/store/6/2/2/8/9/5/index.htt
HDFS Architecture. http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Introduction. Accessed 12 Oct 2017
Lustre. wiki.lustre.org/images/6/64/LustreArchitecture-v4.pdf. Accessed 1 Sept 2017
Message Passing Interface Forum. MPI: A Message-Passing Interface Standard, Version 2.2. University of Tennessee (2009)
Miztli Architecture. http://www.super.unam.mx/index.php/home/acerca-de?start=2. Accessed 02 Jan 2018
MPICH is a high performance and widely portable implementation of the Message Passing Interface (MPI) standard, mpich.org. http://www.mpich.org/documentation/guides/. Accessed 20 Dec 2017
PVFS2 Team: “Parallel Virtual File System, Version 2”, September 2003. http://www.pvfs.org/pvfs2-guide.html
Sampath, S., Sagar, B.B., Subbaraya, C.K., Nanjesh, B.R.: Performance evaluation of parallel applications using MPI in cluster based parallel computing architecture. In: Proceeding of International Conference on “Emerging Research in Computing, Information, Communication and Applications” (2013). ISBN 9789351071020
Top500 Homepage. https://www.top500.org/. Accessed 02 Jan 2018
The OrangeFS Project, OrangeFS 2.9 Documentation, Orangefs.org. http://docs.orangefs.com/v_2_9/index.htm. Accessed 09 Feb 2018
William Gropp, Ewing Lusk and Anthony Skjellum: Using MPI, 3rd Edition (2014)
William Gropp, Torsten Hoefler, Rajeev Thakur and Ewing Lusk: Using Advance MPI, 1st Edition (2014)
Wu, Y., et al.: J. Phys.: Conf. Ser. 219, 062068 (2010). https://doi.org/10.1088/1742-6596/219/6/062068
Acknowledgments
We thank the Programa para el Desarrollo Profesional Docente (PRODEP) for the support granted mentioned in the Official Letter No. 511-6/17/8212, and the Universidad Autónoma de Tamaulipas - Facultad de Medicina e Ingeniería en Sistemas Computacionales de Matamoros, all of them for providing the means to carry out this work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Cruz, H.E.C., Peña, J.H.F., Mariño, J.C.G., de Lourdes Cantú Gallegos, M. (2019). Evaluation of OrangeFS as a Tool to Achieve a High-Performance Storage and Massively Parallel Processing in HPC Cluster. In: Torres, M., Klapp, J., Gitler, I., Tchernykh, A. (eds) Supercomputing. ISUM 2018. Communications in Computer and Information Science, vol 948. Springer, Cham. https://doi.org/10.1007/978-3-030-10448-1_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-10448-1_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-10447-4
Online ISBN: 978-3-030-10448-1
eBook Packages: Computer ScienceComputer Science (R0)