Abstract
In the last decade, we witnessed an increasing interest in High Performance Computing (HPC) infrastructures, which play an important role in both academic and industrial research projects. At the same time, due to the increasing amount of available data, we also witnessed the introduction of new frameworks and applications based on the MapReduce paradigm (e.g., Hadoop). Traditional HPC systems are usually designed for CPU- and memory-intensive applications. However, the use of already available HPC infrastructures for data-intensive applications is an interesting topic, in particular in academia where the budget is usually limited and the same cluster is used by many researchers with different requirements. In this paper, we investigate the integration of Hadoop, and its performance, in an already existing low-budget general purpose HPC cluster characterized by heterogeneous nodes and a low amount of secondary memory per node.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Della Croce, F., Piccolo, E., Nepote, N.: A terascale, cost-effective open solution for academic computing: early experience of the dauin hpc initiative. In: AICA 2011, pp. 1–9 (2011)
Dongarra, J.: Trends in high performance computing: a historical overview and examination of future developments. IEEE Circuits and Devices Magazine 22(1), 22–27 (2006)
Maier, P.: qsort.c (2010), http://www.macs.hw.ac.uk/~pm175/F21DP2/src/
Nepote, N., Piccolo, E., Demartini, C., Montuschi, P.: Why and how using HPC in university teaching? a case study at polito. In: DIDAMATICA 2013, pp. 1019–1028 (2013)
White, T.: Hadoop: The Definitive Guide, 1st edn. O’Reilly Media, Inc. (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Garza, P., Margara, P., Nepote, N., Grimaudo, L., Piccolo, E. (2014). Hadoop on a Low-Budget General Purpose HPC Cluster in Academia. In: Catania, B., et al. New Trends in Databases and Information Systems. Advances in Intelligent Systems and Computing, vol 241. Springer, Cham. https://doi.org/10.1007/978-3-319-01863-8_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-01863-8_21
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-01862-1
Online ISBN: 978-3-319-01863-8
eBook Packages: EngineeringEngineering (R0)