Abstract
Using the first commercially available 100 Gbps Ethernet technology with a link of varying length, we have evaluated the performance of the Lustre file system and its networking layer under different latency scenarios. The results led us to a better understanding of the impact that the network latency has on Lustre’s performance. In particular spanning Lustre’s networking layer, striped small I/O, and the parallel creation of files inside a common directory. The main contribution of this work is the derivation of useful rules of thumbs to help users and system administrators predict the variation in Lustre’s performance produced as a result of changes in the latency of the I/O network.
Chapter PDF
References
NASA. Key Science System Metrics (2010), http://earthdata.nasa.gov/about-eosdis/performance
Gentzsch, W.: DEISA, The Distributed European Infrastructure for Supercomputing Applications. In: Gentzsch, W., Grandinetti, L., Joubert, G.R. (eds.) High Performance Computing Workshop. Advances in Parallel Computing, vol. 18, pp. 141–156. IOS Press (2008)
Simms, S.C., Davy, M., Hammond, B., Link, M., Stewart, C., Bramley, R., Plale, B., Gannon, D., Baik, M.-H., Teige, S., Huffman, J., McMullen, R., Balog, D., Pike, G.: All in a Day’s Work: Advancing Data-Intensive Research with the Data Capacitor. In: Proceedings of the 2006 ACM/IEEE Conference on Supercomputing, SC 2006. ACM, New York (2006)
Simms, S.C., Pike, G.G., Balog, D.: Wide Area Filesystem Performance Using Lustre on the TeraGrid. In: Proceedings of the TeraGrid 2007 Conference (2007)
Simms, S.C., Pike, G.G., Teige, S., Hammond, B., Ma, Y., Simms, L.L., Westneat, C., Balog, D.A.: Empowering Distributed Workflow with the Data Capacitor: Maximizing Lustre Performance Across the Wide Area Network. In: Proceedings of the 2007 Workshop on Service-Oriented Computing Performance: Aspects, Issues, and Approaches, SOCP 2007, pp. 53–58. ACM, New York (2007)
Andrews, P., Kovatch, P., Jordan, C.: Massive High-Performance Global File Systems for Grid Computing. In: Proceedings of the 2005 ACM/IEEE Conference on Supercomputing, SC 2005, p. 53. IEEE Computer Society, Washington, DC (2005)
Filizetti, J.: Lustre Performance over the InfiniBand WAN. In: Proceedings of the 2010 Lustre User Group (2010)
Michael, S., Simms, S., Breckenridge III, W.B., Smith, R., Link, M.: A Compelling Case for a Centralized Filesystem on the TeraGrid: Enhancing an Astrophysical Workflow with the Data Capacitor WAN as a Test Case. In: Proceedings of the 2010 TeraGrid Conference, TG 2010, pp. 13:1–13:7. ACM, New York (2010)
Cai, R., Curnutt, J., Gomez, E., Kaymaz, G., Kleffel, T., Schubert, K., Tafas, J.: A Scalable Distributed Datastore for BioImaging (2008), http://www.r2labs.org/pubs/BioinformaticsDatabase.ps
Rodriguez, J.L., Avery, P., Brody, T., Bourilkov, D., Fu, Y., Kim, B., Prescott, C., Wu, Y.: Wide Area Network Access to CMS Data Using the Lustre Filesystem. Journal of Physics: Conference Series 219(7), 072049 (2010)
Zhao, T., March, V., Dong, S., See, S.: Evaluation of a Performance Model of Lustre File System. In: 2010 Fifth Annual ChinaGrid Conference (ChinaGrid), pp. 191–196 (July 2010)
Oracle, Inc. Lustre 2.0 Operations Manual (2010), http://wiki.lustre.org/images/3/35/821-2076-10.pdf
Lakshman, T.V., Madhow, U.: The Performance of TCP/IP for Networks with High Bandwidth-Delay Products and Random Loss. IEEE/ACM Trans. Netw. 5, 336–350 (1997)
Shan, H., Shalf, J.: Using IOR to Analyze the I/O Performance for HPC Platforms. In: Cray User Group Conference (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Aguilera, A., Kluge, M., William, T., Nagel, W.E. (2012). HPC File Systems in Wide Area Networks: Understanding the Performance of Lustre over WAN. In: Kaklamanis, C., Papatheodorou, T., Spirakis, P.G. (eds) Euro-Par 2012 Parallel Processing. Euro-Par 2012. Lecture Notes in Computer Science, vol 7484. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32820-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-32820-6_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32819-0
Online ISBN: 978-3-642-32820-6
eBook Packages: Computer ScienceComputer Science (R0)