Abstract
Memory copies in messaging systems can be a major source of performance degradation in cluster computing. In this paper we discuss a system which can offload a host CPU from most of the overhead of copying data between distinct regions in the host physical memory. The sistem is implemented as a special-purpose Linux device driver operating a generic, non-programmable Gigabit Ethernet adapter connected to itself. Whenever the descriptor-based DMA engines of the adapter are instructed to start a data communication, the data are read from the host memory and written to the memory itself thanks to the loopback cable; this is semantically equivalent to a non-blocking memory copy operation performed by the two DMA engines. Suitable completion test/waiting routines are also implemented, in order to provide traditional, blocking semantics in a split-phase fashion. An implementation of MPI using this system in place of traditional memcpy() calls on receive shows a significantly lower receive overhead.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chiola, G., Ciaccio, G.: Efficient Parallel Processing on Low-cost Clusters with GAMMA Active Ports. Parallel Computing (26), 333–354 (2000)
Ciaccio, G.: MPI/GAMMA home page, http://www.disi.unige.it/project/gamma/mpigamma/
Geoffray, P., Prylli, L., Tourancheau, B.: BIP-SMP: High performance message passing over a cluster of commodity SMPs. In: Proc. of 11th IEEE - ACM High Performance Networking and Computing Conference, SC 1999 (1999)
Lauria, M., Chien, A.: MPI-FM: High Performance MPI on Workstation Clusters. Journal of Parallel and Distributed Computing 40(1), 4–18 (1997)
Myricom. GM performance (2000), http://www.myri.com/myrinet/performance/
Pakin, S., Lauria, M., Chien, A.: High Performance Messaging on Workstations: Illinois Fast Messages (FM) for Myrinet. In: Proc. Supercomputing 1995, San Diego, California (1995)
Pratt, I., Fraser, K.: Arsenic: A User-Accessible Gigabit Ethernet Interface. In: Proc. Infocom 2001, Anchorage, Alaska, April 2001, IEEE, Los Alamitos (2001)
Prylli, L., Tourancheau, B., Westrelin, R.: The design for a high performance MPI implementation on the myrinet network. In: Margalef, T., Dongarra, J., Luque, E. (eds.) PVM/MPI 1999. LNCS, vol. 1697, pp. 223–230. Springer, Heidelberg (1999)
Prylli, L., Tourancheau, B.: BIP: a new protocol designed for high performance networking on Myrinet. In: Rolim, J.D.P. (ed.) IPPS-WS 1998 and SPDP-WS 1998. LNCS, vol. 1388, pp. 472–485. Springer, Heidelberg (1998)
Shivam, P., Wyckoff, P., Panda, D.: EMP: Zero-copy OS-bypass NIC-driven Gigabit Ethernet Message Passing. In: Proc. 2001 International Conference on Supercomputing (SC 2001), Denver, Colorado (November 2001)
Takahashi, T., Sumimoto, S., Hori, A., Harada, H., Ishikawa, Y.: PM2: High Performance Communication Middleware for Heterogeneous Network Environments. In: Hertzberger, B., Sloot, P.M.A. (eds.) HPCN-Europe 1997. LNCS, vol. 1225, pp. 708–717. Springer, Heidelberg (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ciaccio, G. (2003). Using a Self-connected Gigabit Ethernet Adapter as a memcpy() Low-Overhead Engine for MPI. In: Dongarra, J., Laforenza, D., Orlando, S. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2003. Lecture Notes in Computer Science, vol 2840. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39924-7_37
Download citation
DOI: https://doi.org/10.1007/978-3-540-39924-7_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20149-6
Online ISBN: 978-3-540-39924-7
eBook Packages: Springer Book Archive