Abstract
The technique of user-level communication, where incoming messages wait in a busy loop, is used in most MPI implementations to achieve high communication performance. However, in some cases a kernel-level blocking receive is preferred. Some MPI implementations have an option to switch from user-level to kernel-level blocking with the sacrifice of communication performance. This paper identifies the problems when implementing kernel-level blocking receiving and proposes several techniques to avoid these problems. Evaluations show that the proposed kernel-level blocking techniques may achieve comparable performance with user-level communication.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
von Eicken, T., Basu, A., Buch, V., Vogels, W.: U-net: a user-level network interface for parallel and distributed computing. SIGOPS Oper. Syst. Rev. 29, 40–53 (1995)
von Eicken, T., Culler, D.E., Goldstein, S.C., Schauser, K.E.: Active messages: a mechanism for integrated communication and computation. In: Proceedings of the 19th Annual International Symposium on Computer Architecture, ISCA 1992, pp. 256–266. ACM, New York (1992)
Pakin, S., Karamcheti, V., Chien, A.A.: Fast messages: Efficient, portable communication for workstation clusters and mpps. IEEE Parallel Distrib. Technol. 5, 60–73 (1997)
Tezuka, H., Hori, A., Ishikawa, Y., Sato, M.: Pm: An Operating System Coordinated High Performance Communication Library. In: Hertzberger, B., Sloot, P.M.A. (eds.) HPCN-Europe 1997. LNCS, vol. 1225, pp. 708–717. Springer, Heidelberg (1997)
Ousterhout, J.: Scheduling techniques for concurrent systems. In: 3rd International Conference on Distributed Computing Systems, pp. 22–30. IEEE (1982)
Liu, J., Wu, J., Kini, S.P., Wyckoff, P., Panda, D.K.: High performance RDMA-based MPI implementation over InfiniBand. In: Proceedings of the 17th Annual International Conference on Supercomputing, ICS 2003, pp. 295–304. ACM, New York (2003)
PC Cluster Consortium: SCore, http://www.pccluster.org/
Hori, A.: PMX Specification –DRAFT–, http://www.pccluster.org/score_doc/score-7.0.2/pdf/PMX-spec.pdf
NASA: NAS Parallel Benchmarks, http://www.nas.nasa.gov/Resources/Software/npb.html
MVAPICH Team: MVAPICH2 1.7 User Guide (2012)
OpenFabrics Alliance: OFED, http://www.openfabrics.org/
Damianakis, S., Chen, Y., Felten, E.W.: Reducing waiting costs in user-level communication. In: 11th International Parallel Processing Symposium, pp. 381–387. IEEE Computer Society Press (1997)
Vishnu, A., Song, S., Marquez, A., Barker, K., Kerbyson, D., Cameron, K., Balaji, P.: Designing energy efficient communication runtime systems for data centric programming models. In: Proceedings of the 2010 IEEE/ACM Int’l Conference on Green Computing and Communications & Int’l Conference on Cyber, Physical and Social Computing, GREENCOM-CPSCOM 2010, pp. 229–236. IEEE Computer Society, Washington, DC (2010)
Nieplocha, J., Tipparaj, V., Krishnan, M., Panda, D.K.: High performance remote memory access communication: The armci approach. Int. J. High Perform. Comput. Appl. 20(2), 233–253 (2006)
Message Passing Interface Forum: MPI-2: Extensions to the Message-Passing Interface (2003), http://www.mpi-forum.org/docs/mpi2-report.pdf
Intel Corporation: Intel 64 and IA-32 Architectures Software Developer’s Manual (2011)
Dongarra, J., Choudhary, A., Kale, S., et al.: The International Exascale Software Project Roadmap. White paper, Argonne National Laboratory (October 2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hori, A., Kameyama, T., Tsujita, Y., Namiki, M., Ishikawa, Y. (2012). An Efficient Kernel-Level Blocking MPI Implementation. In: Träff, J.L., Benkner, S., Dongarra, J.J. (eds) Recent Advances in the Message Passing Interface. EuroMPI 2012. Lecture Notes in Computer Science, vol 7490. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33518-1_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-33518-1_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33517-4
Online ISBN: 978-3-642-33518-1
eBook Packages: Computer ScienceComputer Science (R0)