Cluster Computing

, Volume 9, Issue 4, pp 375–384 | Cite as

Performance of parallel communication and spawning primitives on a Linux cluster

  • David J. Johnston
  • Martin Fleury
  • Michael Lincoln
  • Andrew C. Downton


The Linux cluster considered in this paper, formed from shuttle box XPC nodes with 2 GHz Athlon processors connected by dual Gb Ethernet switches, is relatively easily constructed, but, while effective as a throughput engine, may result in disappointing results when running explicitly parallel software if weakly-performing communication mechanisms and process spawning are selected. This paper carefully compares the implementations of communication and spawning primitives in MPICH-2, openMosix, and Linux Remote Procedure Call, forking, and various lower-level communication mechanisms. The test selection compares the provision of both a message-passing library, and a single system image software package, with direct use of lower-level primitives. The information in the paper will be of interest to those considering the use of one of the well-known packages, or directly writing their own distributed applications, or constructing a distributed language by layering on top of an existing set of parallel primitives. The results expose a ranking in terms of process spawning and a similar ranking of communication software performance. They reveal poor performance in certain circumstances, well below the hardware specification, which it is as well that the developer is aware of. In general, the paper emphasizes the importance of efficient transport software to cluster machines.


Linux cluster Software benchmarking MOSIX 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    A. Geist, A. Beguelin, J. Dongarra, W. Jiang, R. Manchek, and V. Sunderam, PVM: Parallel Virtual Machine, A Users’ Guide and Tutorial for Networked Parallel Computing (MIT, Cambridge, MA, 1994).Google Scholar
  2. 2.
    C. Lin and L. Snyder, ZPL: An array sublanguage, in: 6th International Workshop on Languages and Compilers for Parallel Computing (1993) pp. 96–114.Google Scholar
  3. 3.
    J. H. Reppy, Concurrent Programming in ML (Cambridge University Press, Cambridge, UK, 1999).MATHGoogle Scholar
  4. 4.
    D. J. Johnston, M. Fleury, and A. C. Downton, Prototyping application models in concurrent ML, in: Euro-Par 2003 Parallel Processing (Springer, Berlin, 2003), pp. 750–759.Google Scholar
  5. 5.
    W. Gropp, E. Lusk, and R. Thakur, Using MPI-2: Advanced Features of the Message-Passing Interface (MIT, Cambridge, MA, 1999).Google Scholar
  6. 6.
    T. Sterling, Beowulf Cluster Computing with Linux (MIT, Cambridge, MA, 2002).Google Scholar
  7. 7.
    G. A. Geist, J. A. Kohl, and P. M. Papadopoulos, PVM and MPI: a comparison of features, Calculateurs Paralleles 8(2) (1996) 137–150.Google Scholar
  8. 8.
    A. Barak, O. La’dan, and A. Shiloh, Scalable cluster computer with MOSIX on LINUX, in: Linux Expo’99 (1999) pp. 95–100.Google Scholar
  9. 9.
    A. Barak, S. Guday, and R Wheeler, The MOSIX Distributed Operating System, Load Balancing for UNIX (Springer-Verlag, Berlin, 1993).MATHGoogle Scholar
  10. 10.
    M. A. Baker, G. C. Fox, and H. W. Yau, A review of commercial and research cluster management software packages. NHSE Review Electronic Journal, 1(1) (1996), at http://nhse.cs. Scholar
  11. 11.
    A. Barak and O. La’adan, The MOSIX multicomputer operating system for high performance cluster computing, Journal of Future Generation Computer Systems 13(4–5) (1998) 361–372.CrossRefGoogle Scholar
  12. 12.
    M. J. Rochkind, Advanced UNIX Programming, 2nd edition, (Addison-Wesley, Boston, 2004).Google Scholar
  13. 13.
    W. R. Stevens, UNIX Network Programming, Interprocess Communication, 2nd edition, (Prentice Hall, Upper Saddle River, NJ, 1999) Vol. 2.Google Scholar
  14. 14.
    R. W. Hockney, The Science of Computer Benchmarking (SIAM, Philadelphia, PA, 1996).MATHGoogle Scholar
  15. 15.
    I. Pyarali, T. H. Harrison, and D. C. Schmidt, Design and performance of an object-oriented framework for high-speed electronic medical imaging, USENIX Computing Systems 9(3) (1996) 265–298.Google Scholar
  16. 16.
    D. C. Schmidt and T. Suda, Transport system architectures for high-performance communication systems. IEEE Journal on Selected Areas in Communication 11(4) (1993) 489–506.CrossRefGoogle Scholar
  17. 17.
    T. Sterling, Node hardware, in: Beowulf Cluster Computing with Linux. (MIT, Cambridge, MA, 2002) pp. 31–60.Google Scholar
  18. 18.
    D. Ridge, D. Becker, P. Merkey, and T. Sterling, Beowulf: Harnessing the power of parallelism in a pile-of-PCs, IEEE Aerospace 2 (1997) 79–91.Google Scholar
  19. 19.
    R. Breyer and S. Riley, Switched, Fast, and Gigabit Ethernet (Macmillan, San Francisco, CA, 1999).Google Scholar
  20. 20.
    T. Sterling, Network hardware, in: Beowulf Cluster Computing with Linux (MIT, Cambridge, MA, 2002) pp. 113–130.Google Scholar
  21. 21.
    T. H. Dunigan Jr., J. S. Vetter, J. B. White III, and P. H. Worley, Performance evaluation of the Cray XI distributed shared-memory architecture. IEEE Micro 25(1) (2005) 30–40.Google Scholar
  22. 22.
    M. Bar, openMOSIX, an open source Linux cluster project, (2002), at Scholar
  23. 23.
    D. Ashton, W. Gropp, E. Lusk, R. Ross, and B. Ronen, MPICH2 design document. Technical report, Argonne National Laboratory (2003). Report # ANL/MCS-TM-00.Google Scholar
  24. 24.
    W. R. Stevens, UNIX Network Programming: Networking APIs: Sockets and XTI, 2nd edition (Prentice Hall, Upper Saddle River, NJ, 1998).Google Scholar
  25. 25.
    N. Nupairoj and L. Ni, Performance evaluation of some MPI implementations on workstation clusters, in: Scalable Parallel Libraries Conference (1994) pp. 98–105.Google Scholar
  26. 26.
    R. Chandra, L. Dagum, D. Kohr, D. Maydan, J. McDonald, and R. Menon, Parallel Programming in OpenMP (Morgan Kaufmann, San Francisco, CA, 2001).Google Scholar
  27. 27.
    J. Peacock, Gently down the STREAMS, UNIX Review 9 (1992) 33–38.Google Scholar
  28. 28.
    D. Ritchie, A stream input-output system, AT&T Bell Labs Technical Journal 63 (1984) 311–324.Google Scholar
  29. 29.
    W. R. Stevens, UNIX Network Programming, 2nd edition, Sockets and XTI (Prentice Hall, Upper Saddle River, NJ, 1999) Vol. 2.Google Scholar
  30. 30.
    M. Snir, S. W. Otto, S. Huss-Lederman, D. W. Walker, and J. Dongarra, MPI—The Complete Reference: The MPI Core. 2nd edition (MIT, Cambridge, MA, 1998) Vol. 1.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  • David J. Johnston
    • 1
  • Martin Fleury
    • 1
  • Michael Lincoln
    • 1
  • Andrew C. Downton
    • 1
  1. 1.Department of Electronic Systems EngineeringUniversity of EssexEssexUK

Personalised recommendations