The Effect of the 2nd Generation Clusters: Changes in the Parallel Programming Paradigms

  • Jari Porras
  • Pentti Huttunen
  • Jouni Ikonen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3038)


Programming paradigms for networks of symmetric multiprocessor (SMP) workstation (2 nd generation of clusters) are discussed and a new paradigm is introduced. The SMP cluster environments are explored in regard to their advantages and drawbacks with a special focus on memory architectures and communication. The new programming paradigm provides a solution to write efficient parallel applications for the 2 nd generation of clusters. The paradigm aims at improving the overlap of computation and communication and the locality of communication operations. The preliminary results with large message sizes indicate improvements in excess of 30% over traditional MPI implementations.


  1. 1.
    Bader, D.A., Jájá, J.: SIMPLE: A Methodology for Programming High Performance Algorithms on Clusters of Symmetric Multiprocessors (SMPs). Journal of Parallel and Distributed Computing 58, 92–108 (1999)CrossRefGoogle Scholar
  2. 2.
    Chowdappa, A.K., Skjellum, A., Doss, N.E.: Thread-Safe Message Passing with P4 and MPI. Technical Report TR-CS-941025, Computer Science Department and NSF Engineering Research Center, Mississippi State University (1994)Google Scholar
  3. 3.
    Clusters@TOP500 list,
  4. 4.
    Djomehri, M.J., Jin, H.H.: Hybrid MPI+OpenMP Programming of a Overset CFD Solver and Performance Investigations. NAS Technical Report NAS-02-002 (2002)Google Scholar
  5. 5.
    Geist, A., et al.: PVM: Parallel Virtual Machine. MIT Press, Cambridge (1994)zbMATHGoogle Scholar
  6. 6.
    He, Y., Ding, C.H.Q.: MPI and OpenMP Paradigms on Clusters of SMP Architectures: the Vacancy Tracking Algorithm for Multi-Dimensional ArrayGoogle Scholar
  7. 7.
    Hu, Y., Lu, H., Cox, A., Zwaenepoel, W.: OpenMP for Networks of SMPs. Journal of Parallel and Distributed Computing 60, 1512–1530 (2000)zbMATHCrossRefGoogle Scholar
  8. 8.
    Kee, Y.-S., Kim, J.-S., Ha, S.: ParADE: An OpenMP Programming Environment for SMP Cluster Systems. Proceedings of Supercomputing 2003 (2003)Google Scholar
  9. 9.
    Leng, T., Ali, R., Hsieh, J., Mashayekhi, V., Rooholamini, R.: An Empirical Study of Hyper-Threading in High Performance Computing Clusters. In: Proceedings of LCI International Conference on Linux Clusters: The HPC Revolution 2002 (2002)Google Scholar
  10. 10.
    Pancheco, P.: Parallel Programming with MPI. Morgan Kaufmann, San Francisco (1997)Google Scholar
  11. 11.
    Protopopov, B., Skjellum, A.: A multithreaded message passing interface (MPI) Architecture: Performance and Program Issues. Journal of Parallel and Distributed Computing 61, 449–466 (2001)zbMATHCrossRefGoogle Scholar
  12. 12.
    Rauber, T., Runger, G., Trautmann, S.: A Distributed Hierarchical Programming Model for Heterogeneous Cluster of SMPs. In: Proceedings of the International Parallel and Distributed Processing Symposium, pp. 381–392 (2003)Google Scholar
  13. 13.
    Smith, L., Bull, M.: Development of mixed mode MPI / OpenMP Applications. Scientific Programming 9, 83–98 (2001)Google Scholar
  14. 14.
    Tang, H., Yang, T.: Optimizing Threaded MPI Execution on SMP Clusters. In: Proceedings of Supercomputing 2001, pp. 381–392 (2001)Google Scholar
  15. 15.
    Tang, H., Shen, K., Yang, T.: Compile/Run-time Support for Threaded MPI Execution on Multiprogrammed Shared Memory Machines. In: Proceedings of Programming Principles of Parallel Processing, pp. 107–118 (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Jari Porras
    • 1
  • Pentti Huttunen
    • 1
  • Jouni Ikonen
    • 1
  1. 1.Lappeenranta University of TechnologyLappeenrantaFinland

Personalised recommendations