Advertisement

Architecture independent massive parallelization of divide-and-conquer algorithms

  • Klaus Achatz
  • Wolfram Schulte
Contributed Lectures
Part of the Lecture Notes in Computer Science book series (LNCS, volume 947)

Abstract

We present a strategy to develop, in a functional setting, correct, efficient and portable Divide-and-Conquer (DC) programs for massively parallel architectures. Starting from an operational DC program, mapping sequences to sequences, we apply a set of semantics preserving transformation rules, which transform the parallel control structure of DC into a sequential control flow, thereby making the implicit data parallelism in a DC scheme explicit. In the next phase of our strategy, the parallel architecture is fully expressed, where ‘architecture dependent’ higher-order functions are introduced. Then — due to the rising communication complexities on particular architectures — topology dependent communication patterns are optimized in order to reduce the overall communication costs. The advantages of this approach are manifold and are demonstrated with a set of non-trivial examples.

Keywords

Transformation Rule Data Parallelism Communication Operation Processor Element Length Preserve 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [AJ93]
    T. Axford and M. Joy. List processing primitives for parallel computation. Computer Languages, 19(1):1–12, 1993.Google Scholar
  2. [Akl89]
    S. G. Akl. The Design and Analysis of Parallel Algorithms. Prentice-Hall, 1989.Google Scholar
  3. [AS95]
    K. Achatz and W. Schulte. Architecture independent massive parallelization of devide-and-conquer algorithms. Technical Report 05-95, Universität Ulm, Fakultät für Informatik, April 1995.Google Scholar
  4. [Axf92]
    T. Axford. Crystal: The divide-and conquer paradigm as a basis for parallel language design. In L. Kronsjo and D. Shumsheruddin, editors, Advances in Parallel Algorithms, chapter 2. Blackwell, 1992.Google Scholar
  5. [Bat68]
    K. E. Batcher. Sorting networks and their applications. AFIPS Spring Joint Computer Conference, pages 307–314, 1968.Google Scholar
  6. [BGP93]
    E. A. Boiten, A. M. Geerling, and H. A. Partsch. Transformational derivation of (parallel) programs using skeletons. Technical Report 93-20, Katholieke Universiteit Nijmegen, September 1993. Also: Proceedings of Computer Science in the Netherlands 1993, Utrecht.Google Scholar
  7. [Bir89]
    R. Bird. Lectures on constructive functional programming. In M. Broy, editor, Constructive methods in computing science. NATO ASI Series. Series F: Computer and systems sciences 55, pages 151–216, Berlin, 1989. Springer-Verlag.Google Scholar
  8. [Ble92]
    G. E. Blelloch. NESL: A nested data-parallel language (version 2.0). Technical Report CMU-CS-93-129, School of Computer Science, Carnegie Mellon University, April 1992.Google Scholar
  9. [Ble93]
    G. E. Blelloch. Prefix sums and their applications. In J Reif, editor, Synthesis of Parallel Algorithms, chapter 1, pages 35–60. Morgan Kaufmann Publishers, 1993.Google Scholar
  10. [BW88]
    R. Bird and Ph. Wadler. An Introduction to Functional Programming. Prentice-Hall, 1988.Google Scholar
  11. [CC90]
    M. Chen and Y. Choo. Domain morphisms: A new construct for parallel programming and formalizing program optimization. Technical Report DCS/TR-817, Department of Computer Science, Yale University, August 1990.Google Scholar
  12. [CM91]
    B. Carpentieri and G. Mou. Compile-time transformations and optimizations of parallel divide-and conquer algorithms. ACM SIGPLAN Notices, 20(10):19–28, 1991.Google Scholar
  13. [Col89]
    M. Cole. Algorithmic Skeletons: Structured Management of Parallel Computation. MIT Press, 1989.Google Scholar
  14. [Col93]
    M. Cole. List homomorphic parallel algorithms for bracket matching. Technical Report CSR-29-93, Department of Computer Science, University of Edinburgh, August 1993.Google Scholar
  15. [DFH+93]
    J. Darlington, A. Field, P. Harrison, P. Kelly, D. Sharp, Q. Wu, and R. White. Parallel programming using skeleton functions. In A. Bode, M. Reeve, and G. Wolf, editors, PARLE'93 Parallel Architectures and Languages Europe, volume 694 of Lecture Notes in Computer Science, pages 146–160, 1993.Google Scholar
  16. [Fea87]
    M. S. Feather. A survey and classification of some program transformation approaches and techniques. In L.G.L.T. Meertens, editor, Program Specification and Transformation. North-Holland, 1987.Google Scholar
  17. [Fox89]
    G.C. Fox. Parallel computing comes of age: Supercomputer level parallel computations at caltech. Concurrency: Practice and Experience, 1(1):63–103, 1989.Google Scholar
  18. [Gee92]
    A. M. Geerling. Two examples of parallel-program derivation: Parallel-prefix and matrix multiplication. Technical Report DoC 92/33, Imperial College London, November 1992.Google Scholar
  19. [Gee93]
    A. M. Geerling. Formal derivation of SIMD parallelism from non-linear recursive specifications. Technical Report CSI-R9324, Katholieke Universiteit Nijmegen, September 1993.Google Scholar
  20. [Gee94]
    A.M. Geerling. Formal derivation of SIMD parallelism from non-linear recursive specifications. In B. Buchberger and J. Volkert, editors, CON-PAR'94 VAPP VI International Conference on Parallel and Vector Processing, pages 136–147. Springer-Verlag, 1994.Google Scholar
  21. [GL93]
    S. Gorlatch and C. Lengauer. Parallelization of divide-and conquer in the Bird-Meertens formalism. Technical Report 12/93, Fakultät für Mathematik und Informatik, Universität Passau, Dezember 1993.Google Scholar
  22. [JáJ92]
    J. JáJá. An introduction to parallel algorithms. Addison-Wesley, 1992.Google Scholar
  23. [MH88]
    Z.G. Mou and M. Hudak. An algebraic model for divide-and-conquer algorithms and its parallelism. Journal of Supercomputing, 2(3):257–278, 1988.Google Scholar
  24. [NS79]
    D. Nassimi and S. Sahni. Bitonic sort on a mesh-connected parallel computer. IEEE Transactions on Computers, 27(1):2–7, 1979.Google Scholar
  25. [Par90]
    H. A. Partsch. Specification and Transformation of Programs. Springer-Verlag, 1990.Google Scholar
  26. [Par93]
    H. Partsch. Some experiments in transforming towards parallel executability. In R. Paige, J. Reif, and R. Wachter, editors, Parallel Algorithm Derivation and Program Transformation. Kluwer Academic Publisher, 1993.Google Scholar
  27. [Pep93]
    P. Pepper. Deductive derivation of parallel programs. In R. Paige, J. Reif, and R. Wachter, editors, Parallel Algorithm Derivation and Program Transformation. Kluwer Academic Publishers, 1993. Also: Technical Report 92-23, Technische Universität Berlin, July 1992.Google Scholar
  28. [PES93]
    P. Pepper, J. Exner, and M. Südholt. Functional development of massively parallel programs. In D. Bjorner, M. Broy, and I.V. Pottosin, editors, Formal Methods in Programming and Their Applications. Proceedings International Conference Novosibirsk, June/July 1993, volume 735 of Lecture Notes in Computer Science, pages 217–238, Berlin, 1993. Springer-Verlag.Google Scholar
  29. [PH77]
    F. P. Preparata and S. J. Hong. Convex hulls of finite sets of points in two and three dimensions. Communications of The ACM, 20:88–93, 1977.Google Scholar
  30. [Ski93]
    D.B. Skillicorn. A cost calculus for parallel functional programming. Technical Report ISSN-0836-0227-93-348, Department of Computing and Information Science, Queen's University, March 1993.Google Scholar
  31. [Smi85]
    D.R. Smith. The design of divide-and-conquer algorithms. Science of Computer Programming, 5:37–58, 1985.Google Scholar
  32. [Smi93]
    D. R. Smith. Derivation of paralel sorting algorithms. In R. Paige, J. Reif, and R. Wachter, editors, Parallel Algorithm Derivation and Program Transformation. Kluwer Academic Publisher, 1993.Google Scholar
  33. [YC92]
    J. A. Yang and Y. Choo. Data fields as parallel programs. Technical Report CT 06520-2158, Department of Computer Science, Yale University, March 1992.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • Klaus Achatz
    • 1
  • Wolfram Schulte
    • 1
  1. 1.Fakultät für InformatikUniversität UlmGermany

Personalised recommendations