On the distributed implementation of aggregate data structures by program transformation

  • Gabriele Keller
  • Manuel M. T. Chakravarty
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1586)


A critical component of many data-parallel programming languages are operations that manipulate aggregate data structures as a whole—this includes Fortran 90, Nesl, and languages based on BMF. These operations are commonly implemented by a library whose routines operate on a distributed representation of the aggregate structure; the compiler merely generates the control code invoking the library routines and all machine-dependent code is encapsulated in the library. While this approach is convenient, we argue that by breaking the abstraction enforced by the library and by presenting some of internals in the form of a new intermediate language to the compiler back-end, we can optimize on al levels of the memory hierarchy and achieve more flexible data distribution. The new intermediate language allows us to present these optimisations elegantly as program transformations. We report on first results obtained by our approach in the implementation of nested data parallelism on distributed-memory machines.


Local Computation Local Operation Aggregate Structure Memory Hierarchy Program Transformation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Guy E. Blelloch. Programming parallel algorithms. Communications of the ACM, 39(3):85–97, 1996.CrossRefGoogle Scholar
  2. 2.
    Guy E. Blelloch and Siddhartha Chatterjee. VCODE: A data-parallel intermediate language. In Proceedings Frontiers of Massively Parallel Computation, pages 471–480, October 1990.Google Scholar
  3. 3.
    Guy E. Blelloch, Siddhartha Chatterjee, Jonathan C. Hardwick, Margaret Reid-Miller, Jay Sipelstein, and Marco Zagha. CVL: A C vector library. Technical Report CMU-CS-93-114, Carnegie Mellon University, 1993.Google Scholar
  4. 4.
    Guy E. Blelloch, Siddhartha Chatterjee, Jonathan C. Hardwick, Jay Sipelstein, and Marco Zagha. Implementation of a portable nested data-parallel language. In 4th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 1993.Google Scholar
  5. 5.
    Guy E. Blelloch and Gary W. Sabot. Compiling collection-oriented languages onto massively parallel computers. Journal of Parallel and Distributed Computing, 8:119–134, 1990.CrossRefGoogle Scholar
  6. 6.
    George Horatiu Botorog and Herbert Kuchen. Using algorithmic skeletons with dynamic data structures. In IRREGULAR 1996, volume 1117 of Lecture Notes in Computer Science, pages 263–276. Springer Verlag, 1996.Google Scholar
  7. 7.
    S. Chatterjee. Compiling nested data-parallel programs for shared-memory multiprocessors. ACM Transactions on Programming Languages and Systems, 15(3):400–462, july 1993.CrossRefGoogle Scholar
  8. 8.
    S. Chatterjee, Jan F. Prins, and M. Simons, Expressing irregular computations in modern Fortran dialects. In Fourth Workshop on Languages, Compilers, and Run-Time Systems for Scalable Computers, Lecture Notes in Computer Science. Springer Verlag, 1998.Google Scholar
  9. 9.
    J. Darlington, A. J. Field, P. G. Harrison, P. H. J. Kelly, D. W. N. Sharp, Q. Wu, and R. L. While. Parallel programming using skeleton functions. In A. Bode, M. Reeve, and G. Wolf, editors, PARLE ’93: Parallel Architectures and Languages Europe, number 694 in Lecture Notes in Computer Science, pages 146–160, Berlin, Germany, 1993. Springer-Verlag.Google Scholar
  10. 10.
    High Performance Fortran Forum. High Performance Fortran language specification. Technical report, Rice University, 1993. Version 1.0.Google Scholar
  11. 11.
    Andrew J. Gill, John Launchbury, and Simon L. Peyton Jones. A short cut to deforestation. In Arvind, editor, Functional Programming and Computer Architecture, pages 223–232. ACM, 1993.Google Scholar
  12. 12.
    S. Gorlatch. Systematic efficient parallelization of scan and other list homomorphisms. In L. Bouge, P. Fraigniaud, A. Mignotte, and Y. Robert, editors, Euro-Par’96, Parallel Processing, number 1124 in Lecture Notes in Computer Science, pages 401–408. Springer-Verlag, 1996.Google Scholar
  13. 13.
    William Gropp, Steven Huss-Lederman, Andrew Lumsdaine, Ewing Lusk, Bill Nitzberg, William Saphir, and Marc Snir. MPI: The Complete Reference, volume 2—The MPI-2 Extensions. The MIT Press, second edition, 1998.Google Scholar
  14. 14.
    Jonathan C. Hardwick. An efficient implementation of nested data parallelism for irregular divide-and-conquer algorithms. In First International Workshop on High-Level Programming Models and Supportive Environments, 1996.Google Scholar
  15. 15.
    C. Barry Jay. Costing parallel programs as a function of shapes. Invited submission to Science of Computer Programming, September 1998.Google Scholar
  16. 16.
    G. Keller and M. Simons. A calculational approach to flattening nested data parallelism in functional languages. In J. Jaffar, editor, The 1996 Asian Computing Science Conference, Lecture Notes in Computer Science. Springer Verlag, 1996.Google Scholar
  17. 17.
    Gabriele Keller. Transformation-based Implementation of Nested Data Parallelism for Distributed Memory Machines. PhD thesis, Technische Universität Berlin, Fachbereich Informatik, 1998. To appear.Google Scholar
  18. 18.
    Michael Medcalf and John Reid. Fortran 90 Explained. Oxford Science Publications, 1990.Google Scholar
  19. 19.
    Y. Onue, Zhenjiang Hu, Hideya Iwasaki, and Masato Takeichi. A calculational fusion system HYLO. In R. Bird and L. Meertens, editors. Proceedings IFIP TC 2 WG 2.1 Working Conf. on Algorithmic Languages and Calculi, Le Bischenberg, France, 17–22 Feb 1997, pages 76–106. Chapman & Hall, London, 1997.Google Scholar
  20. 20.
    Jan Prins and Daniel Palmer. Transforming high-level data-parallel programs into vector operations. In Proceedings of the Fourth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pages 119–128, San Diego, CA., May 19–22, 1993. ACM.CrossRefGoogle Scholar
  21. 21.
    Jay M. Sipelstein and Guy E. Blelloch. Collection-oriented languages. Proceedings of the IEEE, 79(4):504–523, April 1991.CrossRefGoogle Scholar
  22. 22.
    D. Skillicorn and W. Cai. A cost calculus for parallel functional programming. Journal of Parallel and Distributed Computing, 28:65–83, 1995.zbMATHCrossRefGoogle Scholar
  23. 23.
    D.B. Skillicorn. Foundations of Parallel Programming. Cambridge Series in Parallel Computation 6. Cambridge University Press, 1994.Google Scholar
  24. 24.
    Akihiko Takano and Erik Meijer. Shortcut deforestation in calculational form. In Conf. Record 7th ACM SIGPLAN/SIGARCH Intl. Conf. on Functional Programming Languages and Computer Architecture, pages 306–316. ACM Press, New York, 1995.CrossRefGoogle Scholar
  25. 25.
    Philip Wadler. Deforestation: Transforming programs to eliminate trees. Theoretical Computer Science, 73:231–248, 1990.zbMATHMathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 1999

Authors and Affiliations

  • Gabriele Keller
    • 1
  • Manuel M. T. Chakravarty
    • 2
  1. 1.Fachbereich InformatikTechnische Universität BerlinGermany
  2. 2.Inst. of Inform. Sciences and ElectronicsUniversity of TsukubaJapan

Personalised recommendations