The Data Mover: A Machine-Independent Abstraction for Managing Customized Data Motion

  • Scott B. Baden
  • Stephen J. Fink
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1863)


This paper discusses the Data Mover, an abstraction for expressing machine-independent customized communication algorithms arising in block-structured computations. The Data Mover achieves performance that is competitive with hand-coding in MPI, but enables application-specific optimization to be expressed using intuitive geometric set operations that encapsulate low-level details.


Data Mover Message Passing Interface Member Function Multidimensional Array Message Passing Interface Implementation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fink, S.J.: Hierarchical Programming for Block-Structured Scientific Calculations. Doctor dissertation, Dept. of Computer Science and Engineering, University of California, San Diego (1998)Google Scholar
  2. 2.
    Baden, S.B. and Fink, S.J.: Communication Overlap in Multi-tier Parallel Algorithms. In Proc. SC’ 98, IEEE Computer Society Press (1998)
  3. 3.
    Fink, S.J. and Baden, S.B.: Runtime Support for Multi-tier Programming of Block-Structured Applications on SMP Clusters. In: Ishikawa, Y., Oldehoeft, R, Reyders, J.V.W., and Tholburn, M. (eds.): Scientific Computing in Object-Oriented Parallel Environments. Lecture Notes in Computer Science, Vol. 1343. Springer-Verlag, Berlin Heidelberg New York (1997) 1–8Google Scholar
  4. 4.
    Baden, S.B. and Fink, S. J., and Kohn, S. R.: Efficient Run-Time Support for Irregular Block-Structured Applications. J. Parallel Distrib. Comput. 50(1998) 61–82zbMATHCrossRefGoogle Scholar
  5. 5.
    Howe, J., Baden, S. B., Grimmett, T., and Nomura, K.: Modernization of Legacy Application Software. In Kågström, B., Dongarra, J, Elmroth, E., and Wasnie-wski, J. (eds.): Applied Parallel Computing: Large Scale Scientific and Industrial Problems: 4th International Workshop. Lecture Notes in Computer Science, Vol. 1541. Springer-Verlag, Berlin, Heidelberg New York (1997) 255–262CrossRefGoogle Scholar
  6. 6.
    Kohn, S.R. and Baden, S.B.: A Parallel Software Infrastructure for Structured Adaptive Mesh Methods. In Proc. Supercomputing’ 95, IEEE Computer Society Press (1995)
  7. 7.
    Kohn, S.R., Weare, J. H., Ong, M.E.G., and Baden, S. B.: Software Abstractions and Computational Issues in Parallel Structured Adaptive Mesh Methods for Electronic Structure Calculations. In: Baden, S. B., Chrisochoides, N., Norman, M., and Gannon, D. (eds.): Workshop on Structured Adaptive Mesh Refinement Grid Methods. Lecture Notes in Mathematics, in press. Springer-Verlag, Berlin, Heidelberg New York (1999)Google Scholar
  8. 8.
    Saltz, J., Sussman, A., Graham, S., Demmel, J., Baden, S., and Dongarra, J.: Programming Tools and Environments. Communications of the ACM 41 (1998) 64–73Google Scholar
  9. 9.
    J. Choi, A. Cleary, J. Demmel, I. Dhillon, J. Dongarra, S. Hammarling, G. Henry, S. Ostrouchov, A. Petitet, K. Stanley, D. Walker, and R. C. Whaley: ScaLAPACK: A Portable Linear Algebra Library for Distributed Memory Computers-Design Issues and Performance. In Proc. Supercomputing’ 96, IEEE Computer Society Press (1996)
  10. 10.
    Kumar, V., Grama, A., Gupta, A., and Karypis, G: Introduction to Parallel Computing: Design and Analysis of Algorithms. Benjamin Cummings (1993)Google Scholar
  11. 11.
    Foster, I. Designing and Building Parallel Programs. Addison-Wesley (1995)Google Scholar
  12. 12.
    MPI Forum: The Message Passing Interface (MPI) Standard. (1995)
  13. 13.
    Sohn, A. and Biswas, R.: Communication Studies of DMP and SMP Machines. Technical Report, NAS, NASA Ames Research Center (1997)Google Scholar
  14. 14.
    MPI Forum: MPI-2: Extensions to the Message-Passing Interface. (1997)
  15. 15.
    Gropp, W.W. and Lusk, E. L.: A Taxonomy of Programming Models for Symmetric Multiprocessors and SMP Clusters. In Giloi, W. K. and Jahnichen, S., and Shri-ver, B. D. (eds.): Programming Models for Massively Parallel Computers. IEEE Computer Society Press (1995) 2–7Google Scholar
  16. 16.
    Lumetta, S.S, Mainwaring, A.M. and Culler, D. E. Multi-Protocol Active Messages on a Cluster of SMPs. In Proc. SC’ 97, IEEE Computer Society Press (1997)
  17. 17.
    IBM Corp: Understanding and Using the Communication Low-Level Application Programming Interface (LAPI). In IBM Parallel System Support Programs for AIX Administration Guide, GC23-3897-04, (1997)
  18. 18.
    P. Buonadonna, A. Geweke, and D. Culler. An Implementation and Analysis of the Virtual Interface Architecture. In Proc. SC’ 98, IEEE Computer Society Press (1998)
  19. 19.
    Agrawal, A., Sussman, A, and Saltz, J.: An Integrated Runtime and Compile-Time Approach for Parallelizing Structured and Block Structured Applications. IEEE Transactions on Parallel and Distributed Systems 6 (1995) 747–754CrossRefGoogle Scholar
  20. 20.
    Foster, I., and Karonis, N.T.: A Grid-Enabled MPI: Message Passing in Heterogeneous Distributed Computing Systems. In Proc. SC’ 98, IEEE Computer Society Press (1998)
  21. 21.
    NAS, NASA Ames Research Center: The NAS Parallel Benchmarks. (1997).
  22. 22.
    Geign, R.v.d., and Watts, J.: SUMMA: Scalable Universal Matrix Multiplication Algorithm. Concurrency: Practice and Experience, 9(1997) 255–74CrossRefGoogle Scholar
  23. 23.
    Carpenter, B., Zhang, G., and Wen, Y.: NPAC PCRC Runtime Kernel (Adlib) Definition. Northeast Parallel Architectures Center, Syracuse Univ., (1998)
  24. 24.
    Chamberlain, B. L. Choi, S.-E., and Snyder, L.: A Compiler Abstraction for Machine Independent Parallel Communication Generation. In Li, Z., et al. (eds): Proc. Workshop on Languages and Compilers for Parallel Computation (1997) 261–276Google Scholar
  25. 25.
    Bala, V., et al.: CCL: A Portable and Tunable Collective Communication Library for Scalable Parllel Computers. IEEE. Trans. On Parallel and Distributed Sys., 6(1995) 154–164CrossRefGoogle Scholar
  26. 26.
    Mukherjee, S.S., et al.: Efficient Support for Irregular Applications on Distributed Memory Machines. In 5th SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP) (1995) 68–79Google Scholar
  27. 27.
    Culler, D., Dusseau, A, Goldstein, S., Krishnamurthy, A., Lumetta, S, von Eicken, T., and Yelick, K. Parallel Programming in Split-C. In Conf. Proc. Supercomputing’ 93. IEEE Computer Society Press (1993)Google Scholar
  28. 28.
    Nieplocha, J. and Carpenter, B.: ARMCI: A Portable Remote Memory Copy Library for Distributed Array Libraries and Compiler Run-time Systems. In Proc. 3rd Workshop on Runtime Systems for Parallel Programming (RTSPP) of International Parallel Processing Symposium IPPS/SPDP’ 99, San Juan, Puerto Rico (1999)Google Scholar
  29. 29.
    Donaldson, S., Hill, J., and Skillicorn, D.: BSP Clusters: High-Performance, Reliable and Very Low Cost. Oxford University Computing Laboratory Technical Report PRG-5-98, (1998).

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Scott B. Baden
    • 1
  • Stephen J. Fink
    • 2
  1. 1.Department of Computer Science and EngineeringUniversity of CaliforniaSan DiegoLa JollaUSA
  2. 2.IBM Thomas J. Watson Research CenterYorktown HeightsUSA

Personalised recommendations