Scheduling Program Task Graphs on MIMD Architectures

  • Apostolos Gerasoulis
  • Tao Yang
Part of the The Springer International Series In Engineering and Computer Science book series (SECS, volume 231)


Scheduling is a mapping of parallel tasks onto a set of physical processors and a determination of the starting time of each task. In this paper, we discuss several static scheduling techniques used for distributed memory architectures. We also give an overview of a software system PYRROS [38] that uses the scheduling algorithms to generate parallel code for message passing architectures.


Communication Cost Dependence Graph Task Graph Priority List Parallel Time 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Adam, T., Chandy, K.M. and Dickson, J.R., ‘A Comparison of List Schedules for Parallel Processing Systems’, CACM, 17:12, 1974, pp. 685–690.MATHGoogle Scholar
  2. [2]
    Bokhari, S.H., ‘Assignment Problems in Parallel and Distributed Computing’, Kluwer Academic Publisher, 1990.Google Scholar
  3. [3]
    Callahan, D. and Kennedy, K., ‘Compiling Programs for Distributedmemory Multi-processors’, Journal of Supercomputing, Vol. 2, 1988, pp. 151–169.CrossRefGoogle Scholar
  4. [4]
    Chretienne, Ph., ‘Task Scheduling over Distributed Memory Machines’, Proc. of Inter. Workshop on Parallel and Distributed Algorithms, North Holland, 1989.Google Scholar
  5. [5]
    Cosnard, M., Marrakchi, M., Robert, Y. and Trystram, D., ‘Parallel Gaussian Elimination on an MIMD Computer’, Parallel Computing, vol. 6, 1988, pp. 275–296.MATHCrossRefGoogle Scholar
  6. [6]
    Dongarra, J.J. and Sorensen, D.C., ‘SCHEDULE: Tools for Developing and Analyzing Parallel Fortran Programs’, in The Characteristics of Parallel Algorithms, D.B. Gannon, L.H. Jamieson and R.J. Douglass (Eds), MIT Press, 1987, pp. 363–394.Google Scholar
  7. [7]
    Dongarra, J.J., Duff, I., Sorensen, D.C. and van der Vorst, H.A., ‘Solving Linear Systems on Vector and Shared Memory Computers’, SIAM,1991.Google Scholar
  8. [8]
    Dunigan, T.H., ‘Performance of the INTEL iPSC/860 and nCUBE 6400 Hypercube’, ORNL/TM-11790, Oak Ridge National Lab., TN, 1991.Google Scholar
  9. [9]
    El-Rewini, H. and Lewis, T.G., ‘Scheduling Parallel Program Tasks onto Arbitrary Target Machines’, Journal of Parallel and Distributed Computing, Vol. 9, 1990, pp. 138–153.CrossRefGoogle Scholar
  10. [10]
    Garey, M.R. and Johnson, D.S., ‘Computers and Intractability: a Guide to the Theory of NP-completeness’, W.H. Freeman and Company (New York), 1979.MATHGoogle Scholar
  11. [11]
    Geist, G.A. and Heath, M.T., ‘Matrix Factorization on a Hypercube Multiprocessor’, Hypercube Multiprocessors, SIAM, 1986, pp. 161–180.Google Scholar
  12. [12]
    Gerasoulis, A. and Nelken, I., ‘Static Scheduling for Linear Algebra DAGs’, Proc. of HCCA 4, 1989, pp. 671–674.Google Scholar
  13. [13]
    Gerasoulis, A., Venugopal, S. and Yang, T., ‘Clustering Task Graphs for Message Passing Architectures’, Proc. of 4th ACM Inter. Conf. on Supercomputing, Amsterdam, 1990, pp. 447–456.Google Scholar
  14. [14]
    Gerasoulis, A. and Yang, T., ‘On the Granularity and Clustering of Directed Acyclic Task Graphs’, TR-153, Dept. of Computer Science, Rutgers Univ., 1990.Google Scholar
  15. [15]
    Gerasoulis, A. and Yang, T., ‘A Comparison of Clustering Heuristics for Scheduling DAGs on Multiprocessors’, To appear in Journal of Parallel and Distributed Computing, special issue on scheduling and load balancing, Dec. 1992.Google Scholar
  16. [16]
    George, A., Heath, M.T., and Liu, J., ‘Parallel Cholesky Factorization on a Shared Memory Processor’, Lin. Algebra Appl., Vol. 77, 1986, pp. 165–187.MATHCrossRefGoogle Scholar
  17. [17]
    George, A., Heath, M.T., Liu, J. and Ng, E., ‘Solution of Sparse Positive Definite Systems on a Hypercube’, Report ORNL/TM-10865, Oak Ridge National Lab., 1988.Google Scholar
  18. [18]
    Girkar, M. and Polychronopoulos, C., ‘Partitioning Programs for Parallel Execution’, Proc. of ACM Inter. Conf. on Supercomputing, St. Malo, France, 1988.Google Scholar
  19. [19]
    Heath, M.T. and Romine, C.H., ‘Parallel Solution of Triangular Systems on Distributed Memory Multiprocessors’, SIAM J. Sci. Statist. Comput., Vol. 9, 1988, pp. 558–588.MATHCrossRefGoogle Scholar
  20. [20]
    Hiranandani, S., Kennedy, K. and Tseng, C.W., ‘Compiler Optimizations for Fortran D on MIMD Distributed-Memory Machines’, Proc. of Supercomputing’ 91, IEEE, pp. 86–100.Google Scholar
  21. [21]
    Karmarkar, N., ‘A New Parallel Architecture for Sparse Matrix Computation Based on Finite Project Geometries’, Proc. of Supercomputing’ 91, IEEE, pp. 358–369.Google Scholar
  22. [22]
    Koelbel, C., and Mehrotra, P., ‘Supporting Shared Data Structures on Distributed Memory Architectures’, Proc. of ACM SIGPLAN Sympos. on Principles and Practice of Parallel Programming, 1990, pp. 177–186.Google Scholar
  23. [23]
    Kim, S.J. and Browne, J.C., ‘A General Approach to Mapping of Parallel Computation upon Multiprocessor Architectures’, Proc. of Inter. Conf. on Parallel Processing, Vol. 3, 1988, pp. 1–8.Google Scholar
  24. [24]
    Kung, S.Y., ‘VLSI Array Processors’, Prentice Hall, 1988.Google Scholar
  25. [25]
    Moler, C., ‘Matrix Computation on Distributed Memory Multiprocessors’, Hypercube Multiprocessors 1986, SIAM, pp. 181–195.Google Scholar
  26. [26]
    Ortega, J.M., ‘Introduction to Parallel and Vector Solution of Linear Systems’, Plenum (New York), 1988.MATHGoogle Scholar
  27. [27]
    Papadimitriou, C. and Yannakakis, M., ‘Towards on an Architecture-Independent Analysis of Parallel Algorithms’, SIAM J. Comput., Vol. 19, 1990, pp. 322–328.MATHCrossRefGoogle Scholar
  28. [28]
    Picouleau, C., ‘Two new NP-Complete Scheduling Problems with Communication Delays and Unlimited Number of Processors’, M.A.S.I, Universite Pierre et Marie Curie Tour 45-46 B314, 4, place Jussieu, 75252 Paris Cedex 05, France, 1991.Google Scholar
  29. [29]
    Polychronopoulos, C., Girkar, M., Haghighat, M., Lee, C., Leung, B., and Schouten, D., ‘The Structure of Parafrase-2: an Advanced Parallelizing Compiler for C and Fortran’,, in Languages and Compilers for Parallel Computing, D. Gelernter, A. Nicolau and D. Padua (Eds.), 1990.Google Scholar
  30. [30]
    Saltz, J., Crowley, K., Mirchandaney, R. and Berryman, H., ‘Run-Time Scheduling and Execution of Loops on Message Passing Machines’, Journal of Parallel and Distributed Computing, Vol. 8, 1990, pp. 303–312.Google Scholar
  31. [31]
    Saad, Y., ‘Gaussian Elimination on Hypercubes’, in Parallel Algorithms and Architectures, Cosnard, M. et al. (Eds.), Elsevier Science Publishers, North-Holland, 1986.Google Scholar
  32. [32]
    Sarkar, V., ‘Partitioning and Scheduling Parallel Programs for Execution on Multiprocessors’, MIT Press, 1989.Google Scholar
  33. [33]
    Sarkar, V., ‘Determining Average Program Execution Times and their Variance’, Proc. of 1989 SIGPLAN, ACM, pp. 298–312.Google Scholar
  34. [34]
    Stone, H., ‘High-Performance Computer Architectures’, Addison-Wesley, 1987.Google Scholar
  35. [35]
    Wu, M.Y. and Gajski, D., ‘A Programming Aid for Hypercube Architectures’, Journal of Supercomputing, Vol. 2, 1988, pp. 349–372.CrossRefGoogle Scholar
  36. [36]
    Yang, T. and A. Gerasoulis, A., ‘A Fast Static Scheduling Algorithm for DAGs on an Unbounded Number of Processors’, Proc. of Supercomputing’ 91, IEEE, pp. 633–642.Google Scholar
  37. [37]
    Yang, T. and Gerasoulis, A., ‘List Scheduling with and without Communication Delay’, Report, 1992.Google Scholar
  38. [38]
    Yang, T. and Gerasoulis, A., ‘PYRROS: Static Task Scheduling and Code Generation for Message-Passing Multiprocessors’, Proc. of 6th ACM Inter. Confer. on Supercomputing, Washington D.C., 1992, pp. 428–437.Google Scholar

Copyright information

© Kluwer Academic Publishers 1993

Authors and Affiliations

  • Apostolos Gerasoulis
    • 1
  • Tao Yang
    • 1
  1. 1.Rutgers UniversityNew BrunswickUSA

Personalised recommendations