Advertisement

Decomposition Algorithms for Communication Minimization in Parallel Computing

  • Ioannis T. Christou
  • Robert R. Meyer
Part of the Combinatorial Optimization book series (COOP, volume 7)

Abstract

We present algorithms that minimize the communication overheads in large classes of parallel computing applications of scientific computing and engineering. Communication minimization is essentially equivalent to graph partitioning which in turn can be formulated as quadratic assignment. Decomposition — coordination techniques using genetic algorithms as multi-coordinators have allowed us to solve problems including millions or even billions of variables in a few seconds on a network of workstations. Furthermore, we establish the asymptotically optimal behavior of these algorithms for some regular types of domains.

Keywords

Genetic Algorithm Parallel Computing Optimal Shape Decomposition Algorithm Rectangular Grid 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [TMC, 1991 ]
    (1991). The Connection Machine CM-5 Technical Summary. Thinking Machines Corporation, Cambridge, MA.Google Scholar
  2. [CPL, 1998]
    (1998). Using the CPLEX 6.0 Callable Library. ILOG CPLEX Division, 889 Alder Ave. Incline Village, NV 89451.Google Scholar
  3. [Ahuja et al., 1993]
    Ahuja, R. K., Magnanti, T. L., and Orlin, J. B. (1993). Network Flows. Prentice Hall, Englewood Cliffs, NJ.zbMATHGoogle Scholar
  4. [Chen et al., 1993]
    Chen, R., Meyer, R., and Yackel, J. (1993). A genetic algorithm for diversity minimization and its parallel implementation. In Proceedings of the Fifth International Conference on Genetic Algorithms, pages 163–170, Palo Alto, CA. Morgan Kaufman.Google Scholar
  5. [Christou et al., 1999a]
    Christou, I., Martin, W., and Meyer, R. R. (1999a). Genetic algorithms as multi-coordinators in large scale optimization. In Davis, L. D., Whitley, L. D., DeJong, K., and Vose, M., editors, Evolutionary Algorithms, volume 111, pages 1–15, New York, NY. Springer-Verlag. The IMA Volumes in Mathematics and its Applications.CrossRefGoogle Scholar
  6. [Christou, 1996]
    Christou, I. T. (1996). Distributed Genetic Algorithms for Partitioning Uniform Grids. PhD thesis, Computer Sciences Dept. University of Wisconsin — Madison, Madison, WI.Google Scholar
  7. [Christou and Meyer, 1996a]
    Christou, I. T. and Meyer, R. R. (1996a). Fast distributed genetic algorithms for partitioning uniform grids. In Rolim, H. and Yang, T., editors, Lecture Notes in Computer Science, volume 1117, New York, NY. Springer-Verlag. Proceedings of the Third International Workshop on Parallel Algorithms for Irregularly Structured Problems (IRREGULAR 96).Google Scholar
  8. [Christou and Meyer, 1996b]
    Christou, I. T. and Meyer, R. R. (1996b). Optimal and asymptotically optimal equi-partition of rectangular domains via stripe decomposition. In Fischer, H., Riedmuller, B., and Schaffler, S., editors, Applied Mathematics and Parallel Computing — Festschrift for Klaus Ritter, pages 77–96, Berlin, Germany. Physica-Verlag.CrossRefGoogle Scholar
  9. [Christou and Meyer, 1996c]
    Christou, I. T. and Meyer, R. R. (1996c). Optimal equi-partition of rectangular domains for parallel computation. Journal of Global Optimization, 8:15–34.MathSciNetzbMATHCrossRefGoogle Scholar
  10. [Christou et al., 1999b]
    Christou, I. T., Zakarian, A., Liu, J., and Carter, H. (1999b). A two-phase genetic algorithm for large scale bidline generation problems at delta air lines. Interfaces, 29(5).Google Scholar
  11. [Crandall and Quinn, 1995]
    Crandall, P. and Quinn, M. (1995). Non-uniform 2-d grid partitioning for heterogeneous parallel architectures. In Proceedings of the 9th International Symposium on Parallel Processing, pages 428–435, Los Alamitos, CA. IEEE Computer Society Press.CrossRefGoogle Scholar
  12. [Dantzig and Wolfe, 1960]
    Dantzig, G. B. and Wolfe, P. (1960). Decomposition principle for linear programs. Operations Research, 8:101–111.zbMATHCrossRefGoogle Scholar
  13. [Darwin, 1859]
    Darwin, C. (1859). On the Origin of Species by Means of Natural Selection. J. Murray, London, UK.Google Scholar
  14. [DeLeone et al., 1994]
    DeLeone, R., Meyer, R. R., Kontogiorgis, S., Zakarian, A., and Zakeri, G. (1994). Coordination in coarse-grain decomposition. SIAM Journal on Optimization, 4(4):773–793.MathSciNetGoogle Scholar
  15. [Falkenauer, 1998]
    Falkenauer, E. (1998). Genetic Algorithms and Grouping Problems. John Wiley & Sons, New York, NY.Google Scholar
  16. [Fiedler, 1973]
    Fiedler, M. (1973). Algebraic connectivity of graphs. Czechoslovak Math. Journal, 23:298–305.MathSciNetGoogle Scholar
  17. [Fulkerson, 1963]
    Fulkerson, D. R. (1963). Flows in networks. In Recent Advances in Mathematical Programming, pages 319–332, New York, NY. McGrow — Hill.Google Scholar
  18. [Geist et al., 1994]
    Geist, A., Beguelin, A., Dongarra, J., Jiang, W., Manchek, R., and Sunderam, V. (1994). PVM 3 User’s Guide and Reference Manual. Oak Ridge National Laboratory, Oak Ridge, TN.Google Scholar
  19. [Gilbert et al., 1995]
    Gilbert, J. R., Miller, G. L., and Teng, S. H. (1995). Geometric mesh partitioning: Implementation and experiments. In Proceedings of the 9th International Symposium on Parallel Processing, pages 418–427, Los Alamitos, CA. IEEE Computer Society Press.CrossRefGoogle Scholar
  20. [Hart, 1994]
    Hart, W. E. (1994). Adaptive global optimization with local search. PhD thesis, Computer Science and Engineering Dept. University of California — San Diego, San Diego, CA.Google Scholar
  21. [Hendrickson and Leland, 1993]
    Hendrickson, B. and Leland, R. (1993). Multidimensional spectral load balancing. In Proceedings of the 6th SIAM Conference on Parallel Processing and Scientific Computing, Philadelphia, PA. SIAM.Google Scholar
  22. [Hendrickson and Leland, 1995a]
    Hendrickson, B. and Leland, R. (1995a). The Chaco User’s Guide Version 2.0. Sandia National Laboratories, Sandia, NM.Google Scholar
  23. [Hendrickson and Leland, 1995b]
    Hendrickson, B. and Leland, R. (1995b). An improved spectral graph partitioning algorithm for mapping parallel computations. SIAM Journal on Scientific Computation, 16:452–469.MathSciNetzbMATHCrossRefGoogle Scholar
  24. [Holland, 1992]
    Holland, J. (1992). Adaptation in Natural and Artificial Systems. MIT Press, Cambridge, MA.Google Scholar
  25. [Karypis and Kumar, 1995]
    Karypis, G. and Kumar, V. (1995). Multilevel kway partitioning scheme for irregular graphs. Technical Report 95–064, Computer Science Dept. University of Minnesota.Google Scholar
  26. [Kernighan and Lin, 1970]
    Kernighan, B. W. and Lin, S. (1970). An effective heuristic procedure for partitioning graphs. Bell Systems Tech. Journal, pages 291–308.Google Scholar
  27. [Laguna et al., 1994]
    Laguna, M., Feo, T. A., and Elrod, H. C. (1994). A greedy randomized adaptive search procedure for the two — partition problem. Operations Research.Google Scholar
  28. [Levine, 1995]
    Levine, D. (1995). User’s Guide to the PGAPack Parallel Genetic Algorithm Library Version 0.2. Argonne National Laboratory, Argonne, IL.Google Scholar
  29. [Li et al., 1994]
    Li, Y., Pardalos, P. M., and Resende, M. G. C. (1994). A greedy randomized adaptive search procedure for the quadratic assignment problem. In Pardalos, P. M. and Wolkowicz, H., editors, Quadratic Assignment and Related Problems, pages 237–262, Providence, RI. American Mathematical Society.Google Scholar
  30. [Lin, 1991]
    Lin, K. Y. (1991). Exact solution of the convex polygon perimeter and area generating function. J. Phys. A. Math Gen., 24:2411–2417.CrossRefGoogle Scholar
  31. [Martin, 1998]
    Martin, W. (1998). Fast equi-partitioning of rectangular domains using stripe decomposition. Discrete Applied Mathematics, 82(1– 3): 193–207.MathSciNetzbMATHCrossRefGoogle Scholar
  32. [Melou, 1994]
    Melou, M. B. (1994). Codage des polyominos convexes et equation pour l’enumeration suivant l’aire. Discrete Applied Mathematics, 48:21–43.MathSciNetzbMATHCrossRefGoogle Scholar
  33. [Michalewicz, 1994]
    Michalewicz, Z. (1994). Genetic Algorithms + Data Structures = Evolution Programs. Springer-Verlag, New York, NY.zbMATHGoogle Scholar
  34. [Miller et al., 1993]
    Miller, G. L., Teng, S. H., Thurston, W., and Vavasis, S. A. (1993). Automatic mesh partitioning. In George, A., Gilbert, J. R., and Liu, J. W. H., editors, Graph Theory and Sparse Matrix Computation, New York, NY. Springer-Verlag.Google Scholar
  35. [Muhienbein et al., 1991]
    Muhlenbein, H., Schomisch, M., and Born, J. (1991). The parallel genetic algorithm as function optimizer. In Belew, R. and Booker, L., editors, Proceedings of the Fourth Intl. Conference on Genetic Algorithms, pages 45–52, Los Altos, CA. Morgan Kaufmann Publishers.Google Scholar
  36. [Nemhauser and Wolsey, 1985]
    Nemhauser, G. and Wolsey, L. (1985). Integer and Combinatorial Optimization. John Wiley & Sons, New York, NY.Google Scholar
  37. [Pardalos et al., 1995]
    Pardalos, P., Pitsoulis, L., and Resende, M. (1995). A parallel grasp implementation for the quadratic assignment problem. In Ferreira, A. and Rolim, J., editors, Parallel Algorithms for Irregularly Structured Problems, pages 111–130, Dordrecht, The Netherlands. Kluwer Academic Publishers. Proceedings of the First International Workshop on Parallel A1gorithms for Irregularly Structured Problems (IRREGULAR 94).Google Scholar
  38. [Pardalos et al., 1993]
    Pardalos, P. M., Rendl, F., and Wolkowicz, H. (1993). The quadratic assignment problem: A survey and recent developments. In Pardalos, P. M. and Wolkowicz, H., editors, QuadraticAssignmentandRelated Problems, pages 1–42, Providence, RI. American Mathematical Society.Google Scholar
  39. [Pothen et al., 1990]
    Pothen, A., Simon, H. D., and Liu, K. P. (1990). Partitioning sparse matrices with eigenvectors of graphs. SIAM Journal on Matrix Analysis and Applications, 11:430–452.MathSciNetzbMATHCrossRefGoogle Scholar
  40. [Schalkoff, 1989]
    Schalkoff, R. J. (1989). Digital Image Processing and Computer Vision. John Wiley & Sons, New York, NY.Google Scholar
  41. [Schultz and Meyer, 1991]
    Schultz, G. L. and Meyer, R. R. (1991). An interior point method for block angular optimization. SIAM Journal on Optimization, 1(4) : 5 8 3–602.Google Scholar
  42. [Strikwerda, 1989]
    Strikwerda, J. (1989). Finite Difference Schemes and Partial Differential Equations. Wadsworth & Brooks Cole, Pacific Grove, CA.zbMATHGoogle Scholar
  43. [von Laszewski, 1991]
    von Laszewski, G. (1991). Intelligent structural operators for the k-way graph partitioning problem. In Belew, R. and Booker, L., editors, Proceedings of the Fourth Intl. Conference on Genetic Algorithms, pages 45–52, Los Altos, CA. Morgan Kaufmann Publishers.Google Scholar
  44. [Yackel, 1993]
    Yackel, J. (1993). Minimum Perimeter Tiling in Parallel Computation. PhD thesis, Computer Sciences Dept. University of Wisconsin — Madison, Madison, WI.Google Scholar
  45. [Yackel and Meyer, 1992a]
    Yackel, J. and Meyer, R. R. (1992a). Minimum perimeter decomposition. Technical Report 1078, University of Wisconsin — Madison, Madison, WI.Google Scholar
  46. [Yackel and Meyer, 1992b]
    Yackel, J. and Meyer, R. R. (1992b). Optimal tilings for parallel database design. In Pardalos, P. M., editor, Advances in Optimization and Parallel Computing, pages 293–309, New York, NY. North — Holland.Google Scholar
  47. [Yackel et al., 1997]
    Yackel, J., Meyer, R. R., and Christou, I. T. (1997). Minimum-perimeter domain assignment. Mathematical Programming, 78:283–303.MathSciNetzbMATHGoogle Scholar
  48. [Zakarian, 1995]
    Zakarian, A. (1995). NonLinear Jacobi and epsilon — Relaxation Methods for Parallel Network Optimization. PhD thesis, Computer Sciences Dept. University of Wisconsin — Madison.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2000

Authors and Affiliations

  • Ioannis T. Christou
    • 1
  • Robert R. Meyer
    • 2
  1. 1.Dept. of Operations ResearchDelta Technology Inc.AtlantaUSA
  2. 2.Computer Sciences Dept.University of Wisconsin - MadisonMadisonUSA

Personalised recommendations