Advertisement

Parallel Graph Partitioning Optimization Under PEGASUS DA Application Global State Monitoring

  • Adam SmykEmail author
  • Marek Tudruj
  • Lukasz Grochal
Conference paper
  • 40 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12043)

Abstract

The paper concerns the use of global application states monitoring in distributed programs for advanced graph partitioning optimization. Two strategies for the control design of advanced parallel/distributed graph partitioning algorithms are presented and discussed. In the first one, the parallel algorithm control runs on top of the ready to use basic graph partitioning functions available inside an existing graph partitioning METIS tool. The second control strategy is based on a genetic programing algorithm in which the applied basic graph partitioning primitives and the overall algorithmic parallel/distributed control can be freely designed by the user. In these strategies, the graph partitioning control is executed by processes/threads supervised by the application global states monitoring facilities provided inside a novel distributed program design framework PEGASUS DA. This framework provides system support to construct user-defined strongly consistent global application states and an API to define corresponding execution control. In particular, it concerns computing global control predicates on the constructed global states, the predicates evaluation and asynchronous execution control handling to obtain application global state-driven reactions. Based on such implementation, different features of the graph partitioning optimization strategies have been designed and tested. The experimental results have shown benefits of the new graph partitioning control methods designed with the use of the application global states monitoring.

Keywords

Parallel graph/mesh partitioning Distributed program design tools Application global states monitoring 

References

  1. 1.
    Garey, M., Johnson, D., Stockmeyer, L.: Some simplified NP-complete graph problems. Theor. Comput. Sci. 1, 237–367 (1976)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Karypis, D.: METIS a Software Package for Partitioning Unstructured Graphs, Partitioning Meshes, and Computing Fill-Reducing Orderings of Sparse Matrices Version 5.1.0, Department of Computer Science & Engineering, University of Minnesota, Minneapolis, March (2013)Google Scholar
  3. 3.
  4. 4.
    Khan, M.S., Li, K.F.: Fast graph partitioning algorithms. In: IEEE Pacific Rim Conference on Communication, Computers, and Signal Processing, Victoria, B.C., Canada, May 1995, pp. 337–342 (1995)Google Scholar
  5. 5.
    Kerighan, B.W., Lin, S.: An efficient heuristic procedure for partitioning graphs. AT&T Bell Labs. Tech. J. 49, 291–307 (1970)CrossRefGoogle Scholar
  6. 6.
    Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Tudruj, M., Borkowski, J., Kopanski, D., Laskowski, E., Masko, L., Smyk, A.: PEGASUS DA framework for distributed program execution control based on application global states monitoring. Concurr. Comput.: Pract. Exp. 27(4), 1027–1053 (2015)CrossRefGoogle Scholar
  8. 8.
    Pellegrini, F.: PT-Scotch and libPTScotch 6.0 User’s Guide, (ver. 6) Universite Bordeaux 1 & LaBRI, UMR CNRS 5800 Bacchus team, INRIA Bordeaux Sud-Ouest December (2012)Google Scholar
  9. 9.
  10. 10.
  11. 11.
    Roy, A., Mihailovic, I., Zwaenepoel, W.: X-stream: edge-centric graph processing using streaming partitions. In: 24th ACM Symposium on Operating Systems Principles, pp. 472–488. ACM (2013)Google Scholar
  12. 12.
    Wang, Y., et al.: GPU graph analytics. ACM Trans. Parallel Comput. (TOPC) 4(1) (2017). Invited papers from PPoPP 2016Google Scholar
  13. 13.
    Smyk, A., Tudruj, M., Grochal, L.: Global application states monitoring applied to graph partitioning optimization. In: 17th International Symposium on Parallel and Distributed Computing, ISPDC2018, Geneva, Switzerland, June 2018, pp. 85–92. IEEE CS Press (2018)Google Scholar
  14. 14.
    Poli, R., Langdon, W.B., McPhee, N.F., Koza, J.R.: Genetic programming: an introduction and tutorial, with a survey of techniques and applications. University of Essex, UK Technical report [CES-475] (2007)Google Scholar
  15. 15.
    Wenhua, Y.: Advanced FDTD Methods: Parallelization, Acceleration, and Engineering Applications. Artech House (2011)Google Scholar
  16. 16.
    Tian, Y., et al.: From “think like a vertex” to “think like a graph”. Proc. VLDB Endow. 7(3), 193–204 (2013)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Sanders, P., Schulz, C.: High Quality Graph Partitioning (2013).  https://doi.org/10.1090/conm/588/11700
  18. 18.
    Sanders, P., Schulz, C.: Distributed evolutionary graph partitioning. In: Proceedings of the 12th Workshop on Algorithm Engineering and Experimentation (ALENEX 2012), pp. 16–29 (2012)Google Scholar
  19. 19.
    Raghavan, U.N., Albert, R., Kumara, S.: Near linear time algorithm to detect community structures in large-scale networks. Phys. Rev. E 76(3), 036106 (2007)CrossRefGoogle Scholar
  20. 20.
    Rahimian, F., et al.: JA-BE-JA: a distributed algorithm for balanced graph partitioning. In: 7th International Conference on Self-Adaptive and Self-organizing Systems, pp. 51–60. IEEE (2013)Google Scholar
  21. 21.
    Walshaw, C.: Multilevel refinement for combinatorial optimisation problems. Ann. OR 131(1), 325–372 (2004)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Masko, L., Tudruj, M.: Application global state monitoring in optimization of parallel event-driven simulation. Concurr. Comput. Pract. Exp. 31(19), e5015 (2019)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Polish-Japanese Academy of Information TechnologyWarsawPoland
  2. 2.Institute of Computer SciencePolish Academy of SciencesWarsawPoland

Personalised recommendations