Skip to main content
SpringerLink
Log in

Analyzing Communication Features and Community Structure of HPC Applications

  • Conference paper
  • First Online:
High Performance Computing (CARLA 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 979))

Included in the following conference series:

  • 454 Accesses

Abstract

A few exascale machines are scheduled to become operational in the next couple of years. Reaching such achievement required the HPC community to overcome obstacles in programmability, power management, memory hierarchy, and reliability. Similar challenges are to be faced in the pursuit of greater performance gains. In particular, design of interconnects stands out as a major hurdle. Computer networks for extreme-scale system will need a deeper understanding of the communication characteristics of applications that will run on those systems. We analyzed a set of nine representative HPC applications and created a catalog of well-defined communication patterns that constitute building blocks for modern scientific codes. Furthermore, we found little difference between popular community-detection algorithms, which tend to form few but relatively big communities.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. igraph: The network analysis package (2015). http://igraph.org/

  2. Almeida, H., Guedes, D., Meira, W., Zaki, M.J.: Is there a best quality metric for graph clusters? In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011. LNCS (LNAI), vol. 6911, pp. 44–59. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23780-5_13

    Chapter  Google Scholar 

  3. Barrett, R., et al.: On the role of co-design in high performance computing, vol. 24, pp. 141–155 (2013)

    Google Scholar 

  4. Blondel, V.D., Guillaume, J.L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks. J. Stat. Mech. Theory Exp. 2008(10), P10008 (2008)

    Article  Google Scholar 

  5. Brightwell, R., Barrett, B.W., Hemmert, K.S., Underwood, K.D.: Challenges for high-performance networking for exascale computing. In: 2010 Proceedings of 19th International Conference on Computer Communications and Networks, pp. 1–6, August 2010

    Google Scholar 

  6. Clauset, A., Newman, M.E.J., Moore, C.: Finding community structure in very large networks. Phys. Rev. E 70, 066111 (2004)

    Article  Google Scholar 

  7. CORAL: Collaboration of Oak Ridge, Argonne and Livermore benchmark codes. https://asc.llnl.gov/CORAL-benchmarks

  8. Dongarra, J., et al.: The international exascale software project roadmap (2011)

    Google Scholar 

  9. Girvan, M., Newman, M.E.: Community structure in social and biological networks. Proc. Natl. Acad. Sci. 99(12), 7821–7826 (2002)

    Article  MathSciNet  Google Scholar 

  10. Heroux, M.A., et al.: Improving performance via mini-applications. Technical report SAND2009-5574, Sandia National Laboratories (2009)

    Google Scholar 

  11. Hoefler, T., Snir, M.: Generic topology mapping strategies for large-scale parallel architectures. In: Proceedings of the 2011 ACM International Conference on Supercomputing (ICS 2011), pp. 75–85. ACM, June 2011

    Google Scholar 

  12. Hoefler, T., Jeannot, E., Mercier, G.: An overview of process mapping techniques and algorithms in high-performance computing (2014)

    Google Scholar 

  13. Kogge, P., et al.: Exascale computing study: technology challenges in achieving exascale systems (2008)

    Google Scholar 

  14. Leskovec, J., Lang, K.J., Mahoney, M.: Empirical comparison of algorithms for network community detection. In: Proceedings of the 19th International Conference on World Wide Web, pp. 631–640 (2010)

    Google Scholar 

  15. NAS Parallel Benchmarks. https://www.nas.nasa.gov/publications/npb.html

  16. Newman, M.E.J.: Fast algorithm for detecting community structure in networks. Phys. Rev. E 69, 066133 (2004)

    Article  Google Scholar 

  17. Newman, M.E.J.: Finding community structure in networks using the eigenvectors of matrices. Phys. Rev. E 74, 036104 (2006)

    Article  MathSciNet  Google Scholar 

  18. Newman, M.E.J., Girvan, M.: Finding and evaluating community structure in networks. Phys. Rev. E 69, 026113 (2004)

    Article  Google Scholar 

  19. Radicchi, F., Castellano, C., Cecconi, F., Loreto, V., Parisi, D.: Defining and identifying communities in networks. Proc. Natl. Acad. Sci. U.S.A. 101(9), 2658–2663 (2004)

    Article  Google Scholar 

  20. Raponi, P.G., Petrini, F., Walkup, R., Checconi, F.: Characterization of the communication patterns of scientific applications on blue gene/p. In: 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum (IPDPSW), pp. 1017–1024 (2011)

    Google Scholar 

  21. Riesen, R.: Communication patterns [message-passing patterns]. In: 20th International Parallel and Distributed Processing Symposium, IPDPS 2006, 8 pp. IEEE (2006)

    Google Scholar 

  22. Ropars, T., Guermouche, A., Uçar, B., Meneses, E., Kalé, L.V., Cappello, F.: On the use of cluster-based partial message logging to improve fault tolerance for MPI HPC applications. In: Jeannot, E., Namyst, R., Roman, J. (eds.) Euro-Par 2011. LNCS, vol. 6852, pp. 567–578. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23400-2_53

    Chapter  Google Scholar 

  23. Roth, P.C., Meredith, J.S., Vetter, J.S.: Automated characterization of parallel application communication patterns. In: Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing, pp. 73–84. ACM (2015)

    Google Scholar 

  24. Vetter, J.S., et al.: Quantifying architectural requirements of contemporary extreme-scale scientific applications. In: Jarvis, S.A., Wright, S.A., Hammond, S.D. (eds.) PMBS 2013. LNCS, vol. 8551, pp. 3–24. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10214-6_1

    Chapter  Google Scholar 

  25. Vetter, J.S., Mueller, F.: Communication characteristics of large-scale scientific applications for contemporary cluster architectures. J. Parallel Distrib. Comput. 63(9), 853–865 (2003)

    Article  Google Scholar 

  26. Vetter, J.S., Yoo, A.: An empirical performance evaluation of scalable scientific applications. In: ACM/IEEE 2002 Conference on Supercomputing, p. 16. IEEE (2002)

    Google Scholar 

  27. Vetter, J., Chambreau, C.: mpIP: lightweight, scalable MPI profling (2014). http://mpip.sourceforge.net/

  28. Xue, R., et al.: MPIWiz: subgroup reproducible replay of MPI applications. In: Proceedings of the 14th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2009, pp. 251–260. ACM, New York (2009)

    Google Scholar 

  29. Yang, Z., Algesheimer, R., Tessone, C.J.: A comparative analysis of community detection algorithms on artificial networks. Sci. Rep. 6, 30750 (2016)

    Article  Google Scholar 

Download references

Acknowledgments

This research was partially supported by a machine allocation on Kabré supercomputer at the Costa Rica National High Technology Center.

Author information

Authors and Affiliations

  1. School of Computing, Costa Rica Institute of Technology, Cartago, Costa Rica

    Manfred Calvo & Esteban Meneses

  2. Advanced Computing Laboratory, Costa Rica National High Technology Center, San José, Costa Rica

    Diego Jiménez & Esteban Meneses

Authors
  1. Manfred Calvo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Diego Jiménez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Esteban Meneses
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Diego Jiménez .

Editor information

Editors and Affiliations

  1. Instituto Tecnológico de Costa Rica, Centro Nacional de Alta Tecnología , Pavas, Costa Rica

    Esteban Meneses

  2. Universidad de los Andes, Bogotá, Colombia

    Harold Castro

  3. Universidad Industrial de Santander, Bucaramanga, Colombia

    Carlos Jaime Barrios Hernández

  4. Universidad de Antioquia, Medellín, Colombia

    Raul Ramos-Pollan

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Calvo, M., Jiménez, D., Meneses, E. (2019). Analyzing Communication Features and Community Structure of HPC Applications. In: Meneses, E., Castro, H., Barrios Hernández, C., Ramos-Pollan, R. (eds) High Performance Computing. CARLA 2018. Communications in Computer and Information Science, vol 979. Springer, Cham. https://doi.org/10.1007/978-3-030-16205-4_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-16205-4_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-16204-7

  • Online ISBN: 978-3-030-16205-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation