Skip to main content
SpringerLink
Log in

A Performance and Scalability Analysis of the MPI Based Tools Utilized in a Large Ice Sheet Model Executing in a Multicore Environment

  • Conference paper
  • First Online:
Book cover Algorithms and Architectures for Parallel Processing (ICA3PP 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9531))

Abstract

This paper analyzes the performance and scalability characteristics of both the computational and I/O components of the Parallel Ice Sheet Model (PISM) executing in a multicore supercomputing environment. It examines the impact of multicore technologies on two state-of-the-art parallel I/O systems, both of which are based on the same underlying implementation of the MPI-IO standard, but which exhibit very different performance and scalability characteristics. It also examines these same characteristics for the MPI-based computational engine of the simulation model. One important benefit of studying these three software systems both independently and together is that it exposes a fundamental tradeoff in the ability to provide scalable I/O and scalable computational performance in a multicore environment. This paper also provides what, at least at first glance, appears to be very counter-intuitive performance results. We examine the underlying reasons for such results, and discuss the important insights gained through this examination.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Balay, S., et al.: PETSc Users Manual. Technical report #ANL-95/11 - Revision 3.6. Argonne National Laboratory

    Google Scholar 

  2. Balay, S., Gropp, W.D., McInnes, L.C., Smith, B.F.: Efficient management of parallelism in object oriented numerical software libraries. In: Arge, E., Bruaset, A.M., Langtangen, H.P. (eds.) Modern Software Tools in Scientific Computing, pp. 163–202. Birkhäuser Press, Boston (1997)

    Chapter  Google Scholar 

  3. Baylor, S.J., Rathi, B.D.: An evaluation of the memory reference behavior of engineering/scientific applications in parallel systems. Int. J. High Speed Comput. 1(4), 603–641 (1989)

    Article  MATH  Google Scholar 

  4. de Boer, B., Dolan, A.M., Bernales, J., Gasson, E., Goelzer, H., Golledge, N.R., Sutter, J., Huybrechts, P., Lohmann, G., Rogozhina, I., Abe-Ouchi, A., Saito, F., van de Wal, R.S.W.: Simulating the Antarctic ice sheet in the late-Pliocene warm period: PLISMIP-ANT, an ice-sheet model intercomparison project. Cryosphere. 9(3), 881–903 (2015)

    Article  Google Scholar 

  5. Bueler, E., van Pelt, W.: Mass-conserving subglacial hydrology in the parallel ice sheet model version 0.6. Geosci. Mod. Dev. 8(6), 1613–1635 (2015)

    Article  Google Scholar 

  6. Buntinas, D., Goglin, B., Goodell, D., Mercier, G., Moreaud, S.: Cache-Efficient, Intranode, Large-Message MPI Communication with MPICH2-Nemesis, pp. 462–469, September 2009

    Google Scholar 

  7. CDF-5 Format Specifications. http://cucis.ece.northwestern.edu/projects/PnetCDF/cdf5.html. Accessed 11 September 2013

  8. Coloma, K., Choudhary, A., Liao, W.: DAChe: direct access cache system for parallel I/O. In: Proceedings of the 2005 International Supercomputer Conference (2005)

    Google Scholar 

  9. Crandall, P., Aydt, R.A., Chien, A.A., Reed, D.A.: Input/output characteristics of scalable parallel applications. In: Proceedings of Supercomputing 1995 (1995)

    Google Scholar 

  10. Phillip, D., Timothy, M.: Increasing the scalability of PISM for high resolution ice sheet models. In: Workshop on Parallel and Distributed Scientific and Engineering Computing, Boston, May 2013

    Google Scholar 

  11. Dickens, P.M., Thakur, R.: A performance study of two-phase i/o. In: Pritchard, D., Reeve, J.S. (eds.) Euro-Par 1998. LNCS, vol. 1470, pp. 959–965. Springer, Heidelberg (1998)

    Chapter  Google Scholar 

  12. Documentation for PISM, a parallel Ice Sheet Model. http://pism-docs.org/wiki/doku.php. Accessed 15 May 2015

  13. Feldmann, J., Levermann, A.: Interaction of marine ice-sheet instabilities in two drainage basins: simple scaling of geometry and transition time. Cryosphere 9(2), 631–645 (2015)

    Article  Google Scholar 

  14. Fowler, A.C.: Mathematical Models in the Applied Sciences. Cambridge University Press, Cambridge (1997)

    MATH  Google Scholar 

  15. Jin, H.-W., Sur, S., Chai, L., Panda, D.K.: Lightweight kernel-level primitives for high-performance MPI intra-node communication over multi-core systems, pp. 446–451 (2007)

    Google Scholar 

  16. Liao, W., Choudhary, A.: Dynamically adapting file domain partitioning methods for collective i/o based on underlying parallel file system locking protocols. In: Proceedings of the ACM/IEEE Conference on Supercomputing (SC 2008), pp. 313–344 (2008)

    Google Scholar 

  17. Li, J., Liao, W., Choudhary, A., Ross, R., Thakur, R., Latham, R., Siegel, A., Gallagher, B., Zingale, M.: Parallel netCDF: a high-performance scientific I/O interface. In: Proceedings of Supercomputing (2003)

    Google Scholar 

  18. Liu, Q., et al.: Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks: HELLO ADIOS. Concurrency Comput. Pract. Experience 26(7), 1453–1473 (2014)

    Article  Google Scholar 

  19. Ma, A., Califano, F.: The shallow ice approximation for anisotropic ice- formulation and limits. J. Geophys. Res. 103((B1)), 691–705 (1998)

    Google Scholar 

  20. Ma, T., Bosilca, G., Bouteiller, A., Dongarra, J.: HierKNEM: an adaptive framework for kernel-assisted and topology-aware collective communications on many-core clusters. In: 2012 IEEE 26th International Parallel & Distributed Processing Symposium (Ipdps), pp. 970–982, May 2012

    Google Scholar 

  21. Ma, T., Bosilca, G., Bouteiller, A., Dongarra, J.J.: Kernel-assisted and topology-aware MPI collective communications on multicore/many-core platforms. J. Parallel Distrib. Comput. 73(7), 1000–1010 (2013)

    Article  Google Scholar 

  22. Ma, T., Bosilca, G., Bouteiller, A., Goglin, B., Squyres, J.M., Dongarra, J.J.: Kernel assisted collective intra-node MPI communication among multi-core and many-core CPUs. In: 2011 International Conference on Parallel Processing (ICPP), pp. 532–541 (2011)

    Google Scholar 

  23. Mellanox Technologies. https://www.mellanox.com/. Accessed 21 July 2015

  24. Message Passing Interface (MPI) Forum Home Page. http://www.mpi-forum.org/. Accessed 30 August 2013

  25. Moreaud, S., Goglin, B., Namyst, R., Goodell, D.: Optimizing MPI communication within large multicore nodes with kernel assistance. In: IPDPS Workshops, pp. 1–7 (2010)

    Google Scholar 

  26. MPI-2: Extensions to the Message-Passing Interface Message Passing Interface Forum. http://mpi-forum.org/docs/mpi-20-html/mpi2-report.html. Accessed 31 August 2013

  27. Nieuwejaar, N., Kotz, D., Purakayastha, A., Ellis, C.S., Best, M.: File-access characteristics of parallel scientific workloads. IEEE Trans. Parallel Distrib. Syst. 7(10), 1075–1089 (1996)

    Article  Google Scholar 

  28. Parallel HDF5. http://www.hdfgroup.org/HDF5/PHDF5/. Accessed 31 August 2013

  29. PETSc Web page (2015). http://www.mcs.anl.gov/petsc

  30. PISM, a Parallel Ice Sheet Model (2014). http://www.pism-docs.org

  31. PISM, a Parallel Ice Sheet Model: User’s Manual (2015). http://www.pism-docs.org/wiki/lib/exe/fetch.php?media=manual.pdf

  32. SeaRISE Assessment - Interactive System for Ice sheet Simulation. http://websrv.cs.umt.edu/isis/index.php/SeaRISE_Assessment. Accessed 18 May 2015

  33. Texas Advanced Computing Center – Stampede. http://www.tacc.utexas.edu/resources/hpc/stampede. Accessed 30 August 2013

  34. Thakur, R., Gropp, W., Lusk, E.: On implementing mpi-io portably and with high performance. In: Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pp. 23–32 (1999)

    Google Scholar 

  35. Thakur, R., Lusk, E.: An abstract-device interface for implementing portable parallel-i/o interfaces. In: Proceedings of the 6th Symposium on the Frontiers of Massively Parallel Computatio, pp. 180–187 (1996)

    Google Scholar 

  36. Thakur, R., Lusk, E.: Data sieving and collective i/o in ROMIO. In: Proceedings of the Seventh Symposium on the Frontiers of Massively Parallel Computation, pp. 182–189 (1998)

    Google Scholar 

  37. The HDF Group - Information, Support, and Software. http://www.hdfgroup.org/. Accessed 11 September 2013

  38. Unidata | Home. http://www.unidata.ucar.edu/. Accessed 11 September 2013

  39. Unidata | NetCDF. http://www.unidata.ucar.edu/software/netcdf/. Accessed 11 September 2013

  40. Unidata | Software. http://www.unidata.ucar.edu/software/. Accessed 11 September 2013

  41. Weis, M., Greve, R., Hutter, K.: Theory of shallow ice shelves. Continuum Mech. Thermo-dyn. 11(1999), 15–50 (1999)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI-1053575.

Author information

Authors and Affiliations

  1. School of Computing and Information Sciences, University of Maine, Orono, ME, 04426, USA

    Phillip Dickens

Authors
  1. Phillip Dickens
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Phillip Dickens .

Editor information

Editors and Affiliations

  1. Central South University, Changsha, China

    Guojun Wang

  2. The University of Sydney, Sydney, New South Wales, Australia

    Albert Zomaya

  3. University of Murcia, Murcia, Murcia, Spain

    Gregorio Martinez

  4. Hunan University, Changsha, China

    Kenli Li

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Dickens, P. (2015). A Performance and Scalability Analysis of the MPI Based Tools Utilized in a Large Ice Sheet Model Executing in a Multicore Environment. In: Wang, G., Zomaya, A., Martinez, G., Li, K. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2015. Lecture Notes in Computer Science(), vol 9531. Springer, Cham. https://doi.org/10.1007/978-3-319-27140-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27140-8_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27139-2

  • Online ISBN: 978-3-319-27140-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation