Advertisement

Journal of Scientific Computing

, Volume 46, Issue 3, pp 329–358 | Cite as

Scalability of an Unstructured Grid Continuous Galerkin Based Hurricane Storm Surge Model

  • S. Tanaka
  • S. Bunya
  • J. J. Westerink
  • C. Dawson
  • R. A. LuettichJr.
Article

Abstract

This paper evaluates the parallel performance and scalability of an unstructured grid Shallow Water Equation (SWE) hurricane storm surge model. We use the ADCIRC model, which is based on the generalized wave continuity equation continuous Galerkin method, within a parallel computational framework based on domain decomposition and the MPI (Message Passing Interface) library. We measure the performance of the model run implicitly and explicitly on various grids. We analyze the performance as well as accuracy with various spatial and temporal discretizations. We improve the output writing performance by introducing sets of dedicated writer cores. Performance is measured on the Texas Advanced Computing Center Ranger machine. A high resolution 9,314,706 finite element node grid with 1 s time steps can complete a day of real time hurricane storm surge simulation in less than 20 min of computer wall clock time, using 16,384 cores with sets of dedicated writer cores.

Keywords

Parallel computing Scaling Shallow water equations Hurricane storm surge model Finite element method Forecasting 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Resio, D.T., Westerink, J.J.: Hurricanes and the physics of surges. Phys. Today 61(9), 33–38 (2008) CrossRefGoogle Scholar
  2. 2.
    Kolar, R.L., Westerink, J.J., Cantekin, M.E., Blain, C.A.: Aspects of nonlinear simulations using shallow-water models based on the wave continuity equation. Comput. Fluids 23, 523–528 (1994) MATHCrossRefGoogle Scholar
  3. 3.
    Luettich, R.A., Hu, S., Westerink, J.J.: Development of the direct stress solution technique for three dimensional hydrodynamic models using finite elements. Int. J. Numer. Methods Fluids 19, 295–319 (1994) MATHCrossRefGoogle Scholar
  4. 4.
    Grenier, R.R., Luettich, R.A., Westerink, J.J.: A comparison of the nonlinear frictional characteristics of two-dimensional and three-dimensional models of a shallow tidal embayment. J. Geophys. Res. 100(C7), 13719–13735 (1995) CrossRefGoogle Scholar
  5. 5.
    Kolar, R.L., Gray, W.G., Westerink, J.J.: Boundary conditions in shallow water models—an alternative implementation for finite element codes. Int. J. Numer. Methods Fluids 22, 603–618 (1996) MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Atkinson, J.H., Westerink, J.J., Hervouet, J.M.: Similarities between the quasi-bubble and the generalized wave continuity equation solutions to the shallow water equations. Int. J. Numer. Methods Fluids 45, 689–714 (2004) MATHCrossRefGoogle Scholar
  7. 7.
    Atkinson, J.H., Westerink, J.J., Luettich, R.A.: Two-dimensional dispersion analyses of finite element approximations to the shallow water equations. Int. J. Numer. Methods Fluids 45, 715–749 (2004) MATHCrossRefGoogle Scholar
  8. 8.
    Luettich, R.A., Westerink, J.J.: Formulation and numerical implementation of the 2D/3D ADCIRC finite element model version 44.XX. http://adcirc.org/adcirc_theory_2004_12_08.pdf
  9. 9.
    Dawson, C., Westerink, J.J., Feyen, J.C., Pothina, D.: Continuous, discontinuous and coupled discontinuous-continuous Galerkin finite element methods for the shallow water equations. Int. J. Numer. Methods Fluids 52, 63–88 (2006) MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Kubatko, E.J., Westerink, J.J., Dawson, C.: hp discontinuous Galerkin methods for advection dominated problems in shallow water flow. Comput. Methods Appl. Mech. Eng. 196, 437–451 (2006) MATHCrossRefGoogle Scholar
  11. 11.
    Kubatko, E.J., Westerink, J.J., Dawson, C.: Semi-discrete discontinuous Galerkin methods and stage exceeding order strong stability preserving Runge-Kutta time discretizations. J. Comput. Phys. 222, 832–848 (2007) MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Kubatko, E.J., Dawson, C., Westerink, J.J.: Time step restrictions for Runge-Kutta discontinuous Galerkin methods on triangular grids. J. Comput. Phys. 227, 9697–9710 (2008) MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Kubatko, E.J., Bunya, S., Dawson, C., Westerink, J.J., Mirabito, C.: A performance comparison of continuous and discontinuous finite element shallow water models. J. Sci. Comput. 40, 315–339 (2009) MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Bunya, S., Kubatko, E.J., Westerink, J.J., Dawson, C.: A wetting and drying treatment for the Runge-Kutta discontinuous Galerkin solution to the shallow water equations. Comput. Methods Appl. Mech. Eng. 198, 1548–1562 (2009) CrossRefMathSciNetGoogle Scholar
  15. 15.
    Kubatko, E.J., Bunya, S., Dawson, C., Westerink, J.J.: Dynamic p-adaptive Runge-Kutta discontinuous Galerkin methods for the shallow water equations. Comput. Methods Appl. Mech. Eng. 198, 1766–1774 (2009) CrossRefMathSciNetGoogle Scholar
  16. 16.
    Westerink, J.J., Luettich, R.A., Muccino, J.C.: Modeling tides in the western North Atlantic using unstructured graded grids. Tellus 46A, 178–199 (1994) Google Scholar
  17. 17.
    Blain, C.A., Westerink, J.J., Luettich, R.A.: The influence of domain size on the response characteristics of a hurricane storm surge model. J. Geophys. Res. 99(C9), 18467–18479 (1994) CrossRefGoogle Scholar
  18. 18.
    Blain, C.A., Westerink, J.J., Luettich, R.A.: Grid convergence studies for the prediction of hurricane storm surge. Int. J. Numer. Methods Fluids 26, 369–401 (1998) MATHCrossRefGoogle Scholar
  19. 19.
    Mukai, A., Westerink, J.J., Luettich, R.A., Mark, D.: Eastcoast 2001: A tidal constituent database for the western North Atlantic, Gulf of Mexico, and Caribbean Sea. Tec. Rep. ERDC/CHL TR-02-24, U.S. Army Corps of Engineers, 25 (2002) Google Scholar
  20. 20.
    Westerink, J.J., Luettich, R.A., Feyen, J.C., Atkinson, J.H., Dawson, C., Roberts, H.J., Powell, M.D., Dunion, J.P., Kubatko, E.J., Pourtaheri, H.: A basin to channel scale unstructured grid hurricane storm surge model applied to Southern Louisiana. Mon. Weather Rev. 136(3), 833–864 (2008) CrossRefGoogle Scholar
  21. 21.
    Bunya, S., Dietrich, J.C., Westerink, J.J., Ebersole, B.A., Smith, J.M., Atkinson, J.H., Jensen, R., Resio, D.T., Luettich, R.A., Dawson, C., Cardone, V.J., Cox, A.T., Powell, M.D., Westerink, H.J., Roberts, H.J.: A high resolution coupled riverine flow tide, wind, wind wave and storm surge model for Southern Louisiana and Mississippi: part I—model development and validation. Mon. Weather Rev. (2009). doi: 10.1175/2009MWR2907.1 MATHGoogle Scholar
  22. 22.
    Dietrich, J.C., Bunya, S., Westerink, J.J., Ebersole, B.A., Smith, J.M., Atkinson, J.H., Jensen, R., Resio, D.T., Luettich, R.A., Dawson, C., Cardone, V.J., Cox, A.T., Powell, M.D., Westerink, H.J., Roberts, H.J.: A high resolution coupled riverine flow tide, wind, wind wave and storm surge model for Southern Louisiana and Mississippi: part II—synoptic description and analyses of hurricane Katrina and Rita. Mon. Weather Rev. (2009). doi: 10.1175/2009MWR2906.1 Google Scholar
  23. 23.
  24. 24.
    Kincaid, D.R., Respess, J.R., Young, D.M., Grimes, R.G.: ITPACK 2C: A Fortran package for solving large sparse linear systems by adaptive accelerated iterative methods. ACM Trans. Math. Softw. 8(3) (1982) Google Scholar
  25. 25.
    Karypis, G., Kumar, V.: Multilevel k-way partitioning scheme for irregular graphs. J. Parallel Distrib. Comput. 48(1), 96–129 (1998) CrossRefMathSciNetGoogle Scholar
  26. 26.
  27. 27.
  28. 28.
    URS, Final coastal and riverine high-water marks collection for Hurricane Katrina in Louisiana. FEMA-1603-DR-LA, Task Orders 412 and 419, Federal Emergency Management Agency, Washington DC, 76 pp. (2006) Google Scholar
  29. 29.
    Ebersole, B.A., Westerink, J.J., Resio, D.T., Dean, R.G.: Performance Evaluation of the New Orleans and Southeast Louisiana hurricane protection system, vol. IV—The storm. Final Report of the Interagency Performance Evaluation Task Force, U.S. Army Corps of Engineers, Washington, D.C., 263 pp. (2007) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • S. Tanaka
    • 1
  • S. Bunya
    • 2
  • J. J. Westerink
    • 1
  • C. Dawson
    • 3
  • R. A. LuettichJr.
    • 4
  1. 1.Environmental Fluid Dynamics Laboratories, Department of Civil Engineering and Geological SciencesUniversity of Notre DameNotre DameUSA
  2. 2.Science and Safety Policy Research DivisionMitsubishi Research Institute, Inc.TokyoJapan
  3. 3.Institute for Computational Engineering and SciencesThe University of Texas at AustinAustinUSA
  4. 4.Institute of Marine SciencesUniversity of North Carolina at Chapel HillMorehead CityUSA

Personalised recommendations