Skip to main content

High Order Seismic Simulations on the Intel Xeon Phi Processor (Knights Landing)

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9697))

Abstract

We present a holistic optimization of the ADER-DG finite element software SeisSol targeting the Intel\(^{\textregistered }\) Xeon Phi\(^\mathrm{TM}\) x200 processor, codenamed Knights Landing (KNL). SeisSol is a multi-physics software package performing earthquake simulations by coupling seismic wave propagation and the rupture process. The code was shown to scale beyond 1.5 million cores and achieved petascale performance when using local time stepping for the computationally heavy seismic wave propagation. Advancing further along these lines, we discuss the utilization of KNL’s core features, the exploitation of its two-level memory subsystem (which allows for efficient out-of-core implementations), and optimizations targeting at KNL’s 2D mesh on-die interconnect. Our performance comparisons demonstrate that KNL is able to outperform its previous generation, the Intel\(^{\textregistered }\) Xeon Phi coprocessor x100 family, by more than 2.9\(\times \) in time-to-solution. Additionally, our results show a 3.4\(\times \) speedup compared to latest Intel\(^{\textregistered }\) Xeon\(^{\textregistered }\) E5v3 CPUs.

Optimization Notice: Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to http://www.intel.com/performance.

Intel, Xeon, and Intel Xeon Phi are trademarks of Intel Corporation in the U.S. and/or other countries.

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://github.com/SeisSol/SeisSol, git-tag 201511 was used in this paper.

  2. 2.

    TSX instructions, however, are not considered to be legacy x86 instructions.

  3. 3.

    https://www.github.com/memkind/memkind.

  4. 4.

    https://github.com/hfp/libxsmm.

References

  1. Benjemaa, M., et al.: 3-D dynamic rupture simulations by a finite volume method. Geophys. J. Int. 178, 541–560 (2009)

    Article  Google Scholar 

  2. Bielak, J., et al.: Parallel octree-based finite element method for large-scale earthquake ground motion simulation. Comput. Model. Eng. Sci. 10(2), 99 (2005)

    MathSciNet  MATH  Google Scholar 

  3. Bielak, J., et al.: The shakeout earthquake scenario: verification of three simulation sets. Geophys. J. Int. 180(1), 375–404 (2010)

    Article  Google Scholar 

  4. Borstnik, U., et al.: Sparse matrix multiplication: the distributed block-compressed sparse row library. Parallel Comput. 40(5–6), 47–58 (2014)

    Article  MathSciNet  Google Scholar 

  5. Breuer, A., et al.: High-order ADER-DG minimizes energy- and time-to-solution of SeisSol. In: Kunkel, J.M., Ludwig, T. (eds.) ISC High Performance 2015. LNCS, vol. 9137, pp. 340–357. Springer, Heidelberg (2015)

    Chapter  Google Scholar 

  6. Breuer, A., et al.: Petascale local time stepping for the ADER-DG finite element method. In: Proceedings of IPDPS 2016 (2016). To appear

    Google Scholar 

  7. Breuer, A., et al.: Sustained petascale performance of seismic simulations with SeisSol on SuperMUC. In: Kunkel, J.M., Ludwig, T., Meuer, H.W. (eds.) ISC 2014. LNCS, vol. 8488, pp. 1–18. Springer, Heidelberg (2014)

    Google Scholar 

  8. Carrington, L., et al.: High-frequency simulations of global seismic wave propagation using SPECFEM3D_GLOBE on 62K processors. In: Proceedings of SC 2008 (2008)

    Google Scholar 

  9. Cui, Y., et al.: Physics-based seismic hazard analysis on petascale heterogeneous supercomputers. In: Proceedings of SC 2013 (2013)

    Google Scholar 

  10. Cui, Y., et al.: Scalable earthquake simulation on petascale supercomputers. In: Proceedings of SC 2010 (2010)

    Google Scholar 

  11. Day, S.M., et al.: Tests of 3D elastodynamic codes: final report for lifelines project 1A02. Pacific Earthquake Engineering Research Center (2003)

    Google Scholar 

  12. de la Puente, J., et al.: An arbitrary high-order discontinuous galerkin method for elastic waves on unstructured meshes-IV. Anisotropy. Geophys. J. Int. 169(3), 1210–1228 (2007)

    Article  Google Scholar 

  13. de la Puente, L., et al.: Dynamic rupture modeling on unstructured meshes using a discontinuous Galerkin method. J. Geophys. Res. 114, B10302 (2009)

    Article  Google Scholar 

  14. Dumbser, M., et al.: An arbitrary high-order discontinuous Galerkin method for elastic waves on unstructured meshes-II. The three-dimensional isotropic case. Geophys. J. Int. 167(1), 319–336 (2006)

    Article  MathSciNet  Google Scholar 

  15. Harris, R.A., et al.: The SCEC/USGS dynamic earthquake rupture code verification exercise. Seismol. Res. Lett. 80(1), 119–126 (2009)

    Article  Google Scholar 

  16. Heinecke, A., et al.: Petascale high order dynamic rupture earthquake simulations on heterogeneous supercomputers. In: Proceedings of SC 2014. Gordon Bell Finalist (2014)

    Google Scholar 

  17. Ichimura, T., et al.: Implicit nonlinear wave simulation with 1.08 T DOF and 0.270 T unstructured finite elements to enhance comprehensive earthquake simulation. In: Proceedings of SC 2015 (2015)

    Google Scholar 

  18. Ichimura, I., et al.: Physics-based urban earthquake simulation enhanced by 10.7 BLNDOF \(\times \, 30\) K time-step unstructured fe non-linear seismic wave simulation. In: Proceedings of SC 2014 (2014)

    Google Scholar 

  19. Intel Corporation: Intel(R) 64 and IA-32 Architectures Optimization Reference Manual, January 2016

    Google Scholar 

  20. Käser, M., et al.: An arbitrary high-order discontinuous Galerkin method for elastic waves on unstructured meshes-III. Viscoelastic attenuation. Geophys. J. Int. 168(1), 224–242 (2007)

    Article  Google Scholar 

  21. Komatitsch, D., et al.: High-order finite-element seismic wave propagation modeling with MPI on a large GPU cluster. J. Comput. Phys. 229(20), 7692–7714 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  22. Pelties, C., et al.: Three-dimensional dynamic rupture simulation with a high-order discontinuous Galerkin method on unstructured tetrahedral meshes. J. Geophys. Res. 117, B02309 (2012)

    Article  Google Scholar 

  23. Pelties, C., et al.: Verification of an ADER-DG method for complex dynamic rupture problems. Geosci. Model Dev. Discuss. 6, 5981–6034 (2013)

    Article  Google Scholar 

  24. Shin, J., et al.: Speeding up Nek5000 with autotuning and specialization. In: Proceedings of the 24th ACM International Conference on Supercomputing (ICS 2010), pp. 253–262. ACM, New York (2010)

    Google Scholar 

  25. Sodani, A.: Knights Landing (KNL): 2nd generation Intel(R) Xeon Phi(TM) processor. In: Hotchips-2015 (2015)

    Google Scholar 

  26. Sodani, A., et al.: Knights Landing (KNL): 2nd generation Intel(R) Xeon Phi(TM) processor. IEEE Micro, Hot Chips Special Issue, (2016, to appear)

    Google Scholar 

  27. Tago, J., et al.: A 3D hp-adaptive discontinuous Galerkin method for modeling earthquake dynamics. J. Geophys. Res. 117, B09312 (2012)

    Article  Google Scholar 

  28. Tu, T., et al.: From mesh generation to scientific visualization: an end-to-end approach to parallel supercomputing. In: Proceedings of SC 2006 (2006)

    Google Scholar 

  29. Wilcox, L.C., et al.: A high-order discontinuous Galerkin method for wave propagation through coupled elastic-acoustic media. J. Comput. Phys. 229(24), 9373–9396 (2010)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Alexander Heinecke .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Heinecke, A., Breuer, A., Bader, M., Dubey, P. (2016). High Order Seismic Simulations on the Intel Xeon Phi Processor (Knights Landing). In: Kunkel, J., Balaji, P., Dongarra, J. (eds) High Performance Computing. ISC High Performance 2016. Lecture Notes in Computer Science(), vol 9697. Springer, Cham. https://doi.org/10.1007/978-3-319-41321-1_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41321-1_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-41320-4

  • Online ISBN: 978-3-319-41321-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics