An efficient parallel algorithm for 3D magnetotelluric modeling with edge-based finite element

Computational Geosciences volume 25, pages1–16(2021)Cite this article

Abstract

Three-dimensional magnetotelluric modeling algorithm of high accuracy and high efficiency is required for data interpretation and inversion. In this paper, edge-based finite element method with unstructured mesh is used to solve 3D magnetotelluric problem. Two boundary conditions—Dirichlet boundary condition and Neumann boundary condition—are set for cross-validation and comparison. We propose an efficient parallel algorithm to speed up computation and improve efficiency. The algorithm is based on distributed matrix storage and has three levels of parallelism. The first two are process level parallelization for frequencies and matrix solving, and the last is thread-level parallelization for loop unrolling. The algorithm is validated by several model studies. Scalability tests have been performed on two distributed-memory HPC platforms, one consists of Intel Xeon E5-2660 microprocessors and the other consists of Phytium FT2000 Plus microprocessors. On Intel platform, computation time of our algorithm solving Dublin Test Model-1 with 3,756,373 edges at 21 frequencies is 365 s on 2520 cores. The speedup and efficiency are 1609 and 60% compared to 100 cores. On Phytium platform, scalability test shows that the speedup from 256 cores to 86,016 cores has been increased to 11,255.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

References

  1. 1.

    Ansari, S., Farquharson, C.: 3D finite-element forward modeling of electromagnetic data using vector and scalar potentials and unstructured grids. Geophysics. 79(4), E149–E165 (2014)

    Article  Google Scholar 

  2. 2.

    Arun, S., Rahul, D., Pravin, K.G., et al.: A MATLAB based 3D modeling and inversion code for MT data. Comput Geosci. 104, 1–11 (2017)

    Article  Google Scholar 

  3. 3.

    Cai, H., Hu, X., Li, J., Endo, M., Xiong, B.: Parallelized 3D CSEM modeling using edge-based finite element with total field formulation and unstructured mesh. Comput Geosci. 99, 125–134 (2017)

    Article  Google Scholar 

  4. 4.

    Castillo-Reyes, O., de la Puente, J., & Cela, J. M. (2018). PETGEM: A parallel code for 3D CSEM forward modeling using edge finite elements. Comput Geosci. 119, 123–136. https://doi.org/10.1016/j.cageo.2018.07.005

  5. 5.

    Demmel, J.W., Gilbert, J.R., Li, X.S.: An asynchronous parallel supernodal algorithm for sparse Gaussian elimination. SIAM J Matrix Anal Appl. 20, 915–952 (1999)

    Article  Google Scholar 

  6. 6.

    Elwaseif, M., Robinson, J., Day-Lewis, F.D., Ntarlagiannis, D., Slater, L.D., Lane, J.W., Minsley, B.J., Schultz, G.: A matlab-based frequency-domain electromagnetic inversion code (FEMIC) with graphical user interface. Comput Geosci. 99, 61–71 (2017)

    Article  Google Scholar 

  7. 7.

    Farquharson, C.G., Miensopust, M.P.: Three-dimensional finite-element modelling of magnetotelluric data with a divergence correction. J Appl Geophys. 75, 699–710 (2011)

    Article  Google Scholar 

  8. 8.

    Farquharson, C.G., Oldenburg, D.W., Haber, E., et al., 2002. An Algorithm for the Three-Dimensional Inversion of Magnetotelluric Data, in Proceedings of the 72nd Annual Meeting of the Society of Exploration Geophysicists, Salt Lake City, Utah

  9. 9.

    Geuzaine, C., Remacle, J.F.: Gmsh: a three-dimensional finite element mesh generator with built-in pre- and post-processing facilities. Int J Numer Meth Eng. 79(11), 1309–1331 (2009)

  10. 10.

    Haber, E.: Computational methods in geophysical electromagnetics 1. SIAM, Philadelphia (2014)

    Google Scholar 

  11. 11.

    Jahandari, H., Farquharson, C.G.: Finite-volume modelling of geophysical electromagnetic data on unstructured grids using potentials. Geophys J Int. 202, 1859–1876 (2015)

    Article  Google Scholar 

  12. 12.

    Jin, J.: Finite Element Method in Electromagnetics, Second edn. Wiley-IEEE Press, New York (2002)

  13. 13.

    Li, X.S., Demmel, J.W.: A scalable distributed-memory sparse direct solver for unsymmetric linear systems. ACM Trans Math Softw. 29, 110–140 (2003)

    Article  Google Scholar 

  14. 14.

    Liu, Y., Xu, Z., Li, Y.: Adaptive finite element modelling of three-dimensional magnetotelluric fields in general anisotropic media. J Appl Geophys. 151, 113–124 (2018)

    Article  Google Scholar 

  15. 15.

    Mackie, R.L., Smith, J.T., Madden, T.R.: Three-dimensional electromagnetic modeling using finite difference equations: the magnetotelluric example. Radio Sci. 29, 923–935 (1994)

    Article  Google Scholar 

  16. 16.

    Marion, P.M., Pilar, Q., Alan, G.J., et al.: Magnetotelluric 3-D inversion—a review of two successful workshops on forward and inversion code testing and comparison. Geophys J Int. 193, 1216–1238 (2013)

    Article  Google Scholar 

  17. 17.

    Mitsuhata, Y., Uchida, T.: 3D magnetotelluric modeling using the T-Omega finite-element method. Geophysics. 69, 108–119 (2004)

    Article  Google Scholar 

  18. 18.

    Nam, M.J., Kim, H.J., Yoonho, S., et al.: Three-dimensional topography corrections ofmagnetotelluric data. Geophys J Int. 174(2), 464–474 (2007)

    Article  Google Scholar 

  19. 19.

    Nédélec, J.C.: Mixed finite elements in R3. Numer Math. 35, 315–341 (1980)

    Article  Google Scholar 

  20. 20.

    Newman, G.A.: A review of high-performance computational strategies for modeling and imaging of electromagnetic induction data. Surv Geophys. 35, 85–100 (2014)

    Article  Google Scholar 

  21. 21.

    Octavio, C.R., Josep, D.L.P., José, M.C.: 3PETGEM: a parallel code for 3D CSEM forward modeling using edge finite elements. Comput Geosci. 119, 123–136 (2018)

    Article  Google Scholar 

  22. 22.

    Puzyrev, V., Cela, J.M.: A review of block Krylov subspace methods for multisource electromagnetic modelling. Geophys J Int. 202, 1241–1252 (2015)

    Article  Google Scholar 

  23. 23.

    Puzyrev, V., Koldan, J., Puente, G.J., et al.: A parallel finite-element method for three-dimensional controlled-source electromagnetic forward modelling. Geophys J Int. 193, 678–693 (2013)

    Article  Google Scholar 

  24. 24.

    Puzyrev, V., Koric, S., Wilkin, S.: Evaluation of parallel direct sparse linear solvers in electromagnetic geophysical problems. Comput Geosci. 89, 79–87 (2016)

    Article  Google Scholar 

  25. 25.

    Ren, Z., Kalscheuer, T., Greenhalgh, S., Maurer, H.: A goal-oriented adaptive finite-element approach for plane wave 3-D electromagnetic modelling. Geophys J Int. 194, 700–718 (2013)

    Article  Google Scholar 

  26. 26.

    Silva, N.V., Morgan, J.V., MacGregor, L., et al.: A finite element multifrontal method for 3D CSEM modeling in the frequency domain. Geophysics. 77, E101–E115 (2012)

    Article  Google Scholar 

  27. 27.

    Stratton, J.: Electromagnetic Theory. Wiley-IEEE Press, New Jersey (2007)

    Google Scholar 

  28. 28.

    Um, E.S.: Three-Dimensional finite-Element Time-Domain modeling of the marine controlled-Source electromagnetic method. Ph.D Dissertation. Stanford University (2011)

  29. 29.

    Um, E.S., Harris, J.M., Alumbaugh, D.L.: An iterative finite element time-domain method for simulating three-dimensional electromagnetic diffusion in earth. Geophys J Int. 190, 871–886 (2012)

    Article  Google Scholar 

  30. 30.

    Um, E.S., Commer, M., Newman, G.A.: Efficient pre-conditioned iterative solution strategies for the electromagnetic diffusion in the earth: finite-element frequency-domain approach. Geophys J Int. 193, 1460–1473 (2013)

    Article  Google Scholar 

  31. 31.

    Xiao, T., Liu, Y., Wang, Y., Fu, L.Y.: Three-dimensional magnetotelluric modeling in anisotropic media using edge-based finite element method. J Appl Geophys. 149, 1–9 (2018)

    Article  Google Scholar 

  32. 32.

    Xiong, B.: 2.5-D forward for transient electromagnetic response of a block linear resistivity distribution. J Geophys Eng. 8, 115–121 (2011)

    Article  Google Scholar 

  33. 33.

    Xiong, B., Luo, T., Chen, L.: Direct solutions of 3-D magnetotelluric fields using edge-based finite element. J Appl Geophys. 159, 204–208 (2018)

    Article  Google Scholar 

  34. 34.

    Zhdanov, M.S., Varentsov, I.M., Weaver, J.T., et al.: Methods for modelling electromagnetic fields Results from COMMEMI—the international project on the comparison of modelling methods for electromagnetic induction. J Appl Geophys. 37(3–4), 133–271 (1997)

    Article  Google Scholar 

Download references

Acknowledgments

This research work was supported by the National Key Research and Development Program of China (2018YFB0204301) and the National Natural Science Foundation of China No. 41874145. The authors gratefully acknowledge to Professor Zhengyong Ren, Dr. Fen Zhou, and Dr. Tiaojie Xiao for their valuable discussions.

Computer code availability

The code of the algorithm in this paper can refer to https://github.com/zxxsm/MT3DSZ.git. The code can run under Linux platforms.

Author information

Affiliations

  1. Science and Technology on Parallel and Distributed Processing Laboratory, National University of Defense Technology, Changsha, 410073, China

    Xiaoxiong Zhu, Jie Liu & Chunye Gong

  2. Laboratory of Software Engineering for Complex Systems, National University of Defense Technology, Changsha, 410073, China

    Xiaoxiong Zhu, Jie Liu & Chunye Gong

  3. School of Geosciences and Info-Physics, Central South University, Changsha, 410083, China

    Yian Cui

Authors
  1. Xiaoxiong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yian Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chunye Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Xiaoxiong Zhu performed parallel algorithm implementation, data analysis, scalability tests, and drafted the manuscript; Jie Liu designed the parallel algorithm and revised the manuscript; Yian Cui contributed to magnetotelluric theory; Chunye Gong contributed to scalability tests.

Corresponding author

Correspondence to Jie Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhu, X., Liu, J., Cui, Y. et al. An efficient parallel algorithm for 3D magnetotelluric modeling with edge-based finite element. Comput Geosci 25, 1–16 (2021). https://doi.org/10.1007/s10596-020-09976-z

Download citation

Keywords

  • Magnetotelluric
  • Edge-based finite element
  • Unstructured mesh
  • Parallel algorithm