Skip to main content

Advertisement

SpringerLink
Log in

Efficient use of sparsity by direct solvers applied to 3D controlled-source EM problems

  • Original Paper
  • Published:
Computational Geosciences Aims and scope Submit manuscript

Abstract

Controlled-source electromagnetic (CSEM) surveying becomes a widespread method for oil and gas exploration, which requires fast and efficient software for inverting large-scale EM datasets. In this context, one often needs to solve sparse systems of linear equations with a large number of sparse right-hand sides, each corresponding to a given transmitter position. Sparse direct solvers are very attractive for these problems, especially when combined with low-rank approximations which significantly reduce the complexity and the cost of the factorization. In the case of thousands of right-hand sides, the time spent in the sparse triangular solve tends to dominate the total simulation time, and here we propose several approaches to reduce it. A significant reduction is demonstrated for marine CSEM application by utilizing the sparsity of the right-hand sides (RHS) and of the solutions that results from the geometry of the problem. Large gains are achieved by restricting computations at the forward substitution stage to exploit the fact that the RHS matrix might have empty rows (vertical sparsity) and/or empty blocks of columns within a non-empty row (horizontal sparsity). We also adapt the parallel algorithms that were designed for the factorization to solve-oriented algorithms and describe performance optimizations particularly relevant for the very large numbers of right-hand sides of the CSEM application. We show that both the operation count and the elapsed time for the solution phase can be significantly reduced. The total time of CSEM simulation can be divided by approximately a factor of 3 on all the matrices from our set (from 3 to 30 million unknowns, and from 4 to 12 thousands RHSs).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Amestoy, P.R., Buttari, A., L’Excellent, J.Y., Mary, T.: On the complexity of the block low-rank multifrontal factorization. SIAM J. Sci. Comput. 39(4), A1710–A1740 (2017). https://doi.org/10.1137/16M1077192

    Article  Google Scholar 

  2. Amestoy, P.R., Buttari, A., L’Excellent, J.Y., Mary, T.: Performance and scalability of the block low-rank multifrontal factorization on multicore architectures. ACM Trans. Math. Softw. 45(1), 2:1–2:26 (2019). https://doi.org/10.1145/3242094

    Article  Google Scholar 

  3. Amestoy, P.R., Duff, I.S., Koster, J., L’Excellent, J.Y.: A fully asynchronous multifrontal solver using distributed dynamic scheduling. SIAM J Matrix Anal Appl 23(1), 15–41 (2001)

    Article  Google Scholar 

  4. Amestoy, P.R., Duff, I.S., L’Excellent, J.Y., Rouet, F.H.: Parallel computation of entries of A-1. SIAM J. Sci. Comput. 37(2), C268–C284 (2015)

    Article  Google Scholar 

  5. Amestoy, P.R., Guermouche, A., L’Excellent, J.Y., Pralet, S.: Hybrid scheduling for the parallel solution of linear systems. Parallel Comput. 32(2), 136–156 (2006)

    Article  Google Scholar 

  6. Amestoy, P.R., L’Excellent, J.Y., Moreau, G.: On exploiting sparsity of multiple right-hand sides in sparse direct solvers. SIAM J. Sci. Comput. 41, A269–A291 (2019)

    Article  Google Scholar 

  7. Amestoy, P.R., L’Excellent, J.Y., Rouet, F.H., Sid-Lakhdar, W.M.: Modeling 1D distributed-memory dense kernels for an asynchronous multifrontal sparse solver. In: High Performance Computing for Computational Science, VECPAR 2014 - 11th International Conference, Eugene, Oregon, USA, June 30 - July 3, 2014, Revised Selected Papers, pp. 156–169 (2014)

    Google Scholar 

  8. Avdeev, D.B.: Three-dimensional electromagnetic modelling and inversion from theory to application. Surv. Geophys. 26(6), 767–799 (2005). https://doi.org/10.1007/s10712-005-1836-x

    Article  Google Scholar 

  9. Blackford, L.S., Choi, J., Cleary, A., D’Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., Whaley, R.C.: ScaLAPACK users’ guide. SIAM Press (1997)

  10. Börner, R.U.: Numerical modelling in geo-electromagnetics: advances and challenges. Surv. Geophys. 31(2), 225–245 (2010). https://doi.org/10.1007/s10712-009-9087-x

    Article  Google Scholar 

  11. Constable, S.: Ten years of marine CSEM for hydrocarbon exploration. Geophysics 75(5), 75A67–75A81 (2010). https://doi.org/10.1190/1.3483451

    Article  Google Scholar 

  12. Dongarra, J.J., Du Croz, J., Duff, I.S., Hammarling, S.: Algorithm 679: a set of level 3 basic linear algebra subprograms. ACM Trans. Math. Softw. 16, 1–17 (1990)

    Article  Google Scholar 

  13. Duff, I.S., Erisman, A.M., Reid, J.K.: Direct Methods for Sparse Matrices, 2nd edn. Oxford University Press, London (2017)

    Book  Google Scholar 

  14. Duff, I.S., Reid, J.K.: The multifrontal solution of indefinite sparse symmetric linear systems. ACM Trans. Math. Softw. 9, 302–325 (1983)

    Article  Google Scholar 

  15. Ellingsrud, S., Eidesmo, T., Johansen, S., Sinha, M.C., MacGregor, L.M., Constable, S.: Remote sensing of hydrocarbon layers by seabed logging (SBL): results from a cruise offshore Angola. Lead. Edge 21(10), 972–982 (2002). https://doi.org/10.1190/1.1518433

    Article  Google Scholar 

  16. George, J.A.: Nested dissection of a regular finite-element mesh. SIAM J. Numer. Anal. 10(2), 345–363 (1973)

    Article  Google Scholar 

  17. Gilbert, J.R.: Predicting structure in sparse matrix computations. SIAM J. Matrix Anal. Appl. 15, 62–79 (1994)

    Article  Google Scholar 

  18. Gilbert, J.R., Liu, J.W.H.: Elimination structures for unsymmetric sparse LU factors. SIAM J. Matrix Anal. Appl. 14, 334–352 (1993)

    Article  Google Scholar 

  19. Hanssen, P., Nguyen, A.K., Fogelin, L.T.T., Jensen, H.R., Skaro, M., Mittet, R., Rosenquist, M., Suilleabhain, L.O., van der Sman, P.: The next generation offshore CSEM acquisition system, pp. 1194–1198. Society of Exploration Geophysicists. https://library.seg.org/doi/abs/10.1190/segam2017-17725809.1 (2017)

  20. Hiner, M., Martinez, Y., Sun, S.: Delineating salt bodies with 3D CSEM technology. In: Salt Challenges in Hydrocarbon Exploration, SEG Annual Meeting Post-convention Workshop. New Orleans (2015)

  21. Lötstedt, P., Nilsson, M.: A minimal residual interpolation method for linear equations with multiple right-hand sides. SIAM J. Sci. Comput. 25(6), 2126–2144 (2004)

    Article  Google Scholar 

  22. Mary, T.: Block low-rank multifrontal solvers: complexity, performance, and scalability. PhD thesis, Université de Toulouse (2017)

  23. Nguyen, A.K., Nordskag, J.I., Wiik, T., Bjorke, A.K., Boman, L., Pedersen, O.M., Ribaudo, J., Mittet, R.: Comparing large-scale 3D Gauss-Newton and BFGS CSEM inversions, pp. 872–877. Society of Exploration Geophysicists (2016). https://doi.org/10.1190/segam2016-13858633.1

  24. Plessix, R.E., Darnet, M., Mulder, W.A.: An approach for 3D multisource, multifrequency CSEM modeling. Geophysics 72(5), SM177–SM184 (2007)

    Article  Google Scholar 

  25. Pothen, A., Sun, C.: A mapping algorithm for parallel sparse Cholesky factorization. SIAM J. Sci. Comput. 14(5), 1253–1257 (1993)

    Article  Google Scholar 

  26. Rouet, F.H.: Memory and performance issues in parallel multifrontal factorizations and triangular solutions with sparse right-hand sides. PhD thesis, Institut National Polytechnique de Toulouse (2012)

  27. Shantsev, D., Jaysaval, P., de la Kethulle de Ryhove, S., Amestoy, P.R., Buttari, A., L’Excellent, J.Y., Mary, T.: Large-scale 3-D EM modeling with a Block Low-Rank multifrontal direct solver. Geophys. J. Int. 209(3), 1558–1571 (2017)

    Article  Google Scholar 

  28. Slavova, Tz.: Parallel triangular solution in the out-of-core multifrontal approach for solving large sparse linear systems. Ph.D. dissertation, Institut National Polytechnique de Toulouse (2009). Available as CERFACS Report TH/PA/09/59

  29. Streich, R.: Controlled-source electromagnetic approaches for hydrocarbon exploration and monitoring on land. Surv. Geophys. 37(1), 47–80 (2016). https://doi.org/10.1007/s10712-015-9336-0

    Article  Google Scholar 

  30. Yee, K.: Numerical solution of initial boundary value problems involving Maxwell’s equations in isotropic media. IEEE Trans. Antennas Propag. 14(3), 302–307 (1966)

    Article  Google Scholar 

  31. Zach, J., Bjorke, A., Storen, T., Maao, F.: 3D inversion of marine CSEM data using a fast finite-difference time-domain forward code and approximate Hessian-based optimization. In: SEG Technical Program Expanded Abstracts 2008, pp. 614–618 (2008). https://library.seg.org/doi/abs/10.1190/1.3063726

Download references

Funding

This work was partially supported by the MUMPS consortium and by LABEX MILYON (ANR-10-LABX-0070) of Université de Lyon, within the program “Investissements d’Avenir” (ANR-11-IDEX-0007) operated by the French National Research Agency (ANR).

Author information

Author notes

  1. Patrick R. Amestoy & Jean-Yves L’Excellent

    Present address: Mumps Technologies, Lyon, France

  2. Sébastien de la Kethulle de Ryhove

    Present address: Kongsberg Defence & Aerospace, Asker, Norway

Authors and Affiliations

  1. University Toulouse, INPT, IRIT UMR5505, Toulouse, France

    Patrick R. Amestoy

  2. EMGS, Trondheim, Norway

    Sébastien de la Kethulle de Ryhove

  3. University Lyon, CNRS, ENS Lyon, Inria, UCBL, LIP UMR5668, Lyon, France

    Jean-Yves L’Excellent & Gilles Moreau

  4. EMGS, I&I Technology Center, Oslo, Norway

    Daniil V. Shantsev

Authors
  1. Patrick R. Amestoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sébastien de la Kethulle de Ryhove
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean-Yves L’Excellent
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gilles Moreau
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daniil V. Shantsev
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jean-Yves L’Excellent.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Amestoy, P.R., de la Kethulle de Ryhove, S., L’Excellent, JY. et al. Efficient use of sparsity by direct solvers applied to 3D controlled-source EM problems. Comput Geosci 23, 1237–1258 (2019). https://doi.org/10.1007/s10596-019-09883-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10596-019-09883-y

Keywords

Mathematics Subject Classification (2010)

Search

Navigation