Adaptive SOR methods based on the Wolfe conditions

Abstract

Because the expense of estimating the optimal value of the relaxation parameter in the successive over-relaxation (SOR) method is usually prohibitive, the parameter is often adaptively controlled. In this paper, new adaptive SOR methods are presented that are applicable to a variety of symmetric positive definite linear systems and do not require additional matrix-vector products when updating the parameter. To this end, we regard the SOR method as an algorithm for minimising a certain objective function, which yields an interpretation of the relaxation parameter as the step size following a certain change of variables. This interpretation enables us to adaptively control the step size based on some line search techniques, such as the Wolfe conditions. Numerical examples demonstrate the favourable behaviour of the proposed methods.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Notes

  1. 1.

    A function \(f\in \mathbb {R}^{n}\to \mathbb {R}\) is said to be strictly convex if and only if for all \(\boldsymbol {x},\boldsymbol {y}\in \mathbb {R}^{n}\) (xy) and λ ∈ (0, 1), it holds that f(λx + (1 − λ)y) < λf(x) + (1 − λ)f(y).

  2. 2.

    A function \(f\in \mathbb {R}^{n}\to \mathbb {R}\) is said to be coercive if and only if f(x) → for ∥x∥→.

References

  1. 1.

    Bai, Z.Z., Chi, X.B.: Asymptotically optimal successive overrelaxation methods for systems of linear equations. J. Comput. Math. 21, 603–612 (2003)

    MathSciNet  MATH  Google Scholar 

  2. 2.

    Gonzalez, O.: Time integration and discrete Hamiltonian systems. J. Nonlinear Sci. 6, 449–467 (1996). https://doi.org/10.1007/s003329900018

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    Grimm, V., McLachlan, R.I., McLaren, D., Quispel, G.R.W., Schönlieb, C.B.: Discrete gradient methods for solving variational image regularisation models. J. Phys. A 50, 295201 (2017). https://doi.org/10.1088/1751-8121/aa747c

    MathSciNet  Article  MATH  Google Scholar 

  4. 4.

    Hadjidimos, A.: Successive overrelaxation (SOR) and related methods. J. Comput. Appl. Math. 123(1–2), 177–199 (2000). https://doi.org/10.1016/S0377-0427(00)00403-9. Numerical analysis 2000, Vol. III. Linear algebra

    MathSciNet  Article  Google Scholar 

  5. 5.

    Hageman, L.A., Young, D.M.: Applied Iterative Methods. Academic Press, New York (1981)

    Google Scholar 

  6. 6.

    Hairer, E., Lubich, C.: Energy-diminishing integration of gradient systems. IMA J. Numer. Anal. 34, 452–461 (2014). https://doi.org/10.1093/imanum/drt031

    MathSciNet  Article  MATH  Google Scholar 

  7. 7.

    Hairer, E., Wanner, G.: Solving Ordinary Differential Equations II: Stiff and Diffirential-Algebraic Problems. Springer Series in Computational Mathematics, 2nd edn., vol. 14. Springer, Berlin (1996)

  8. 8.

    Itoh, T., Abe, K.: Hamiltonian-conserving discrete canonical equations based on variational difference quotients. J. Comput. Phys. 76, 85–102 (1988). https://doi.org/10.1016/0021-9991(88)90132-5

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Matsuo, T., Furihata, D.: A stabilization of multistep linearly implicit schemes for dissipative systems. J. Comput. Appl. Math. 264, 38–48 (2014). https://doi.org/10.1016/j.cam.2013.12.028

    MathSciNet  Article  MATH  Google Scholar 

  10. 10.

    McLachlan, R.I., Quispel, G.R.W., Robidoux, N.: Unified approach to Hamiltonian systems, Poisson systems, gradient systems, and systems with Lyapunov functions or first integrals. Phys. Rev. Lett. 81, 2399–2403 (1998). https://doi.org/10.1103/PhysRevLett.81.2399

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    McLachlan, R.I., Quispel, G.R.W., Robidoux, N.: Geometric integration using discrete gradients. R. Soc. Lond. Philos. Trans. Ser. A Math. Phys. Eng. Sci. 357, 1021–1045 (1999). https://doi.org/10.1098/rsta.1999.0363

    MathSciNet  Article  MATH  Google Scholar 

  12. 12.

    Meng, G.Y.: A practical asymptotical optimal SOR method. Appl. Math. Comput. 242, 707–715 (2014). https://doi.org/10.1016/j.amc.2014.06.034

    MathSciNet  Article  MATH  Google Scholar 

  13. 13.

    Miyatake, Y., Sogabe, T., Zhang, S.: On the equivalence between SOR-type methods for linear systems and the discrete gradient methods for gradient systems. J. Comput. Appl. Math. 342, 58–69 (2018). https://doi.org/10.1016/j.cam.2018.04.013

    MathSciNet  Article  MATH  Google Scholar 

  14. 14.

    Quispel, G.R.W., McLaren, D.I.: A new class of energy-preserving numerical integration methods. J. Phys. A 41(045), 206 (2008). https://doi.org/10.1088/1751-8113/41/4/045206

    MathSciNet  Article  MATH  Google Scholar 

  15. 15.

    Quispel, G.R.W., Turner, G.S.: Discrete gradient methods for solving ODEs numerically while preserving a first integral. J. Phys. A 29, L341–L349 (1996). https://doi.org/10.1088/0305-4470/29/13/006 https://doi.org/10.1088/0305-4470/29/13/006

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    Ren, L., Ren, F., Wen, R.: A selected method for the optimal parameters of the AOR iteration. J. Inequal. Appl. 2016, 279 (2016). https://doi.org/10.1186/s13660-016-1196-8

    MathSciNet  Article  MATH  Google Scholar 

  17. 17.

    Ringholm, T., Lazić, J., Schönlieb, C.B.: Variational image regularization with Euler’s elastica using a discrete gradient scheme. SIAM J. Imaging Sci. 11, 2665–2691 (2018). https://doi.org/10.1137/17M1162354

    MathSciNet  Article  MATH  Google Scholar 

  18. 18.

    Varga, R.S.: Matrix Iterative Analysis, 2nd edn. Springer, Berlin (2000)

    Google Scholar 

Download references

Acknowledgements

The authors are grateful for various comments by anonymous referees.

Funding

This work has been supported in part by JSPS, Japan KAKENHI Grant Numbers 16K17550, 16KT0016, 17H02829, and 18H05392.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yuto Miyatake.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Miyatake, Y., Sogabe, T. & Zhang, S. Adaptive SOR methods based on the Wolfe conditions. Numer Algor 84, 117–132 (2020). https://doi.org/10.1007/s11075-019-00748-0

Download citation

Keywords

  • Linear systems
  • SOR methods
  • Discrete gradient methods
  • Optimisation

Mathematics Subject Classification (2010)

  • 65F10
  • 65L05
  • 65N22