Advertisement

Adaptive SOR methods based on the Wolfe conditions

  • Yuto MiyatakeEmail author
  • Tomohiro Sogabe
  • Shao-Liang Zhang
Original Paper
  • 21 Downloads

Abstract

Because the expense of estimating the optimal value of the relaxation parameter in the successive over-relaxation (SOR) method is usually prohibitive, the parameter is often adaptively controlled. In this paper, new adaptive SOR methods are presented that are applicable to a variety of symmetric positive definite linear systems and do not require additional matrix-vector products when updating the parameter. To this end, we regard the SOR method as an algorithm for minimising a certain objective function, which yields an interpretation of the relaxation parameter as the step size following a certain change of variables. This interpretation enables us to adaptively control the step size based on some line search techniques, such as the Wolfe conditions. Numerical examples demonstrate the favourable behaviour of the proposed methods.

Keywords

Linear systems SOR methods Discrete gradient methods Optimisation 

Mathematics Subject Classification (2010)

65F10 65L05 65N22 

Notes

Acknowledgements

The authors are grateful for various comments by anonymous referees.

Funding information

This work has been supported in part by JSPS, Japan KAKENHI Grant Numbers 16K17550, 16KT0016, 17H02829, and 18H05392.

References

  1. 1.
    Bai, Z.Z., Chi, X.B.: Asymptotically optimal successive overrelaxation methods for systems of linear equations. J. Comput. Math. 21, 603–612 (2003)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Gonzalez, O.: Time integration and discrete Hamiltonian systems. J. Nonlinear Sci. 6, 449–467 (1996).  https://doi.org/10.1007/s003329900018 MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Grimm, V., McLachlan, R.I., McLaren, D., Quispel, G.R.W., Schönlieb, C.B.: Discrete gradient methods for solving variational image regularisation models. J. Phys. A 50, 295201 (2017).  https://doi.org/10.1088/1751-8121/aa747c MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Hadjidimos, A.: Successive overrelaxation (SOR) and related methods. J. Comput. Appl. Math. 123(1–2), 177–199 (2000).  https://doi.org/10.1016/S0377-0427(00)00403-9. Numerical analysis 2000, Vol. III. Linear algebraMathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Hageman, L.A., Young, D.M.: Applied Iterative Methods. Academic Press, New York (1981)zbMATHGoogle Scholar
  6. 6.
    Hairer, E., Lubich, C.: Energy-diminishing integration of gradient systems. IMA J. Numer. Anal. 34, 452–461 (2014).  https://doi.org/10.1093/imanum/drt031 MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Hairer, E., Wanner, G.: Solving Ordinary Differential Equations II: Stiff and Diffirential-Algebraic Problems. Springer Series in Computational Mathematics, 2nd edn., vol. 14. Springer, Berlin (1996)Google Scholar
  8. 8.
    Itoh, T., Abe, K.: Hamiltonian-conserving discrete canonical equations based on variational difference quotients. J. Comput. Phys. 76, 85–102 (1988).  https://doi.org/10.1016/0021-9991(88)90132-5 MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Matsuo, T., Furihata, D.: A stabilization of multistep linearly implicit schemes for dissipative systems. J. Comput. Appl. Math. 264, 38–48 (2014).  https://doi.org/10.1016/j.cam.2013.12.028 MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    McLachlan, R.I., Quispel, G.R.W., Robidoux, N.: Unified approach to Hamiltonian systems, Poisson systems, gradient systems, and systems with Lyapunov functions or first integrals. Phys. Rev. Lett. 81, 2399–2403 (1998).  https://doi.org/10.1103/PhysRevLett.81.2399 MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    McLachlan, R.I., Quispel, G.R.W., Robidoux, N.: Geometric integration using discrete gradients. R. Soc. Lond. Philos. Trans. Ser. A Math. Phys. Eng. Sci. 357, 1021–1045 (1999).  https://doi.org/10.1098/rsta.1999.0363 MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Meng, G.Y.: A practical asymptotical optimal SOR method. Appl. Math. Comput. 242, 707–715 (2014).  https://doi.org/10.1016/j.amc.2014.06.034 MathSciNetzbMATHGoogle Scholar
  13. 13.
    Miyatake, Y., Sogabe, T., Zhang, S.: On the equivalence between SOR-type methods for linear systems and the discrete gradient methods for gradient systems. J. Comput. Appl. Math. 342, 58–69 (2018).  https://doi.org/10.1016/j.cam.2018.04.013 MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Quispel, G.R.W., McLaren, D.I.: A new class of energy-preserving numerical integration methods. J. Phys. A 41(045), 206 (2008).  https://doi.org/10.1088/1751-8113/41/4/045206 MathSciNetzbMATHGoogle Scholar
  15. 15.
    Quispel, G.R.W., Turner, G.S.: Discrete gradient methods for solving ODEs numerically while preserving a first integral. J. Phys. A 29, L341–L349 (1996).  https://doi.org/10.1088/0305-4470/29/13/006  https://doi.org/10.1088/0305-4470/29/13/006 MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Ren, L., Ren, F., Wen, R.: A selected method for the optimal parameters of the AOR iteration. J. Inequal. Appl. 2016, 279 (2016).  https://doi.org/10.1186/s13660-016-1196-8 MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Ringholm, T., Lazić, J., Schönlieb, C.B.: Variational image regularization with Euler’s elastica using a discrete gradient scheme. SIAM J. Imaging Sci. 11, 2665–2691 (2018).  https://doi.org/10.1137/17M1162354 MathSciNetCrossRefGoogle Scholar
  18. 18.
    Varga, R.S.: Matrix Iterative Analysis, 2nd edn. Springer, Berlin (2000)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Cybermedia CenterOsaka University, ToyonakaOsakaJapan
  2. 2.Department of Applied Physics, Graduate School of EngineeringNagoya UniversityNagoyaJapan

Personalised recommendations