Adaptive SOR methods based on the Wolfe conditions

  • Yuto MiyatakeEmail author
  • Tomohiro Sogabe
  • Shao-Liang Zhang
Original Paper


Because the expense of estimating the optimal value of the relaxation parameter in the successive over-relaxation (SOR) method is usually prohibitive, the parameter is often adaptively controlled. In this paper, new adaptive SOR methods are presented that are applicable to a variety of symmetric positive definite linear systems and do not require additional matrix-vector products when updating the parameter. To this end, we regard the SOR method as an algorithm for minimising a certain objective function, which yields an interpretation of the relaxation parameter as the step size following a certain change of variables. This interpretation enables us to adaptively control the step size based on some line search techniques, such as the Wolfe conditions. Numerical examples demonstrate the favourable behaviour of the proposed methods.


Linear systems SOR methods Discrete gradient methods Optimisation 

Mathematics Subject Classification (2010)

65F10 65L05 65N22 



The authors are grateful for various comments by anonymous referees.

Funding information

This work has been supported in part by JSPS, Japan KAKENHI Grant Numbers 16K17550, 16KT0016, 17H02829, and 18H05392.


  1. 1.
    Bai, Z.Z., Chi, X.B.: Asymptotically optimal successive overrelaxation methods for systems of linear equations. J. Comput. Math. 21, 603–612 (2003)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Gonzalez, O.: Time integration and discrete Hamiltonian systems. J. Nonlinear Sci. 6, 449–467 (1996). MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Grimm, V., McLachlan, R.I., McLaren, D., Quispel, G.R.W., Schönlieb, C.B.: Discrete gradient methods for solving variational image regularisation models. J. Phys. A 50, 295201 (2017). MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Hadjidimos, A.: Successive overrelaxation (SOR) and related methods. J. Comput. Appl. Math. 123(1–2), 177–199 (2000). Numerical analysis 2000, Vol. III. Linear algebraMathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Hageman, L.A., Young, D.M.: Applied Iterative Methods. Academic Press, New York (1981)zbMATHGoogle Scholar
  6. 6.
    Hairer, E., Lubich, C.: Energy-diminishing integration of gradient systems. IMA J. Numer. Anal. 34, 452–461 (2014). MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Hairer, E., Wanner, G.: Solving Ordinary Differential Equations II: Stiff and Diffirential-Algebraic Problems. Springer Series in Computational Mathematics, 2nd edn., vol. 14. Springer, Berlin (1996)Google Scholar
  8. 8.
    Itoh, T., Abe, K.: Hamiltonian-conserving discrete canonical equations based on variational difference quotients. J. Comput. Phys. 76, 85–102 (1988). MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Matsuo, T., Furihata, D.: A stabilization of multistep linearly implicit schemes for dissipative systems. J. Comput. Appl. Math. 264, 38–48 (2014). MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    McLachlan, R.I., Quispel, G.R.W., Robidoux, N.: Unified approach to Hamiltonian systems, Poisson systems, gradient systems, and systems with Lyapunov functions or first integrals. Phys. Rev. Lett. 81, 2399–2403 (1998). MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    McLachlan, R.I., Quispel, G.R.W., Robidoux, N.: Geometric integration using discrete gradients. R. Soc. Lond. Philos. Trans. Ser. A Math. Phys. Eng. Sci. 357, 1021–1045 (1999). MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Meng, G.Y.: A practical asymptotical optimal SOR method. Appl. Math. Comput. 242, 707–715 (2014). MathSciNetzbMATHGoogle Scholar
  13. 13.
    Miyatake, Y., Sogabe, T., Zhang, S.: On the equivalence between SOR-type methods for linear systems and the discrete gradient methods for gradient systems. J. Comput. Appl. Math. 342, 58–69 (2018). MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Quispel, G.R.W., McLaren, D.I.: A new class of energy-preserving numerical integration methods. J. Phys. A 41(045), 206 (2008). MathSciNetzbMATHGoogle Scholar
  15. 15.
    Quispel, G.R.W., Turner, G.S.: Discrete gradient methods for solving ODEs numerically while preserving a first integral. J. Phys. A 29, L341–L349 (1996). MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Ren, L., Ren, F., Wen, R.: A selected method for the optimal parameters of the AOR iteration. J. Inequal. Appl. 2016, 279 (2016). MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Ringholm, T., Lazić, J., Schönlieb, C.B.: Variational image regularization with Euler’s elastica using a discrete gradient scheme. SIAM J. Imaging Sci. 11, 2665–2691 (2018). MathSciNetCrossRefGoogle Scholar
  18. 18.
    Varga, R.S.: Matrix Iterative Analysis, 2nd edn. Springer, Berlin (2000)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Cybermedia CenterOsaka University, ToyonakaOsakaJapan
  2. 2.Department of Applied Physics, Graduate School of EngineeringNagoya UniversityNagoyaJapan

Personalised recommendations