Advertisement

Journal of Scientific Computing

, Volume 74, Issue 2, pp 1146–1162 | Cite as

On a New Updating Rule of the Levenberg–Marquardt Parameter

Article
  • 165 Downloads

Abstract

A new Levenberg–Marquardt (LM) algorithm is proposed for nonlinear equations, where the iterate is updated according to the ratio of the actual reduction to the predicted reduction as usual, but the update of the LM parameter is no longer just based on that ratio. When the iteration is unsuccessful, the LM parameter is increased; but when the iteration is successful, it is updated based on the value of the gradient norm of the merit function. The algorithm converges globally under certain conditions. It also converges quadratically under the local error bound condition, which does not require the nonsingularity of the Jacobian at the solution.

Keywords

Levenberg–Marquardt method Trust region method Nonlinear equations Local error bound Quadratic convergence 

References

  1. 1.
    Fan, J.Y.: A modified Levenberg–Marquardt algorithm for singular system of nonlinear equations. J. Comput. Math. 21, 625–636 (2003)MathSciNetMATHGoogle Scholar
  2. 2.
    Fan, J.Y.: The modified Levenberg–Marquardt method for nonlinear equations with cubic convergence. Math. Comput. 81, 447–466 (2012)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Fan, J.Y., Pan, J.Y., Song, H.Y.: A retrospective trust region algorithm with trust region converging to zero. J. Comput. Math. 34, 421–436 (2016)Google Scholar
  4. 4.
    Fan, J.Y., Yuan, Y.X.: On the quadratic convergence of the Levenberg–Marquardt method without nonsingularity assumption. Computing 74, 23–39 (2005)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Levenberg, K.: A method for the solution of certain nonlinear problems in least squares. Quardt. Appl. Math. 2, 164–166 (1944)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear inequalities. SIAM J. Appl. Math. 11, 431–441 (1963)CrossRefMATHGoogle Scholar
  7. 7.
    Moré, J.J.: The Levenberg–Marquardt algorithm: implementation and theory. Numer. Anal. 630, 105–116 (1978)MathSciNetMATHGoogle Scholar
  8. 8.
    Moré, J.J., Garbow, B.S., Hillstrom, K.H.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Osborne, M.R.: Nonlinear least squares-the Levenberg–Marquardt algorithm revisited. J. Aust. Math. Soc. 19, 343–357 (1976)CrossRefMATHGoogle Scholar
  10. 10.
    Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming 2, pp. 1–27. Academic Press, New York (1975)Google Scholar
  11. 11.
    Schnabel, R.B., Frank, P.D.: Tensor methods for nonlinear equations. SIAM J. Numer. Anal. 21, 815–843 (1984)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Stewart, G.W., Sun, J.G.: Matrix Perturbation Theory. Academic Press, San Diego (1990)MATHGoogle Scholar
  13. 13.
    Wright, S.J., Holt, J.N., Holt, J.N.: An inexact Levenberg–Marquardt method for large sparse nonlinear least squares. J. Aust. Math. Soc. 26, 387–403 (1985)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg–Marquardt mehod. Computing (Supplement) 15, 237–249 (2001)MATHGoogle Scholar
  15. 15.
    Yuan, Y.X.: Trust region algorithms for nonlinear equations. Information 1, 7–20 (1998)MathSciNetMATHGoogle Scholar
  16. 16.
    Yuan, Y.X.: Recent advances in numerical methods for nonlinear equations and nonlinear least sqaures. Numer. Algebra Control Optim. 1, 15–34 (2011)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Yuan, Y.X.: Recent advances in trust region algorithms. Math. Program. Ser. B 151, 249–281 (2015)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Zhao, R.X., Fan, J.Y.: Global complexity bound of the Levenberg–Marquardt method. Optim. Methods Softw. 31, 805–814 (2016)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.School of Mathematical Sciences, and MOE-LSCShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations