Journal of Scientific Computing

, Volume 74, Issue 2, pp 1146–1162

# On a New Updating Rule of the Levenberg–Marquardt Parameter

Article

## Abstract

A new Levenberg–Marquardt (LM) algorithm is proposed for nonlinear equations, where the iterate is updated according to the ratio of the actual reduction to the predicted reduction as usual, but the update of the LM parameter is no longer just based on that ratio. When the iteration is unsuccessful, the LM parameter is increased; but when the iteration is successful, it is updated based on the value of the gradient norm of the merit function. The algorithm converges globally under certain conditions. It also converges quadratically under the local error bound condition, which does not require the nonsingularity of the Jacobian at the solution.

## Keywords

Levenberg–Marquardt method Trust region method Nonlinear equations Local error bound Quadratic convergence

## References

1. 1.
Fan, J.Y.: A modified Levenberg–Marquardt algorithm for singular system of nonlinear equations. J. Comput. Math. 21, 625–636 (2003)
2. 2.
Fan, J.Y.: The modified Levenberg–Marquardt method for nonlinear equations with cubic convergence. Math. Comput. 81, 447–466 (2012)
3. 3.
Fan, J.Y., Pan, J.Y., Song, H.Y.: A retrospective trust region algorithm with trust region converging to zero. J. Comput. Math. 34, 421–436 (2016)Google Scholar
4. 4.
Fan, J.Y., Yuan, Y.X.: On the quadratic convergence of the Levenberg–Marquardt method without nonsingularity assumption. Computing 74, 23–39 (2005)
5. 5.
Levenberg, K.: A method for the solution of certain nonlinear problems in least squares. Quardt. Appl. Math. 2, 164–166 (1944)
6. 6.
Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear inequalities. SIAM J. Appl. Math. 11, 431–441 (1963)
7. 7.
Moré, J.J.: The Levenberg–Marquardt algorithm: implementation and theory. Numer. Anal. 630, 105–116 (1978)
8. 8.
Moré, J.J., Garbow, B.S., Hillstrom, K.H.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981)
9. 9.
Osborne, M.R.: Nonlinear least squares-the Levenberg–Marquardt algorithm revisited. J. Aust. Math. Soc. 19, 343–357 (1976)
10. 10.
Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming 2, pp. 1–27. Academic Press, New York (1975)Google Scholar
11. 11.
Schnabel, R.B., Frank, P.D.: Tensor methods for nonlinear equations. SIAM J. Numer. Anal. 21, 815–843 (1984)
12. 12.
Stewart, G.W., Sun, J.G.: Matrix Perturbation Theory. Academic Press, San Diego (1990)
13. 13.
Wright, S.J., Holt, J.N., Holt, J.N.: An inexact Levenberg–Marquardt method for large sparse nonlinear least squares. J. Aust. Math. Soc. 26, 387–403 (1985)
14. 14.
Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg–Marquardt mehod. Computing (Supplement) 15, 237–249 (2001)
15. 15.
Yuan, Y.X.: Trust region algorithms for nonlinear equations. Information 1, 7–20 (1998)
16. 16.
Yuan, Y.X.: Recent advances in numerical methods for nonlinear equations and nonlinear least sqaures. Numer. Algebra Control Optim. 1, 15–34 (2011)
17. 17.
Yuan, Y.X.: Recent advances in trust region algorithms. Math. Program. Ser. B 151, 249–281 (2015)
18. 18.
Zhao, R.X., Fan, J.Y.: Global complexity bound of the Levenberg–Marquardt method. Optim. Methods Softw. 31, 805–814 (2016)