Local convergence of the Levenberg–Marquardt method under Hölder metric subregularity

  • Masoud Ahookhosh
  • Francisco J. Aragón ArtachoEmail author
  • Ronan M. T. FlemingEmail author
  • Phan T. Vuong


We describe and analyse Levenberg–Marquardt methods for solving systems of nonlinear equations. More specifically, we propose an adaptive formula for the Levenberg–Marquardt parameter and analyse the local convergence of the method under Hölder metric subregularity of the function defining the equation and Hölder continuity of its gradient mapping. Further, we analyse the local convergence of the method under the additional assumption that the Łojasiewicz gradient inequality holds. We finally report encouraging numerical results confirming the theoretical findings for the problem of computing moiety conserved steady states in biochemical reaction networks. This problem can be cast as finding a solution of a system of nonlinear equations, where the associated mapping satisfies the Łojasiewicz gradient inequality assumption.


Nonlinear equation Levenberg–Marquardt method Local convergence rate Hölder metric subregularity Łojasiewicz inequality 

Mathematics Subject Classification (2010)

65K05 65K10 90C26 92C42 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



We would like to thank Mikhail Solodov for suggesting the use of Levenberg–Marquardt methods for solving the system of nonlinear equations arising in biochemical reaction networks. Thanks also go to Michael Saunders for his useful comments on the first version of this manuscript. We are grateful to two anonymous reviewers for their constructive comments, which helped us improving the paper.


F.J. Aragón was supported by MINECO of Spain and ERDF of EU, as part of the Ramón y Cajal program (RYC-2013-13327) and the I+D grant MTM2014-59179-C2-1-P. M. Ahookhosh, R.M.T. Fleming, and P.T. Vuong were supported by the U.S. Department of Energy, Offices of Advanced Scientific Computing Research and the Biological and Environmental Research as part of the Scientific Discovery Through Advanced Computing program, grant no. DE-SC0010429. P.T. Vuong was also supported by the Austrian Science Fund (FWF), grant M2499-N32.


  1. 1.
    Ahookhosh, M., Fleming, R.M.T., Vuong, P.T.: Finding zeros of Hölder metrically subregular mappings via globally convergent Levenberg–Marquardt methods, arXiv:1812.00818
  2. 2.
    Aragón Artacho, F.J., Fleming, R.: Globally convergent algorithms for finding zeros of duplomonotone mappings. Optim. Lett. 9(3), 569–584 (2015)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Aragón Artacho, F.J., Fleming, R., Vuong, P.T.: Accelerating the DC algorithm for smooth functions. Math. Program. 169B(1), 95–118 (2018)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. 116(1-2), 5–16 (2009)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program. 137A(1-2), 91–129 (2013)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Behling, R., Iusem, A.: The effect of calmness on the solution set of systems of nonlinear equations. Math. Program. 137A(1-2), 155–165 (2013)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Bellavia, S., Cartis, C., Gould, N., Morini, B., Toint, P.L.: Convergence of a regularized Euclidean residual algorithm for nonlinear least squares. SIAM J. Numer. Anal. 48(1), 1–29 (2010)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Bellavia, S., Morini, B.: Strong local convergence properties of adaptive regularized methods for nonlinear least squares. IMA J. Numer. Anal. 35(2), 947–968 (2015)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Bolte, J., Daniilidis, A., Lewis, A.: The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optimiz. 17(4), 1205–1223 (2007)zbMATHGoogle Scholar
  10. 10.
    Bolte, J., Daniilidis, A., Ley, O., Mazet, L.: Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity. Trans. Amer. Math. Soc. 362(6), 3319–3363 (2010)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Cibulka, R., Dontchev, A.L., Kruger, A.Y.: Strong metric subregularity of mappings in variational analysis and optimization. J. Math. Anal. Appl. 457(2), 1247–1282 (2018)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91B(2), 201–213 (2002)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Dontchev, A.L., Rockafellar, R.T.: Implicit Functions and Solution Mappings, 2 edn. Springer Series in Operations Research and Financial Engineering. Springer, New York (2014)Google Scholar
  14. 14.
    Eilenberger, G.: Solitons: Mathematical Methods for Physicists. Springer, Berlin (1983)zbMATHGoogle Scholar
  15. 15.
    Fan, J.: Convergence rate of the trust region method for nonlinear equations under local error bound condition. Comput. Optim. Appl. 34(2), 215–227 (2006)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Fan, J.: The modified Levenberg–Marquardt method for nonlinear equations with cubic convergence. Math. Comput. 81(277), 447–466 (2012)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Fan, J., Pan, J.: A note on the Levenberg–Marquardt parameter. Appl. Math. Comput. 207, 351–359 (2009)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Fan, J., Yuan, Y.: On the quadratic convergence of the Levenberg–Marquardt method without nonsingularity assumption. Computing 74(1), 23–39 (2005)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Fischer, A.: Local behavior of an iterative framework for generalized equations with nonisolated solutions. Math. Program. 94B(1), 91–124 (2002)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Fischer, A., Herrich, M., Izmailov, A.F., Solodov, M.V.: A globally convergent LP–newton method. SIAM J. Optim. 26(4), 2012–2033 (2015)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Fleming, R., Thiele, I.: Mass conserved elementary kinetics is sufficient for the existence of a non-equilibrium steady state concentration. J. Theoret. Biol. 314, 173–181 (2012)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Fleming, R.M., Vlassis, N., Thiele, I., Saunders, M.A.: Conditions for duality between fluxes and concentrations in biochemical networks. J. Theoret. Biol. 409, 1–10 (2016)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Gevorgyan, A., Poolman, M., Fell, D.: Detection of stoichiometric inconsistencies in biomolecular models. Bioinformatics 24(19), 2245–2251 (2008)Google Scholar
  24. 24.
    Guo, L., Lin, G.H., Ye, J.J.: Solving mathematical programs with equilibrium constraints. J. Optim. Theory Appl. 166(1), 234–256 (2015)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Gwoździewicz, J.: The Łojasiewicz exponent of an analytic function at an isolated zero. Comment. Math. Helv. 74(3), 364–375 (1999)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Haraldsdóttir, H.S., Fleming, R.M.: Identification of conserved moieties in metabolic networks by graph theoretical analysis of atom transition networks. PLos Comput. Biol. 12(11), e1004,999 (2016)Google Scholar
  27. 27.
    Hasegawa, A.: Plasma Instabilities and Nonlinear Effects. Springer, Berlin (1975)Google Scholar
  28. 28.
    Heirendt, L., et al.: Creation and analysis of biochemical constraint-based models: the COBRA Toolbox v3.0. To appear in Nat. Protoc.,
  29. 29.
    Hoffman, A.: On approximate solutions of systems of linear inequalities. J. Res. Nat. Bur. Standards 49, 263–265 (1952)MathSciNetGoogle Scholar
  30. 30.
    Izmailov, A.F., Solodov, M.V.: Error bounds for 2-regular mappings with Lipschitzian derivatives and their applications. Math. Program. 89B(3), 413–435 (2001)MathSciNetzbMATHGoogle Scholar
  31. 31.
    Izmailov, A.F., Solodov, M.V.: The theory of 2-regularity for mappings with Lipschitzian derivatives and its applications to optimality conditions. Math. Oper. Res. 27(3), 614–635 (2002)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Izmailov, A.F., Solodov, M.V.: Newton-Type Methods for Optimization and Variational Problems. Springer, Berlin (2014)zbMATHGoogle Scholar
  33. 33.
    Kanzow, C., Yamashita, N., Fukushima, M.: Levenberg–Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints. J. Comput. Appl. Math. 172(2), 375–397 (2004)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Karas, E.W., Santos, S.A., Svaiter, B.F.: Algebraic rules for computing the regularization parameter of the Levenberg–Marquardt method. Comput. Optim. Appl. 65(3), 723–751 (2016)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Kelley, C.: Iterative Methods for Optimization. Frontiers Appl Math, vol. 18. SIAM, Philadelphia (1999)Google Scholar
  36. 36.
    Klamt, S., Haus, U.U., Theis, F.: Hypergraphs and cellular networks. PLoS Comput. Biol. 5(5), e1000,385 (2009)MathSciNetGoogle Scholar
  37. 37.
    Kruger, A.: Error bounds and Hölder metric subregularity. Set-valued Var. Anal. 23(4), 705–736 (2015)MathSciNetzbMATHGoogle Scholar
  38. 38.
    Kurdyka, K., Spodzieja, S.: Separation of real algebraic sets and the Łojasiewicz exponent. Proc. Amer. Math. Soc. 142(9), 3089–3102 (2014)MathSciNetzbMATHGoogle Scholar
  39. 39.
    Li, G., Mordukhovich, B.: Hölder metric subregularity with applications to proximal point method. SIAM J. Optim. 22(4), 1655–1684 (2012)MathSciNetzbMATHGoogle Scholar
  40. 40.
    Lojasiewicz, S.: Ensembles semi-analytiques université de Gracovie (1965)Google Scholar
  41. 41.
    Ma, C., Jiang, L.: Some research on Levenberg–Marquardt method for the nonlinear equations. Appl. Math. Comput. 184, 1032–1040 (2007)MathSciNetzbMATHGoogle Scholar
  42. 42.
    Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I. Springer, Berlin (2006)Google Scholar
  43. 43.
    Mordukhovich, B.S., Ouyang, W.: Higher-order metric subregularity and its applications. J. Global Optim. 63(4), 777–795 (2015)MathSciNetzbMATHGoogle Scholar
  44. 44.
    Moré, J., Garbow, B., Hillstrom, K.: Testing unconstrained optimization software. ACM Trans. Math. Software 7(1), 17–41 (1981)MathSciNetzbMATHGoogle Scholar
  45. 45.
    Ngai, H.V.: Global error bounds for systems of convex polynomials over polyhedral constraints. SIAM J. on Optim. 25(1), 521–539 (2015)MathSciNetzbMATHGoogle Scholar
  46. 46.
    Nocedal, J., Wright, S.: Numerical Optimization. Springer, New York (2006)zbMATHGoogle Scholar
  47. 47.
    Ortega, J., Rheinboldt, W.: Iterative solution of nonlinear equations in several variables. Society for industrial and applied mathematics (2000)Google Scholar
  48. 48.
    Pang, J.: Error bounds in mathematical programming. Math. Program. 79B (1–3), 299–332 (1997)MathSciNetzbMATHGoogle Scholar
  49. 49.
    Parks, H., Krantz, S.: A Primer of Real Analytic Functions. Birkhäuser, Cambridge (1992)zbMATHGoogle Scholar
  50. 50.
    Vui, H.: Global Holderian̈ error bound for nondegenerate polynomials. SIAM J. Optim. 23(2), 917–933 (2013)MathSciNetzbMATHGoogle Scholar
  51. 51.
    Whitham, G.B.: Linear and Nonlinear Waves. Wiley, New York (1974)zbMATHGoogle Scholar
  52. 52.
    Yamashita, N., Fukushima, M. Alefeld, G., Chen, X. (eds.): On the rate of convergence of the Levenberg–Marquardt method, vol. 15. Springer, Vienna (2001)Google Scholar
  53. 53.
    Yuan, Y.: Recent advances in trust region algorithms. Math. Program. 151B (1), 249–281 (2015)MathSciNetzbMATHGoogle Scholar
  54. 54.
    Zhu, X., Lin, G.H.: Improved convergence results for a modified Levenberg–Marquardt method for nonlinear equations and applications in MPCC. Optim. Methods Softw. 31(4), 791–804 (2016)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Systems Biochemistry Group, Luxembourg Center for Systems BiomedicineUniversity of LuxembourgEsch-sur-AlzetteLuxembourg
  2. 2.Department of Electrical Engineering (ESAT-STADIUS) - KU LeuvenLeuvenBelgium
  3. 3.Department of MathematicsUniversity of AlicanteAlicanteSpain
  4. 4.Faculty of MathematicsUniversity of ViennaViennaAustria

Personalised recommendations