Mathematical Programming

, Volume 108, Issue 1, pp 177–205 | Cite as

Cubic regularization of Newton method and its global performance

Article

Abstract

In this paper, we provide theoretical analysis for a cubic regularization of Newton method as applied to unconstrained minimization problem. For this scheme, we prove general local convergence results. However, the main contribution of the paper is related to global worst-case complexity bounds for different problem classes including some nonconvex cases. It is shown that the search direction can be computed by standard linear algebra technique.

Keywords

General nonlinear optimization Unconstrained optimization Newton method Trust-region methods Global complexity bounds Global rate of convergence 

Mathematics Subject Classification (1991)

49M15 49M37 58C15 90C25 90C30 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bennet, A.A.: Newton's method in general analysis. Proc. Nat. Ac. Sci. USA. 2 (10), 592–598 (1916)Google Scholar
  2. 2.
    Conn, A.B., Gould, N.I.M., Toint, Ph.L.: Trust Region Methods. SIAM, Philadelphia, 2000Google Scholar
  3. 3.
    Dennis, J.E., Jr., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. SIAM, Philadelphia, 1996Google Scholar
  4. 4.
    Fletcher, R.: Practical Methods of Optimization, Vol. 1, Unconstrained Minimization. John Wiley, NY, 1980Google Scholar
  5. 5.
    Goldfeld, S., Quandt, R., Trotter, H.: Maximization by quadratic hill climbing. Econometrica. 34, 541–551 (1966)MATHMathSciNetGoogle Scholar
  6. 6.
    Kantorovich, L.V.: Functional analysis and applied mathematics. Uspehi Matem. Nauk. 3 (1), 89–185 (1948), (in Russian). Translated as N.B.S. Report 1509, Washington D.C. (1952)MathSciNetGoogle Scholar
  7. 7.
    Levenberg, K.: A method for the solution of certain problems in least squares. Quart. Appl. Math. 2, 164–168 (1944)MATHMathSciNetGoogle Scholar
  8. 8.
    Marquardt, D.: An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11, 431–441 (1963)CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    Nemirovsky, A., Yudin, D.: Informational complexity and efficient methods for solution of convex extremal problems. Wiley, New York, 1983Google Scholar
  10. 10.
    Nesterov, Yu.: Introductory lectures on convex programming: a basic course. Kluwer, Boston, 2004Google Scholar
  11. 11.
    Nesterov, Yu., Nemirovskii, A.: Interior-Point Polynomial Algorithms in Convex Programming. SIAM, Philadelphia, 1994Google Scholar
  12. 12.
    Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, NY, 1970Google Scholar
  13. 13.
    Polyak, B.T.: Gradient methods for minimization of functionals. USSR Comp. Math. Math. Phys. 3 (3), 643–653 (1963)Google Scholar
  14. 14.
    Polyak, B.T.: Convexity of quadratic transformations and its use in control and optimization. J. Optim. Theory and Appl. 99 (3), 553–583 (1998)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  1. 1.Center for Operations Research and Econometrics (CORE)Catholic University of Louvain (UCL)Louvain-la-NeuveBelgium
  2. 2.Institute of Control ScienceMoscowRussia

Personalised recommendations