Computational Optimization and Applications

, Volume 59, Issue 1–2, pp 321–351 | Cite as

A regularized Newton method without line search for unconstrained optimization

  • Kenji Ueda
  • Nobuo Yamashita


In this paper, we propose a regularized Newton method without line search. The proposed method controls a regularization parameter instead of a step size in order to guarantee the global convergence. We show that the proposed algorithm has the following convergence properties. (a) The proposed algorithm has global convergence under appropriate conditions. (b) It has superlinear rate of convergence under the local error bound condition. (c) An upper bound of the number of iterations required to obtain an approximate solution \(x\) satisfying \(\Vert \nabla f(x) \Vert \le \varepsilon \) is \(O(\varepsilon ^{-2})\), where \(f\) is the objective function and \(\varepsilon \) is a given positive constant.


Regularized Newton method Global complexity bound   Global convergence Superlinear convergence 


  1. 1.
    Ashcraft, C., Grimes, R.G., Lewis, J.G.: Accurate symmetric indefinite linear equation solvers. SIAM J. Matrix Anal. Appl. 20, 513–561 (1998)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Cartis, C., Gould, N.I.M., Toint, P.L.: On the complexity of steepest descent, Newton’s and regularized Newton’s methods for nonconvex unconstrained optimization problems. SIAM J. Optim. 20, 2833–2852 (2010)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Cheng, S.H., Higham, N.J.: A modified Cholesky algorithm based on a symmetric indefinite factorization. SIAM J. Matrix Anal. Appl. 19, 1097–1110 (1998)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust-Region Methods. SIAM, Philadelphia (2000)CrossRefMATHGoogle Scholar
  5. 5.
    Crouzeix, J.P.: On second order conditions for quasiconvexity. Math. Program. 18, 349–352 (1980)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Gould, N.I.M., Orban, D., Toint, P.L.: CUTEr (and SifDec), a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373–394 (2003)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Gratton, S., Sartenaer, A., Toint, P.L.: Recursive trust-region methods for multiscale nonlinear optimization. SIAM J. Optim. 19, 414–444 (2008)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Li, D.H., Fukushima, M., Qi, L., Yamashita, N.: Regularized Newton methods for convex minimization problems with singular solutions. Comput. Optim. Appl. 28, 131–147 (2004)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Li, Y.J., Li, D.H.: Truncated regularized Newton method for convex minimizations. Comput. Optim. Appl. 43, 119–131 (2009)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Nesterov, Y.: Introductory Lectures on Convex Optimization. Kluwer Academic, Dordrecht (2004)CrossRefMATHGoogle Scholar
  12. 12.
    Nesterov, Y., Polyak, B.T.: Cubic regularization of Newton method and its global performance. Math. Program. 108, 177–205 (2006)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Pang, J.S.: Error bounds in mathematical programming. Math. Program. 79, 299–332 (1997)MathSciNetMATHGoogle Scholar
  14. 14.
    Polyak, R.A.: Regularized Newton method for unconstrained convex optimization. Math. Program. 120, 125–145 (2009)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Shen, C., Chen, X., Liang, Y.: A regularized Newton method for degenerate unconstrained optimization problems. Optim. Lett. 6, 1913–1933 (2012)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Ueda, K.: A regularized Newton method without line search for unconstrained optimization. Master’s thesis, Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University (2009).
  17. 17.
    Ueda, K., Yamashita, N.: Convergence properties of the regularized Newton method for the unconstrained nonconvex optimization. Appl. Math. Optim. 62, 27–46 (2010)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg–Marquardt method. Computing Supplementum(Wien) 15, 227–238 (2001)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Advanced Technology R&D CenterMitsubishi Electric CorporationHyogoJapan
  2. 2.Department of Applied Mathematics and Physics, Graduate School of InformaticsKyoto UniversityKyotoJapan

Personalised recommendations