Abstract
In this chapter, we study Black-Box second-order methods. In the first two sections, these methods are based on cubic regularization of the second-order model of the objective function. With an appropriate proximal coefficient, this model becomes a global upper approximation of the objective function. At the same time, the global minimum of this approximation is computable in polynomial time even if the Hessian of the objective is not positive semidefinite. We study global and local convergence of the Cubic Newton Method in convex and non-convex cases. In the next section, we derive the lower complexity bounds and show that this method can be accelerated using the estimating sequences technique. In the last section, we consider a modification of the standard Gauss–Newton method for solving systems of nonlinear equations. This modification is also based on an overestimating principle as applied to the norm of the residual of the system. Both global and local convergence results are justified.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
It could be a good exercise for the reader to prove that there are no uniformly convex functions with degree p ∈ (0, 2).
- 2.
This is the main difference with the technique presented in Sect. 2.2.1: we update the estimating function by a linearization computed at the new point x k+1.
- 3.
Since we do not assume that the norm ∥x∥, \(x \in \mathbb {E}_1\), is strongly convex, this problem may have a non-trivial convex set of global solutions.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Nesterov, Y. (2018). Second-Order Methods. In: Lectures on Convex Optimization. Springer Optimization and Its Applications, vol 137. Springer, Cham. https://doi.org/10.1007/978-3-319-91578-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-91578-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91577-7
Online ISBN: 978-3-319-91578-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)