Convex Problems: The Main Questions

Abstract

• Why. Convex optimization includes linear programming, quadratic programming, semidefinite programming, least squares problems and shortest distance problems. It is necessary to have theoretical tools to solve these problems. Finding optimal solutions exactly or by means of a law that characterizes them, is possible for a small minority of problems, but this minority contains very interesting problems. Therefore, most problems have to be solved numerically, by an algorithm. An analysis of the performance of algorithms for convex optimization requires techniques that are different from those presented in this book; therefore such an analysis falls outside the scope of this book.

• What. We give the answers to the main questions of convex optimization (except the last one), and in this chapter these answers are compared to those for smooth optimization:

1. What are the conditions for optimality? A convex function f assumes its infimum at a point $\widehat x$ if and only if $0\in \partial f(\widehat x)$—Fermat’s theorem in the convex case. Now assume, moreover, that a convex perturbation has been chosen: a convex function of two vector variables F(x, y) for which f(x) = F(x, 0) ∀x; here $y\in Y={\mathbb {R}}^m$ represents changes in some data of the problem to minimize f (such as prices or budgets). Then f assumes its infimum at $\widehat x$ if and only if $f(\widehat x)=\mathcal {L}(\widehat x, \eta ,\eta _0)$ and $0\in \partial \mathcal {L}(\widehat x,\eta ,\eta _0)$ for some selection of multipliers (η, η ₀) (provided the bad case η ₀ = 0 is excluded)—Lagrange’s multiplier method in the convex case. Here $\mathcal {L}$ is the Lagrange function of the perturbation function F, which will be defined in this chapter. The conditions of optimality play the central role in the complete analysis of optimization problems. The complete analysis can always be done in one and the same systematic four step method. This is illustrated for some examples of optimization problems.

2. How to establish in advance the existence of optimal solutions? A suitable compactness assumption implies that there exists at least one optimal solution.

3. How to establish in advance the uniqueness of the optimal solution? A suitable strict convexity assumption implies that there is at most one solution.

4. What is the sensitivity of the optimal valueS(y) for small changesyin the data of the problem? The multiplier η is an element of the subdifferential ∂S(0) provided η ₀ = 1, so then it is a measure for the sensitivity.

5. Can one find in a guaranteed efficient way approximations for the optimal solution, by means of an algorithm? This is possible for almost all convex problems.

Road Map

1. Section 7.1 (convex optimization problem, types: LP, QP, SDP, least squares, shortest distance).

2. Theorems 7.2.1–7.2.3, Definition 7.2.5, Theorems 7.2.6, 7.2.7, Example 7.2.3 (existence and uniqueness for convex problems).

3. Section 7.3 (definitions leading up to the concept of smooth local minimum).

4. Theorem 7.4.1, Example 7.4.4 (Fermat’s theorem (smooth case)).

5. Figure 7.2, Proposition 7.5.1 (no need for local minima in convex optimization).

6. Figure 7.3, Theorem 7.6.1, Example 7.6.5 (Fermat’s theorem (convex case)).

7. Section 7.7 (perturbation of a problem).

8. Theorem 7.8.1, Example 7.8.3 (Lagrange multipliers (smooth case)).

9. Figure 7.7, Proposition 7.9.1, Definitions 7.9.2–7.9.4, Theorem 7.9.5, Definition 8.2.2 (Lagrange multipliers (convex case).

10. Figure 7.8 (generalized optimal solutions always exist).

11. Section 7.11 (list advantages convex optimization).