Optimality Conditions: Reformulations

Abstract

• Why. In applications, many different forms of the conditions for optimality for convex optimization are used: duality theory, the Karush–Kuhn–Tucker (KKT) conditions, the minimax and saddle point theorem, Fenchel duality. Therefore, it is important to know what they are and how they are related.

• What.

– Duality theory. To a convex optimization problem, one can often associate a concave optimization problem with a completely different variable vector of optimization, but with the same optimal value. This is called the dual problem.

– KKT. This is the most popular form for the optimality conditions. We also present a reformulation of KKT in terms of subdifferentials, which is easier to work with.

– Minimax and saddle point. When two parties optimize against each other, then this sometimes leads to an equilibrium, where both have no incentive to change the current situation. This equilibrium can be described, in formal language, by a saddle point, that is, by vectors $\widehat x\in {\mathbb {R}}^n$ and $\widehat y\in {\mathbb {R}}^m$ for which, for a suitable function F(x, y), one has that $F(\widehat x,\widehat y)$ equals both min_xmax_yF(x, y) and max_ymin_xF(x, y).

– Fenchel duality. We do not describe this result in this abstract.

Road Map

• Figure 8.1 and Definition 8.1.2 (the dual problem and its geometric intuition).

• Proposition 8.1.3 (explicit description of the dual problem).

• Figure 8.2, Theorem 8.1.5 (duality theory in one picture).

• Definitions 8.2.1–8.2.4, Theorem 8.2.5, numerical example (convex programming problem, Lagrange function, Slater condition, KKT theorem).

• The idea of Definition 8.2.8 (KKT in the context of a perturbation of a problem).

• Theorem 8.3.1, numerical example (KKT in subdifferential form).

• Section 8.4 (minimax, maximin, saddle point, minimax theorem of von Neumann).

• Section 8.5 (Fenchel duality).