Convergence of an Inexact Majorization-Minimization Method for Solving a Class of Composite Optimization Problems

Abstract

We suggest a majorization-minimization method for solving nonconvex minimization problems. The method is based on minimizing at each iterate a properly constructed consistent majorizer of the objective function. We describe a variety of classes of functions for which such a construction is possible. We introduce an inexact variant of the method, in which only approximate minimization of the consistent majorizer is performed at each iteration. Both the exact and the inexact algorithms are shown to be descent methods whose accumulation points have a property which is stronger than standard stationarity. We give examples of cases in which the exact method can be applied. Finally, we show that the inexact method can be applied to a specific problem, called sparse source localization, by utilizing a fast optimization method on a smooth convex dual of its subproblems.

Appendix: A Proof of Lemma 2

We provide a proof of Lemma 2. The necessity is proven very similarly to the proof given in [4, Theorem 9.2] for the case where F is continuously differentiable.

Proof

Let x ^∗ be a local minimizer of problem (13.2). Assume to the contrary that x ^∗ is not a stationary point of (13.2). Then, recalling that F is directionally differentiable (dd), there exists y ∈dom(F) such that F ^′(x ^∗;y −x ^∗) < 0. By the definition of a directional derivative, it follows that there exists a number 0 < δ < 1 such that F(x ^∗ + t(y −x ^∗)) < F(x ^∗) for all 0 < t < δ. Since dom(F) is convex (as F is dd), we have x ^∗ + t(y −x ^∗) = (1 − t)x ^∗ + t y ∈dom(F) for all 0 < t < δ, contradicting the local minimality of x ^∗.

As for the sufficiency part when F is convex, let x ^∗ be a stationary point of (13.2), and assume to the contrary that x ^∗ is not a global minimizer of (13.2). Then there exists y ∈ dom(F) such that F(y) < F(x ^∗). By the stationarity of x ^∗ and the convexity of F, we obtain

$$\displaystyle \begin{aligned} \begin{array}{rcl}0&\displaystyle \leq&\displaystyle F^{\prime}({\mathbf{x}}^*;\mathbf{y}-{\mathbf{x}}^*)=\lim_{t\rightarrow0^+}\frac{F({\mathbf{x}}^*+t(\mathbf{y}-{\mathbf{x}}^*))-F({\mathbf{x}}^*)}{t}\\ &\displaystyle =&\displaystyle \lim_{t\rightarrow0^+}\frac{F(t\mathbf{y}+(1-t){\mathbf{x}}^*)-F({\mathbf{x}}^*)}{t} \leq\lim_{t\rightarrow0^+}\frac{t F(\mathbf{y})+(1-t)F({\mathbf{x}}^*)-F({\mathbf{x}}^*)}{t}\\ &\displaystyle =&\displaystyle \lim_{t\rightarrow0^+}\frac{t (F(\mathbf{y})-F({\mathbf{x}}^*))}{t}=F(\mathbf{y})-F({\mathbf{x}}^*)<0,\end{array} \end{aligned} $$

which is a contradiction. □