An Approximate ADMM for Solving Linearly Constrained Nonsmooth Optimization Problems with Two Blocks of Variables

Abstract

Nonsmooth convex optimization problems with two blocks of variables subject to linear constraints are considered. A new version of the alternating direction method of multipliers is developed for solving these problems. In this method the subproblems are solved approximately. The convergence of the method is studied. New test problems are designed and used to verify the efficiency of the proposed method and to compare it with two versions of the proximal bundle method.

Appendix

This appendix contains all test problems used in numerical experiments. In these problems the objective functions are presented as:

$$\displaystyle \begin{aligned}F(x,y) = f_1(x) + f_2(y). \end{aligned}$$

Therefore, in the description of test problems only functions f ₁ and f ₂ are presented and the following notations are used:

• $(x^0,y^0) \in \mathbb {R}^n \times \mathbb {R}^m$—starting point;

• $(x^*,y^*) \in \mathbb {R}^n \times \mathbb {R}^m$—known best solution;

• f ^∗—known best value.

Problem 1

Dimension: 3 (2,1),

Component functions:

$f_1(x)= \max \limits _{i=1,2,3} f^i(x)$,

$f_2(y)= \max \limits _{i=4,5,6} f^i(y)$,

$f^1(x) = x_1^2+x_2^4, f^2(x)=(2-x_1)^2+(2-x_2)^2, f^3(x)=\exp (x_2-x_1),$

$f^4(y) = y_1^2, f^5(y)=(2-y_1)^2, f^6(y)=\exp (-y_1),$

Linear constraints: − x 1 − x 2 + y 1 = −0.5,

Starting point: (x 0, y 0) = (2, 2, 2)T,

Optimum point: (x ∗, y ∗) = (0.8333333, 0.8333333, 1.1666667)T,

Optimum value: f ∗ = 4.0833333.

Problem 2

Dimension: 4 (2,2),
Component functions:
$f_1(x) = \max \{f_1^1(x), f_1^2(x), f_1^3(x)\}$,
$f_2(y) = \max \{f_2^1(y), f_2^2(y), f_2^3(y)\}$,
$f_1^1(x) = x_1^4 + x_2^2,~f_1^2(x) = (2-x_1)^2 + (2-x_2)^2, f_1^3(x) = 2e^{-x_1+x_2},$
$f_2^1(y) = y_1^2 - 2y_1+y_2^2 - 4y_2 +4,~f_2^2(y) = 2y_1^2 - 5y_1+y_2^2 -2y_2 +4,$
$f_2^3(y) = y_1^2 + 2y_2^2-4y_2+1,$
Linear constraints:
x 1 + x 2 − y 1 + y 2 = 0,
− 3x 1 + 5x 2 − y 1 − 2y 2 = 0,
Starting point: (x 0, y 0) = (2, 2, 2, 2)T,
Optimum point: (x ∗, y ∗) = (0.4964617, 0.6982659, 1.4638, 0.2690723)T,
Optimum value: f ∗ = 6.1663599.

Problem 3

Dimension: 4 (2,2),
Component functions:
$f_1(x) = | x_1 - 1| + 200 \max \{0, | x_1| - x_2\}$,
f 2(y) = 100(|y 1|− y 2),
Linear constraints:
x 1 − x 2 + 2y 1 − 3y 2 = 1,
− x 1 + 2x 2 − 2y 1 − y 2 = 6,
Starting point: (x 0, y 0) = (−1.2, 1, 1, 1)T,
Optimum point: (x ∗, y ∗) = (5.6666667, 5.6666667, 0, −0.3333333)T,
Optimum value: f ∗ = 38.

Problem 4

Dimension: 10 (3,7),
Component functions:
f 1(x) = 9|x 1| + |x 2| + |x 3|,
$f_2(y) = \sum \limits _{i=1}^7 i|y_i|$,
Linear constraints:
x 1 + x 2 − y 1 = 1,
x 1 − y 2 = −10,
x 1 + y 3 = 10,
x 2 − y 4 = 1,
x 2 + y 5 = 10,
x 3 − y 6 = −10,
x 3 + y 7 = 1,
Starting point: x 0 = (1, 1, 1)T, y 0 = (2, …, 2)T,
Optimum point: x ∗ = (0, 1, 0.0000001)T,
y ∗ = (0, 10, 10, 0, 9, 10, 1)T,
Optimum value: f ∗ = 163.

Problem 5

Dimension: 10 (3,7),
Component functions:
f 1(x) = |x 1 + 3x 2 + x 3| + 4|x 1 − x 2|,
$f_2(y) = 10\sum \limits _{i=1}^4 \max \limits \{0,-y_i\}+\sum \limits _{i=1}^7 i|y_i|$,
Linear constraints:
− x 1 + 6x 2 + 4x 3 − y 1 = 3,
x 1 + x 2 + x 3 = 1,
x 1 + x 2 − y 5 = 0,
x 2 + x 3 − y 6 = 0,
x 1 + x 3 − y 7 = 0,
Starting point: x 0 = (1, −2, 3)T, y 0 = (2, …, 2)T,
Optimum point: x ∗ = (0.3333408, 0.3333408, 0.3333237)T,
y ∗ = (0, 0, 0, 0, 0.6666869, 0.6666638, 0.6666638)T,
Optimum value: f ∗ = 13.6673059.

Problem 6

Dimension: 10 (3,7),
Component functions:
f 1(x) = 9 + 8x 1 + 6x 2 + 4x 3 + 2|x 1| + 2|x 2| + |x 3|,
+ |x 1 + x 2| + |x 1 + x 3|,
$f_2(y) = 2|y_1|+|y_2|+|y_3| + 10 \sum \limits _{i=1}^4 \max \limits \{0,-y_{i+3}\}$,
Linear constraints:
x 1 + x 2 + 2x 3 + y 4 = 3,
4x 1 − 2x 2 − x 3 − 3y 1 + y 2 − y 3 = 6,
x 1 + x 2 − y 5 = 0,
x 2 + x 3 − y 6 = 0,
x 1 + x 3 − y 7 = 0,
Starting point: x 0 = (−1, 2, −3)T, y 0 = (1, …, 1)T,
Optimum point: x ∗ = (0, 0, 0)T,
y ∗ = (−2, 0, 0, 3, 0, 0, 0)T,
Optimum value: f ∗ = 13.

Problem 7

Dimension: 12 (7,5),
Component functions:
$f_1(x) = \sum \limits _{i=1}^6 |x_i-x_{i+1}|$,
$f_2(y) = 10\sum \limits _{j=1}^5 \max \limits \{0,y_j\}$,
Linear constraints:
x 1 + x 2 + 2x 3 − y 1 = 3,
x 2 − x 3 + x 4 − 3y 2 = 1,
x 1 + x 5 − y 1 − y 3 = 1,
x 1 + x 6 − y 1 − y 4 = 2,
x 1 + x 7 − y 1 − y 5 = 1,
Starting point: x 0 = (1, 2, 3, 4, −5, −6, −7)T, y 0 = (1, −2, 3, −4, 5)T,
Optimum point: x ∗ = (0.75, 0.75, 0.75, 0.3381465, 0.25, 0.25, 0.25)T,
y ∗ = (0, 0.2206178, 0, −1, 0)T,
Optimum value: f(x ∗, y ∗) = 0.5.

Problem 8

Dimension: 14 (8,6),
Component functions:
$f_1(x) = \sum \limits _{i=1}^8 \max \limits \{0,x_i\},$
f 2(y) = |y 1| + 2|y 2| + 2|y 3| + |y 4| + |y 5| + 5|y 6| + y 2 − y 5 − 4y 6,
Linear constraints:
x 1 + 2x 2 + x 3 + x 4 + y 1 = 5,
3x 1 + x 2 + 2x 3 − x 4 + y 2 = 4,
x 5 + 2x 6 + x 7 + x 8 + y 3 = 5,
3x 5 + x 6 + 2x 7 − x 8 + y 4 = 4,
x 2 + 4x 3 − y 5 = 1.5,
Starting point: x 0 = (1, …, 1)T, y 0 = (2, …, 2)T,
Optimum point:
x ∗ = (0.6, 2.2, 0, 0, 0.6, 2.2, 0, 0)T,
y ∗ = (0, 0, 0, 0, 0.7, 0)T,
Optimum value:f ∗ = 5.6.

Problem 9

Dimension: 17 (10,7),
Component functions:
f 1(x) = |x 1| + |x 2| + |x 1 + x 2|− x 1 − 2x 2 + |x 3 − 10| + 4|x 4 − 5|,
+ |x 5 − 3| + 2|x 6 − 1| + 5|x 7| + 7|x 8 − 11| + 2|x 9 − 10| + |x 10 − 7|,
$f_2(y) = |y_1|+|y_2|+10\sum \limits _{i=3}^6 \max \limits \{0,y_i-y_{i+1}\}$,
Linear constraints:
4x 1 + 5x 2 − 3x 7 + 9x 8 + 2y 1 + y 3 + y 4 = 105,
10x 1 − 8x 2 − 17x 7 + 2x 8 + y 2 + y 4 + 5y 5 = 0,
8x 1 − 2x 2 − 5x 9 + 2x 10 − 3y 5 − y 6 = −12,
3x 1 − 6x 2 − 12x 9 + 7x 10 − 2y 6 − y 7 = −96,
Starting point: x 0 = (2, …, 2)T, y 0 = (1, …, 1)T,
Optimum point:
x ∗ = (1.4431818, 1.6136364, 10, 5, 3, 1, 0, 11, 10, 7)T,
y ∗ = (0, 0, −3.9204544, −3.9204544, −3.9204544, −3.9204544, 27.4886356)T,
Optimum value: f ∗ = 1.4431818.

Problem 10

Dimension: 30 (15,15),
Component functions:
$f_1(x) = 15 \max \limits _{i=1,\ldots ,15} |x_i|,$
$f_2(y) = \sum \limits _{i=1}^{15} |y_i|$,
Linear constraints: x i + 2x i+1 − 3x i+2 − y i − 2y i+2 = i − 1, i = 1, …, 13,
Starting point:
x 0 = (x 1, …, x 15)T, x i = i, i ≤ 7, x i = −i, i > 7,
y 0 = (y 1, …, y 15)T, y i = −i, i ≤ 7, y i = i, i > 7,
Optimum point:
x ∗ = (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)T,
y ∗ = (0, 0, 0, −0.5, −1, −1.25, −1.5, −1.88, −2.25, −2.56, −2.86,
− 3.22, −3.56, −3.89, −4.22)T,
Optimum value: f ∗ = 28.703125.