Abstract
Nonsmooth convex optimization problems with two blocks of variables subject to linear constraints are considered. A new version of the alternating direction method of multipliers is developed for solving these problems. In this method the subproblems are solved approximately. The convergence of the method is studied. New test problems are designed and used to verify the efficiency of the proposed method and to compare it with two versions of the proximal bundle method.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
A. Bagirov, N. Karmitsa, and M. Makela. Introduction to Nonsmooth Optimization: Theory, Practice and Software. Springer, Cham, 2014.
A. Bnouhachem, M.H. Xu, M. Khalfaoui, and Sh. Zhaohana. A new alternating direction method for solving variational inequalities. Computers and Mathematics with Applications, 62 (2011), 626–634.
S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1) (2011), 1–122.
Sh. Cao, Y. Xiao, and H. Zhu. Linearized alternating directions method for l 1 -norm inequality constrained l 1 -norm minimization. Applied Numerical Mathematics, 85 (2014), 142–153.
F.H. Clarke. Optimization and Nonsmooth Analysis. Canadian Mathematical Society series of monographs and advanced texts. Wiley-Interscience, 1983.
E.D. Dolan and J.J. Moré. Benchmarking optimization software with performance profiles. Mathematical Programming, 91(2) (2002), 201–213.
J. Eckstein and W. Yao. Augmented Lagrangian and alternating direction methods for convex optimization: A tutorial and some illustrative computational results. In RUTCOR Research Reports, 32. 2012.
M. El Anbari, S. Alam, and H. Bensmail. COFADMM: A computational features selection with alternating direction method of multipliers. Procedia Computer Science, 29 (2014), 821–830.
M. Fukushima. Application of the alternating direction method of multipliers to separable convex programming problems. Computational Optimization and Applications, 1(1) (1992), 93–111.
D. Gabay and B. Mercier. A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Computers and Mathematics with Applications, 2 (1976) 17–40.
R. Glowinski and A. Marrocco. Sur l ‘ approximation par éléments finis d ‘ ordre 1 et la résolution par pénalisation-dualité d ‘ une classe de problémes de dirichlet. RAIRO, 2 (1975), 41–76.
D. Han and H.K. Lo. A new stepsize rule in He and Zhou’s alternating direction method. Applied Mathematics Letters, 15 (2002), 181–185.
B. He and J. Zhou. A modified alternating direction method for convex minimization problems. Applied Mathematics Letters, 13 (2000), 123–130.
M.R. Hestenes. Multiplier and gradient methods. Journal of Optimization Theory and Applications, 4 (1969) 302–320.
K.C. Kiwiel. An alternating linearization bundle method for convex optimization and nonlinear multicommodity flow problems. Mathematical Programming, Ser. A, 130 (2011), 59–84.
X. Li, L. Mo, X. Yuan, and J. Zhang. Linearized alternating direction method of multipliers for sparse group and fused lasso models. Computational Statistics and Data Analysis, 79 (2014), 203–221.
L. Luks̈an and J. Vlc̈ek. Algorithm 811: NDA: algorithms for nondifferentiable optimization. ACM Transactions on Mathematical Software, 27(2) (2001), 193–213.
M.J.D. Powell. A method for nonlinear constraints in minimization problems. In R. Fletcher, editor, Optimization, pages 283–298. Academic Press, NY, 1969.
A. Rakotomamonjy. Applying alternating direction method of multipliers for constrained dictionary learning. Neurocomputing, 106 (2013), 126–136.
Y. Shen and M.H. Xu. On the O(1∕t) convergence rate of Ye–Yuan’s modified alternating direction method of multipliers. Applied Mathematics and Computation, 226 (2014), 367–373.
J. Sun and S. Zhang. A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs. European Journal of Operational Research, 207 (2010), 1210–1220.
K. Zhao and G. Yao. Application of the alternating direction method for an inverse monic quadratic eigenvalue problem. Applied Mathematics and Computation, 244 (2014), 32–41.
Acknowledgements
The authors would like to thank the anonymous referee for valuable comments that helped to improve the quality of this paper.
This research was started when Dr. A.M. Bagirov visited Chongqing Normal University and the visit was supported by this university. The research by A.M. Bagirov and S.Taheri was also supported by Australian Research Council’s Discovery Projects funding scheme (Project No. DP190100580).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
This appendix contains all test problems used in numerical experiments. In these problems the objective functions are presented as:
Therefore, in the description of test problems only functions f 1 and f 2 are presented and the following notations are used:
-
\((x^0,y^0) \in \mathbb {R}^n \times \mathbb {R}^m\)—starting point;
-
\((x^*,y^*) \in \mathbb {R}^n \times \mathbb {R}^m\)—known best solution;
-
f ∗—known best value.
Problem 1
Dimension: 3 (2,1), |
Component functions: |
\(f_1(x)= \max \limits _{i=1,2,3} f^i(x)\), |
\(f_2(y)= \max \limits _{i=4,5,6} f^i(y)\), |
\(f^1(x) = x_1^2+x_2^4, f^2(x)=(2-x_1)^2+(2-x_2)^2, f^3(x)=\exp (x_2-x_1),\) |
\(f^4(y) = y_1^2, f^5(y)=(2-y_1)^2, f^6(y)=\exp (-y_1),\) |
Linear constraints: − x 1 − x 2 + y 1 = −0.5, |
Starting point: (x 0, y 0) = (2, 2, 2)T, |
Optimum point: (x ∗, y ∗) = (0.8333333, 0.8333333, 1.1666667)T, |
Optimum value: f ∗ = 4.0833333. |
Problem 2
Dimension: 4 (2,2), | |
Component functions: | |
\(f_1(x) = \max \{f_1^1(x), f_1^2(x), f_1^3(x)\}\), | |
\(f_2(y) = \max \{f_2^1(y), f_2^2(y), f_2^3(y)\}\), | |
\(f_1^1(x) = x_1^4 + x_2^2,~f_1^2(x) = (2-x_1)^2 + (2-x_2)^2, f_1^3(x) = 2e^{-x_1+x_2},\) | |
\(f_2^1(y) = y_1^2 - 2y_1+y_2^2 - 4y_2 +4,~f_2^2(y) = 2y_1^2 - 5y_1+y_2^2 -2y_2 +4,\) | |
\(f_2^3(y) = y_1^2 + 2y_2^2-4y_2+1,\) | |
Linear constraints: | |
x 1 + x 2 − y 1 + y 2 = 0, | |
− 3x 1 + 5x 2 − y 1 − 2y 2 = 0, | |
Starting point: (x 0, y 0) = (2, 2, 2, 2)T, | |
Optimum point: (x ∗, y ∗) = (0.4964617, 0.6982659, 1.4638, 0.2690723)T, | |
Optimum value: f ∗ = 6.1663599. |
Problem 3
Dimension: 4 (2,2), | ||
Component functions: | ||
\(f_1(x) = | x_1 - 1| + 200 \max \{0, | x_1| - x_2\}\), | ||
f 2(y) = 100(|y 1|− y 2), | ||
Linear constraints: | ||
x 1 − x 2 + 2y 1 − 3y 2 = 1, | ||
− x 1 + 2x 2 − 2y 1 − y 2 = 6, | ||
Starting point: (x 0, y 0) = (−1.2, 1, 1, 1)T, | ||
Optimum point: (x ∗, y ∗) = (5.6666667, 5.6666667, 0, −0.3333333)T, | ||
Optimum value: f ∗ = 38. |
Problem 4
Dimension: 10 (3,7), | |||
Component functions: | |||
f 1(x) = 9|x 1| + |x 2| + |x 3|, | |||
\(f_2(y) = \sum \limits _{i=1}^7 i|y_i|\), | |||
Linear constraints: | |||
x 1 + x 2 − y 1 = 1, | |||
x 1 − y 2 = −10, | |||
x 1 + y 3 = 10, | |||
x 2 − y 4 = 1, | |||
x 2 + y 5 = 10, | |||
x 3 − y 6 = −10, | |||
x 3 + y 7 = 1, | |||
Starting point: x 0 = (1, 1, 1)T, y 0 = (2, …, 2)T, | |||
Optimum point: x ∗ = (0, 1, 0.0000001)T, | |||
y ∗ = (0, 10, 10, 0, 9, 10, 1)T, | |||
Optimum value: f ∗ = 163. |
Problem 5
Dimension: 10 (3,7), | ||||
Component functions: | ||||
f 1(x) = |x 1 + 3x 2 + x 3| + 4|x 1 − x 2|, | ||||
\(f_2(y) = 10\sum \limits _{i=1}^4 \max \limits \{0,-y_i\}+\sum \limits _{i=1}^7 i|y_i|\), | ||||
Linear constraints: | ||||
− x 1 + 6x 2 + 4x 3 − y 1 = 3, | ||||
x 1 + x 2 + x 3 = 1, | ||||
x 1 + x 2 − y 5 = 0, | ||||
x 2 + x 3 − y 6 = 0, | ||||
x 1 + x 3 − y 7 = 0, | ||||
Starting point: x 0 = (1, −2, 3)T, y 0 = (2, …, 2)T, | ||||
Optimum point: x ∗ = (0.3333408, 0.3333408, 0.3333237)T, | ||||
y ∗ = (0, 0, 0, 0, 0.6666869, 0.6666638, 0.6666638)T, | ||||
Optimum value: f ∗ = 13.6673059. |
Problem 6
Dimension: 10 (3,7), | |||||
Component functions: | |||||
f 1(x) = 9 + 8x 1 + 6x 2 + 4x 3 + 2|x 1| + 2|x 2| + |x 3|, | |||||
+ |x 1 + x 2| + |x 1 + x 3|, | |||||
\(f_2(y) = 2|y_1|+|y_2|+|y_3| + 10 \sum \limits _{i=1}^4 \max \limits \{0,-y_{i+3}\}\), | |||||
Linear constraints: | |||||
x 1 + x 2 + 2x 3 + y 4 = 3, | |||||
4x 1 − 2x 2 − x 3 − 3y 1 + y 2 − y 3 = 6, | |||||
x 1 + x 2 − y 5 = 0, | |||||
x 2 + x 3 − y 6 = 0, | |||||
x 1 + x 3 − y 7 = 0, | |||||
Starting point: x 0 = (−1, 2, −3)T, y 0 = (1, …, 1)T, | |||||
Optimum point: x ∗ = (0, 0, 0)T, | |||||
y ∗ = (−2, 0, 0, 3, 0, 0, 0)T, | |||||
Optimum value: f ∗ = 13. |
Problem 7
Dimension: 12 (7,5), | ||||||
Component functions: | ||||||
\(f_1(x) = \sum \limits _{i=1}^6 |x_i-x_{i+1}|\), | ||||||
\(f_2(y) = 10\sum \limits _{j=1}^5 \max \limits \{0,y_j\}\), | ||||||
Linear constraints: | ||||||
x 1 + x 2 + 2x 3 − y 1 = 3, | ||||||
x 2 − x 3 + x 4 − 3y 2 = 1, | ||||||
x 1 + x 5 − y 1 − y 3 = 1, | ||||||
x 1 + x 6 − y 1 − y 4 = 2, | ||||||
x 1 + x 7 − y 1 − y 5 = 1, | ||||||
Starting point: x 0 = (1, 2, 3, 4, −5, −6, −7)T, y 0 = (1, −2, 3, −4, 5)T, | ||||||
Optimum point: x ∗ = (0.75, 0.75, 0.75, 0.3381465, 0.25, 0.25, 0.25)T, | ||||||
y ∗ = (0, 0.2206178, 0, −1, 0)T, | ||||||
Optimum value: f(x ∗, y ∗) = 0.5. |
Problem 8
Dimension: 14 (8,6), | |||||||
Component functions: | |||||||
\(f_1(x) = \sum \limits _{i=1}^8 \max \limits \{0,x_i\},\) | |||||||
f 2(y) = |y 1| + 2|y 2| + 2|y 3| + |y 4| + |y 5| + 5|y 6| + y 2 − y 5 − 4y 6, | |||||||
Linear constraints: | |||||||
x 1 + 2x 2 + x 3 + x 4 + y 1 = 5, | |||||||
3x 1 + x 2 + 2x 3 − x 4 + y 2 = 4, | |||||||
x 5 + 2x 6 + x 7 + x 8 + y 3 = 5, | |||||||
3x 5 + x 6 + 2x 7 − x 8 + y 4 = 4, | |||||||
x 2 + 4x 3 − y 5 = 1.5, | |||||||
Starting point: x 0 = (1, …, 1)T, y 0 = (2, …, 2)T, | |||||||
Optimum point: | |||||||
x ∗ = (0.6, 2.2, 0, 0, 0.6, 2.2, 0, 0)T, | |||||||
y ∗ = (0, 0, 0, 0, 0.7, 0)T, | |||||||
Optimum value:f ∗ = 5.6. |
Problem 9
Dimension: 17 (10,7), | ||||||||
Component functions: | ||||||||
f 1(x) = |x 1| + |x 2| + |x 1 + x 2|− x 1 − 2x 2 + |x 3 − 10| + 4|x 4 − 5|, | ||||||||
+ |x 5 − 3| + 2|x 6 − 1| + 5|x 7| + 7|x 8 − 11| + 2|x 9 − 10| + |x 10 − 7|, | ||||||||
\(f_2(y) = |y_1|+|y_2|+10\sum \limits _{i=3}^6 \max \limits \{0,y_i-y_{i+1}\}\), | ||||||||
Linear constraints: | ||||||||
4x 1 + 5x 2 − 3x 7 + 9x 8 + 2y 1 + y 3 + y 4 = 105, | ||||||||
10x 1 − 8x 2 − 17x 7 + 2x 8 + y 2 + y 4 + 5y 5 = 0, | ||||||||
8x 1 − 2x 2 − 5x 9 + 2x 10 − 3y 5 − y 6 = −12, | ||||||||
3x 1 − 6x 2 − 12x 9 + 7x 10 − 2y 6 − y 7 = −96, | ||||||||
Starting point: x 0 = (2, …, 2)T, y 0 = (1, …, 1)T, | ||||||||
Optimum point: | ||||||||
x ∗ = (1.4431818, 1.6136364, 10, 5, 3, 1, 0, 11, 10, 7)T, | ||||||||
y ∗ = (0, 0, −3.9204544, −3.9204544, −3.9204544, −3.9204544, 27.4886356)T, | ||||||||
Optimum value: f ∗ = 1.4431818. |
Problem 10
Dimension: 30 (15,15), | |||||||||
Component functions: | |||||||||
\(f_1(x) = 15 \max \limits _{i=1,\ldots ,15} |x_i|,\) | |||||||||
\(f_2(y) = \sum \limits _{i=1}^{15} |y_i|\), | |||||||||
Linear constraints: x i + 2x i+1 − 3x i+2 − y i − 2y i+2 = i − 1, i = 1, …, 13, | |||||||||
Starting point: | |||||||||
x 0 = (x 1, …, x 15)T, x i = i, i ≤ 7, x i = −i, i > 7, | |||||||||
y 0 = (y 1, …, y 15)T, y i = −i, i ≤ 7, y i = i, i > 7, | |||||||||
Optimum point: | |||||||||
x ∗ = (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)T, | |||||||||
y ∗ = (0, 0, 0, −0.5, −1, −1.25, −1.5, −1.88, −2.25, −2.56, −2.86, | |||||||||
− 3.22, −3.56, −3.89, −4.22)T, | |||||||||
Optimum value: f ∗ = 28.703125. |
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Bagirov, A.M., Taheri, S., Bai, F., Wu, Z. (2019). An Approximate ADMM for Solving Linearly Constrained Nonsmooth Optimization Problems with Two Blocks of Variables. In: Hosseini, S., Mordukhovich, B., Uschmajew, A. (eds) Nonsmooth Optimization and Its Applications. International Series of Numerical Mathematics, vol 170. Birkhäuser, Cham. https://doi.org/10.1007/978-3-030-11370-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-11370-4_2
Published:
Publisher Name: Birkhäuser, Cham
Print ISBN: 978-3-030-11369-8
Online ISBN: 978-3-030-11370-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)