Advertisement

Journal of Optimization Theory and Applications

, Volume 166, Issue 3, pp 906–929 | Cite as

Further Study on the Convergence Rate of Alternating Direction Method of Multipliers with Logarithmic-quadratic Proximal Regularization

  • Caihua Chen
  • Min Li
  • Xiaoming Yuan
Article

Abstract

In the literature, the combination of the alternating direction method of multipliers with the logarithmic-quadratic proximal regularization has been proved to be convergent, and its worst-case convergence rate in the ergodic sense has been established. In this paper, we focus on a convex minimization model and consider an inexact version of the combination of the alternating direction method of multipliers with the logarithmic-quadratic proximal regularization. Our primary purpose is to further study its convergence rate and to establish its worst-case convergence rates measured by the iteration complexity in both the ergodic and non-ergodic senses. In particular, existing convergence rate results for this combination are subsumed by the new results.

Keywords

Convex programming Alternating direction method of multipliers Logarithmic-quadratic proximal Convergence rate Iteration complexity  

Mathematics Subject Classification

90C25 90C33 65K05 

Notes

Acknowledgments

The first author was supported in part by the Natural Science Foundation of Jiangsu Province Grant BK20130550 and the NSFC Grant 11401300. The second author was supported by the Program for New Century Excellent Talents in University Grant NCET-12-0111, Qing Lan Project and the Natural Science Foundation of Jiangsu Province Grant BK2012662. The third author was partially supported by the Grant from Hong Kong Baptist University: FRG2/13-14/061 and the General Research Fund from Hong Kong Research Grants Council: 203613.

References

  1. 1.
    Glowinski, R., Marroco, A.: Sur l’approximation, paréléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. ESAIM: Math. Model. Numer. Anal. – Modélisation Mathématique et Analyse Numérique 9(R2), 41–76 (1975)Google Scholar
  2. 2.
    Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite-element approximations. Comput. Math. Appl. 2, 17–40 (1976)CrossRefMATHGoogle Scholar
  3. 3.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)CrossRefMATHGoogle Scholar
  4. 4.
    Eckstein, J., Yao W.: Augmented Lagrangian and alternating direction methods for convex optimization: A tutorial and some illustrative computational results. RUTCOR Research Report RRR 32-2012 (2012)Google Scholar
  5. 5.
    Glowinski, R.: On alternating direction methods of multipliers: a historical perspective. In: Fitzgibbon, W., Kuznetsov, Yu. A., Neittaanmaki, P., Pironneau, O. (eds.), Modeling, Simulation and Optimization for Science and Technology. Computational Methods in Applied Sciences, vol. 34, pp. 59–82. Springer, Dordrecht (2014)Google Scholar
  6. 6.
    Auslender, A., Teboulle, M., Ben-Tiba, S.: A logarithmic-quadratic proximal method for variational inequalities. Comput. Optim. Appl. 12, 31–40 (1999)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Auslender, A., Teboulle, M.: Entropic proximal decomposition methods for convex programs and variational inequalities. Math. Program. 91, 33–47 (2001)MathSciNetMATHGoogle Scholar
  8. 8.
    Auslender, A., Teboulle, M.: Interior gradient and epsilon-subgradient descent methods for constrained convex minimization. Math. Oper. Res. 29, 1–26 (2004)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Auslender, A., Teboulle, M.: Interior gradient and proximal methods for convex and conic optimization. SIAM J. Optim. 16, 697–725 (2006)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Auslender, A., Teboulle, M.: Lagrangian duality and related multiplier methods for variational inequality problems. SIAM J. Optim. 10, 1097–1115 (2000)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    He, B.S., Liao, L-Z, Yuan, X.M.: A LQP based interior prediction-correction method for nonlinear complementarity problems. J. Comput. Math. 24(1), 33–44 (2006)Google Scholar
  12. 12.
    Tao, M., Yuan, X.M.: On the \(O(1/t)\) convergence rate of alternating direction method with logarithmic-quadratic proximal regularization. SIAM J. Optim. 22, 1431–1448 (2012)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Yuan, X.M., Li, M.: An LQP-based decomposition method for solving a class of variational inequalities. SIAM J. Optim. 21(4), 1309–1318 (2011)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    He, B.S., Yuan, X.M.: On the \(O(1/n)\) convergence rate of Douglas-Rachford alternating direction method. SIAM J. Num. Anal. 50, 700–709 (2012)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Li, M., Liao, L.-Z, Yuan, X.M.: Inexact alternating direction method of multipliers with logarithmic-quadratic proximal regularization. J. Optim. Theory Appl. 159, 412–436 (2013)Google Scholar
  16. 16.
    Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    He, B.S., Yuan, X.M.: On nonergodic convergence rate of Douglas-Rachford alternating direction method of multipliers. Numer. Math. under minor revision.Google Scholar
  18. 18.
    Nemirovsky, A.S., Yudin, D.B.: Problem Complexity and Method Efficiency in Optimization. Wiley, New York (1983)MATHGoogle Scholar
  19. 19.
    Nesterov, Y.E.: A method for solving the convex programming problem with convergence rate \(O(1/k^2)\). Dokl. Akad. Nauk SSSR 269, 543–547 (1983). (in Russian)MathSciNetGoogle Scholar
  20. 20.
    Deng, W., Lai, M.J., Peng, Z.M., Yin, W.T.: Parallel multi-block ADMM with \(o(1/k)\) convergence. manuscript (2013)Google Scholar
  21. 21.
    Ito, K., Kunisch, K.: Lagrange Multiplier Approach to Variational Problems and Applications. Society for Industrial and Applied Mathematics, Philadelphia (2008)CrossRefMATHGoogle Scholar
  22. 22.
    Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer Series in Operations Research. Springer, New York (2003)Google Scholar
  23. 23.
    Nesterov, Y.: Gradient methods for minimizing composite objective function. Core Discuss. Paper 2007(76), 1–30 (2007)MathSciNetGoogle Scholar
  24. 24.
    Li, M., Yuan, X.M.: A strictly contractive Peaceman-Rachford splitting method with logarithmic-quadratic proximal regularization for convex programming. to appear in Math. Oper. Res. (2014).Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.School of Management and EngineeringNanjing UniversityNanjingChina
  2. 2.School of Economics and ManagementSoutheast UniversityNanjingChina
  3. 3.Department of MathematicsHong Kong Baptist UniversityHong KongChina

Personalised recommendations