Skip to main content
Log in

Further Study on the Convergence Rate of Alternating Direction Method of Multipliers with Logarithmic-quadratic Proximal Regularization

Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In the literature, the combination of the alternating direction method of multipliers with the logarithmic-quadratic proximal regularization has been proved to be convergent, and its worst-case convergence rate in the ergodic sense has been established. In this paper, we focus on a convex minimization model and consider an inexact version of the combination of the alternating direction method of multipliers with the logarithmic-quadratic proximal regularization. Our primary purpose is to further study its convergence rate and to establish its worst-case convergence rates measured by the iteration complexity in both the ergodic and non-ergodic senses. In particular, existing convergence rate results for this combination are subsumed by the new results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Notes

  1. Note that, in [15], the same convergence rate has been established also for the combination of the general ADMM in [16] with the LQP regularization. But in this paper, we stick to the original ADMM and do not discuss the general ADMM.

References

  1. Glowinski, R., Marroco, A.: Sur l’approximation, paréléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. ESAIM: Math. Model. Numer. Anal. – Modélisation Mathématique et Analyse Numérique 9(R2), 41–76 (1975)

  2. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite-element approximations. Comput. Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  3. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)

    Article  MATH  Google Scholar 

  4. Eckstein, J., Yao W.: Augmented Lagrangian and alternating direction methods for convex optimization: A tutorial and some illustrative computational results. RUTCOR Research Report RRR 32-2012 (2012)

  5. Glowinski, R.: On alternating direction methods of multipliers: a historical perspective. In: Fitzgibbon, W., Kuznetsov, Yu. A., Neittaanmaki, P., Pironneau, O. (eds.), Modeling, Simulation and Optimization for Science and Technology. Computational Methods in Applied Sciences, vol. 34, pp. 59–82. Springer, Dordrecht (2014)

  6. Auslender, A., Teboulle, M., Ben-Tiba, S.: A logarithmic-quadratic proximal method for variational inequalities. Comput. Optim. Appl. 12, 31–40 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  7. Auslender, A., Teboulle, M.: Entropic proximal decomposition methods for convex programs and variational inequalities. Math. Program. 91, 33–47 (2001)

    MathSciNet  MATH  Google Scholar 

  8. Auslender, A., Teboulle, M.: Interior gradient and epsilon-subgradient descent methods for constrained convex minimization. Math. Oper. Res. 29, 1–26 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  9. Auslender, A., Teboulle, M.: Interior gradient and proximal methods for convex and conic optimization. SIAM J. Optim. 16, 697–725 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  10. Auslender, A., Teboulle, M.: Lagrangian duality and related multiplier methods for variational inequality problems. SIAM J. Optim. 10, 1097–1115 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  11. He, B.S., Liao, L-Z, Yuan, X.M.: A LQP based interior prediction-correction method for nonlinear complementarity problems. J. Comput. Math. 24(1), 33–44 (2006)

  12. Tao, M., Yuan, X.M.: On the \(O(1/t)\) convergence rate of alternating direction method with logarithmic-quadratic proximal regularization. SIAM J. Optim. 22, 1431–1448 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  13. Yuan, X.M., Li, M.: An LQP-based decomposition method for solving a class of variational inequalities. SIAM J. Optim. 21(4), 1309–1318 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  14. He, B.S., Yuan, X.M.: On the \(O(1/n)\) convergence rate of Douglas-Rachford alternating direction method. SIAM J. Num. Anal. 50, 700–709 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  15. Li, M., Liao, L.-Z, Yuan, X.M.: Inexact alternating direction method of multipliers with logarithmic-quadratic proximal regularization. J. Optim. Theory Appl. 159, 412–436 (2013)

  16. Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  17. He, B.S., Yuan, X.M.: On nonergodic convergence rate of Douglas-Rachford alternating direction method of multipliers. Numer. Math. under minor revision.

  18. Nemirovsky, A.S., Yudin, D.B.: Problem Complexity and Method Efficiency in Optimization. Wiley, New York (1983)

    MATH  Google Scholar 

  19. Nesterov, Y.E.: A method for solving the convex programming problem with convergence rate \(O(1/k^2)\). Dokl. Akad. Nauk SSSR 269, 543–547 (1983). (in Russian)

    MathSciNet  Google Scholar 

  20. Deng, W., Lai, M.J., Peng, Z.M., Yin, W.T.: Parallel multi-block ADMM with \(o(1/k)\) convergence. manuscript (2013)

  21. Ito, K., Kunisch, K.: Lagrange Multiplier Approach to Variational Problems and Applications. Society for Industrial and Applied Mathematics, Philadelphia (2008)

    Book  MATH  Google Scholar 

  22. Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer Series in Operations Research. Springer, New York (2003)

  23. Nesterov, Y.: Gradient methods for minimizing composite objective function. Core Discuss. Paper 2007(76), 1–30 (2007)

    MathSciNet  Google Scholar 

  24. Li, M., Yuan, X.M.: A strictly contractive Peaceman-Rachford splitting method with logarithmic-quadratic proximal regularization for convex programming. to appear in Math. Oper. Res. (2014).

Download references

Acknowledgments

The first author was supported in part by the Natural Science Foundation of Jiangsu Province Grant BK20130550 and the NSFC Grant 11401300. The second author was supported by the Program for New Century Excellent Talents in University Grant NCET-12-0111, Qing Lan Project and the Natural Science Foundation of Jiangsu Province Grant BK2012662. The third author was partially supported by the Grant from Hong Kong Baptist University: FRG2/13-14/061 and the General Research Fund from Hong Kong Research Grants Council: 203613.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xiaoming Yuan.

Additional information

Communicated by Roland Glowinski.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, C., Li, M. & Yuan, X. Further Study on the Convergence Rate of Alternating Direction Method of Multipliers with Logarithmic-quadratic Proximal Regularization. J Optim Theory Appl 166, 906–929 (2015). https://doi.org/10.1007/s10957-014-0682-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-014-0682-8

Keywords

Mathematics Subject Classification

Navigation