Skip to main content
SpringerLink
Log in

Two-Step Fixed-Point Proximity Algorithms for Multi-block Separable Convex Problems

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

Multi-block separable convex problems recently received considerable attention. Optimization problems of this type minimize separable convex objective functions with linear constraints. Challenges encountered in algorithmic development applying the classic alternating direction method of multipliers (ADMM) come from the fact that ADMM for the optimization problems of this type is not necessarily convergent. However, it is observed that ADMM applying to problems of this type outperforms numerically many of its variants with guaranteed theoretical convergence. The goal of this paper is to develop convergent and computationally efficient algorithms for solving multi-block separable convex problems. We first characterize the solutions of the optimization problems by proximity operators of the convex functions involved in their objective functions. We then design a class of two-step fixed-point iterative schemes for solving these problems based on the characterization. We further prove convergence of the iterative schemes and show that they have \(O\left( \frac{1}{k}\right) \) of convergence rate in the ergodic sense and the sense of the partial primal-dual gap, where k denotes the iteration number. Moreover, we derive specific two-step fixed-point proximity algorithms (2SFPPA) from the proposed iterative schemes and establish their global convergence. Numerical experiments for solving the sparse MRI problem demonstrate the numerical efficiency of the proposed 2SFPPA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Attouch, H., Briceno-Arias, L.M., Combettes, P.L.: A parallel splitting method for coupled monotone inclusions. SIAM J. Control Optim. 48, 3246–3270 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, AMS Books in Mathematics. Springer, New York (2011)

    Book  MATH  Google Scholar 

  3. Cai, J., Chan, R., Shen, Z.: A framelet-based image inpainting algorithm. Appl. Comput. Harmonic Anal. 24, 131–149 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  4. Cai, X., Han, D., Yuan, X.: On the convergence of the direct extension of ADMM for three-block separable convex minimization models with one strongly convex function. Comput. Optim. Appl. (2016). doi:10.1007/s10589-016-9860-y

    Google Scholar 

  5. Cai, J., Osher, S., Shen, Z.: Linearized Bregman iteration for frame based image deblurring. SIAM J. Imaging Sci. 2, 226–252 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  6. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120–145 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  7. Chan, R., Chan, T., Shen, L., Shen, Z.: Wavelet algorithms for high-resolution image reconstruction. SIAM J. Sci. Comput. 24, 1408–1432 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  8. Chan, R., Riemenschneider, S.D., Shen, L., Shen, Z.: Tight frame: the efficient way for high-resolution image reconstruction. Appl. Comput. Harmonic Anal. 17, 91–115 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chen, C., He, B., Ye, Y., Yuan, X.: The direct extension of admm for multi-block convex minimization problems is not necessarily convergent. Math. Program. 155, 57–79 (2016)

  10. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20, 273–297 (1995)

    MATH  Google Scholar 

  11. Davis, D., Yin, W.: A three-operator splitting scheme and its optimization applications. UCLA CAM Report 15–13

  12. Deng, W., Lai, M.-J., Peng, Z., Yin, W.: Parallel multi-block admm with o(1/k) convergence, UCLA CAM 13–64 (2014)

  13. Esser, E., Zhang, X., Chan, T.F.: A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM J. Imaging Sci. 3, 1015–1046 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. comput. math. appl. 2(1), 17–40. Comput. Math.Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  15. Goldstein, T., Osher, S.: The split Bregman method for \(\ell ^1\) regularization problems. SIAM J. Imaging Sci. 2, 323–343 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  16. He, B., Tao, M., Yuan, X.: Alternating direction method with gaussian back substitution for separable convex programming. SIAM J. Optim. 22, 313–340 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  17. He, B., Yuan, X.: Linearized alternating direction method of multipliers with gaussian back substitution for seperable convex programming. Numer. Algebra Control Optim. 22, 247–260 (2013)

    Article  MATH  Google Scholar 

  18. Li, M., Sun, D., Toh, K.-C.: A convergent 3-block semi-proximal admm for convex minimization problems with one strongly convex block. Asia Pacific J. Oper. Res. 32, 1550024 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  19. Li, Q., Micchelli, C.A., Shen, L., Xu, Y.: A proximity algorithm accelerated by Gauss-Seidel iterations for L1/TV denoising models. Inverse Probl. 28, 095003 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  20. Li, Q., Shen, L., Yuesheng, X., Zhang, N.: Multi-step fixed-point proximity algorithms for solving a class of convex optimization problems arising from image processing. Adv. Comput. Math. 41, 387–422 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  21. Li, Q., Shen, L., Yang, L.: Split-bregman iteration for framelet based image inpainting. Appl. Comput. Harmonic Anal. 32, 145–154 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  22. Li, Q., Zhang, N.: Fast proximity-gradient algorithms for structured convex optimization problems. Appl. Comput. Harmonic Anal. 41, 491–517 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  23. Lin, T., Ma, S., Zhang, S.: On the convergence rate of multi-block admm. J. Oper. Res. Soc. China 3, 251–274 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  24. Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn. Reson. Med. 58, 1182–1195 (2007)

    Article  Google Scholar 

  25. Micchelli, C.A., Shen, L., Xu, Y.: Proximity algorithms for image models: denoising. Inverse Probl. 27, 045009 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  26. Moreau, J.J.: Fonctions convexes duales et points proximaux dans un espace hilbertien. C.R. Acad. Sci. Paris Sér. A Math. 255, 1897–2899 (1962)

    MathSciNet  MATH  Google Scholar 

  27. Nikolova, M.: Local strong homogeneity of a regularized estimator. SIAM J. Appl. Math. 61, 633–658 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  28. Rudin, L.I., Osher, S.: Total variation based image restoration with free local constraints, In: IEEE International Conference on Image Processing, pp. 31–35 (1994)

  29. Ruszczynski, A.: Parallel decomposition of multistage stochastic programming problems. Math. Program. 58, 201–228 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  30. Sawatzky, A., Qi, X., Schirra, C.O., Anastasio, M.A.: Proximal ADMM for multi-channel image reconstruction in spectral X-ray CT. IEEE Trans. Med. Imaging 33, 1657–1668 (2014)

    Article  Google Scholar 

  31. Shi, W., Ling, W., Wu, W., Yin, W.: Extra: an exact first-order algorithm for decentralized consensus optimization. SIAM J. Optim. 25, 944–966 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  32. Sun, D., Toh, K.C., Yang, L.: A convergent 3-block semi-proximal alternating direction method of multipliers for conic programming with \(4\)-type of constraints. SIAM J. Optim. 25, 882–915 (2014)

  33. Tyrrell Rockafellar, R.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  34. Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., Knight, K.: Sparsity and smoothness via the fused lasso. J. R. Stat. Soc. 67, 91–108 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  35. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  36. Wen, Z., Goldfarb, D., Yin, W.: Alternating direction augmented lagrangian methods for semidefinite programming. Math. Program. Comput. 2, 203–230 (2010)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Applied Mathematics, College of Mathematics and Informatics, South China Agricultural University, Guangzhou, 510642, People’s Republic of China

    Na Zhang

  2. School of Data and Computer Sciences and Guangdong Province Key Laboratory of Computational Science, Sun Yat-sen University, Guangzhou, 510275, People’s Republic of China

    Qia Li & Yuesheng Xu

  3. Syracuse University, Syracuse, NY, 13244, USA

    Yuesheng Xu

Authors
  1. Qia Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuesheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Na Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Na Zhang.

Additional information

This research is supported in part by Guangdong Provincial Government of China through the “Computational Science Innovative Research Team” program, by the Natural Science Foundation of China under Grants 11501584, 11471013 and 91530117, by the US National Science Foundation under Grant DMS-1522332, and by the Natural Science Foundation of Guangdong Province under Grants 2014A030310332 and 2014A030310414.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Q., Xu, Y. & Zhang, N. Two-Step Fixed-Point Proximity Algorithms for Multi-block Separable Convex Problems. J Sci Comput 70, 1204–1228 (2017). https://doi.org/10.1007/s10915-016-0278-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-016-0278-6

Keywords

Search

Navigation