# QSDPNAL: a two-phase augmented Lagrangian method for convex quadratic semidefinite programming

- 10 Downloads

## Abstract

In this paper, we present a two-phase augmented Lagrangian method, called QSDPNAL, for solving convex quadratic semidefinite programming (QSDP) problems with constraints consisting of a large number of linear equality and inequality constraints, a simple convex polyhedral set constraint, and a positive semidefinite cone constraint. A first order algorithm which relies on the inexact Schur complement based decomposition technique is developed in QSDPNAL-Phase I with the aim of solving a QSDP problem to moderate accuracy or using it to generate a reasonably good initial point for the second phase. In QSDPNAL-Phase II, we design an augmented Lagrangian method (ALM) wherein the inner subproblem in each iteration is solved via inexact semismooth Newton based algorithms. Simple and implementable stopping criteria are designed for the ALM. Moreover, under mild conditions, we are able to establish the rate of convergence of the proposed algorithm and prove the R-(super)linear convergence of the KKT residual. In the implementation of QSDPNAL, we also develop efficient techniques for solving large scale linear systems of equations under certain subspace constraints. More specifically, simpler and yet better conditioned linear systems are carefully designed to replace the original linear systems and novel shadow sequences are constructed to alleviate the numerical difficulties brought about by the crucial subspace constraints. Extensive numerical results for various large scale QSDPs show that our two-phase algorithm is highly efficient and robust in obtaining accurate solutions. The software reviewed as part of this submission was given the DOI (Digital Object Identifier) https://doi.org/10.5281/zenodo.1206980.

## Keywords

Quadratic semidefinite programming Schur complement Augmented Lagrangian Inexact semismooth Newton method## Mathematics Subject Classification

90C06 90C20 90C22 90C25 65F10## References

- 1.Alfakih, A.Y., Khandani, A., Wolkowicz, H.: Solving Euclidean distance matrix completion problems via semidefinite programming. Comput. Optim. Appl.
**12**, 13–30 (1999)MathSciNetCrossRefMATHGoogle Scholar - 2.Bauschke, H.H., Borwein, J.M., Li, W.: Strong conical hull intersection property, bounded linear regularity, Jameson’s property (G), and error bounds in convex optimization. Math. Program.
**86**, 135–160 (1999)MathSciNetCrossRefMATHGoogle Scholar - 3.Biswas, P., Liang, T.C., Toh, K.-C., Wang, T.C., Ye, Y.: Semidefinite programming approaches for sensor network localization with noisy distance measurements. IEEE Trans. Autom. Sci. Eng.
**3**, 360–371 (2006)CrossRefGoogle Scholar - 4.Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2000)CrossRefMATHGoogle Scholar
- 5.Burkard, R.E., Karisch, S.E., Rendl, F.: QAPLIB—a quadratic assignment problem library. J. Global Optim.
**10**, 391–403 (1997)MathSciNetCrossRefMATHGoogle Scholar - 6.Chen, L., Sun, D.F., Toh, K.-C.: An efficient inexact symmetric Gauss–Seidel based majorized ADMM for high-dimensional convex composite conic programming. Math. Program.
**161**, 237–270 (2017)MathSciNetCrossRefMATHGoogle Scholar - 7.Cui, Y., Sun, D.F., Toh, K.-C.: On the asymptotic superlinear convergence of the augmented Lagrangian method for semidefinite programming with multiple solutions, arXiv:1610.00875 (2016)
- 8.Clarke, F.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)MATHGoogle Scholar
- 9.Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer, New York (2003)MATHGoogle Scholar
- 10.Han, D., Sun, D., Zhang, L.: Linear rate convergence of the alternating direction method of multipliers for convex composite programming. Math. Oper. Res. (2017). https://doi.org/10.1287/moor.2017.0875
- 11.Higham, N.J.: Computing the nearest correlation matrix—a problem from finance. IMA J. Numer. Anal.
**22**, 329–343 (2002)MathSciNetCrossRefMATHGoogle Scholar - 12.Hiriart-Urruty, J.-B., Strodiot, J.-J., Nguyen, V.H.: Generalized Hessian matrix and second-order optimality conditions for problems with \({C}^{1,1}\) data. Appl. Math. Optim.
**11**, 43–56 (1984)MathSciNetCrossRefMATHGoogle Scholar - 13.Jiang, K., Sun, D.F., Toh, K.-C.: An inexact accelerated proximal gradient method for large scale linearly constrained convex SDP. SIAM J. Optim.
**22**, 1042–1064 (2012)MathSciNetCrossRefMATHGoogle Scholar - 14.Jiang, K., Sun, D.F., Toh, K.-C.: A partial proximal point algorithm for nuclear norm regularized matrix least squares problems. Math. Program. Comput.
**6**, 281–325 (2014)MathSciNetCrossRefMATHGoogle Scholar - 15.Krislock, N., Lang, J., Varah, J., Pai, D.K., Seidel, H.-P.: Local compliance estimation via positive semidefinite constrained least squares. IEEE Trans. Robot.
**20**, 1007–1011 (2004)CrossRefGoogle Scholar - 16.Li, L., Toh, K.-C.: An inexact interior point method for l1-regularized sparse covariance selection. Math. Program. Comput.
**2**, 291–315 (2010)MathSciNetCrossRefMATHGoogle Scholar - 17.Li, X.D.: A Two-Phase Augmented Lagrangian Method for Convex Composite Quadratic Programming, PhD thesis, Department of Mathematics, National University of Singapore (2015)Google Scholar
- 18.Li, X.D., Sun, D.F., Toh, K.-C.: A Schur complement based semi-proximal ADMM for convex quadratic conic programming and extensions. Math. Program.
**155**, 333–373 (2016)MathSciNetCrossRefMATHGoogle Scholar - 19.Nie, J.W., Yuan, Y.X.: A predictor-corrector algorithm for QSDP combining Dikin-type and Newton centering steps. Ann. Oper. Res.
**103**, 115–133 (2001)MathSciNetCrossRefMATHGoogle Scholar - 20.Pang, J.-S., Sun, D.F., Sun, J.: Semismooth homeomorphisms and strong stability of semidefinite and Lorentz complementarity problems. Math. Oper. Res.
**28**, 39–63 (2003)MathSciNetCrossRefMATHGoogle Scholar - 21.Povh, J., Rendl, F.: Copositive and semidefinite relaxations of the quadratic assignment problem. Discrete Optim.
**6**, 231–241 (2009)MathSciNetCrossRefMATHGoogle Scholar - 22.Qi, H.D.: Local duality of nonlinear semidefinite programming. Math. Oper. Res.
**34**, 124–141 (2009)MathSciNetCrossRefMATHGoogle Scholar - 23.Qi, H.D., Sun, D.F.: A quadratically convergent Newton method for computing the nearest correlation matrix. SIAM J. Matrix Anal. Appl.
**28**, 360–385 (2006)MathSciNetCrossRefMATHGoogle Scholar - 24.Rockafellar, R.T.: Conjugate Duality and Optimization, CBMS-NSF Regional Conf. Ser. Appl. Math. vol. 16. SIAM, Philadelphia (1974)Google Scholar
- 25.Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim.
**14**, 877–898 (1976)MathSciNetCrossRefMATHGoogle Scholar - 26.Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res.
**1**, 97–116 (1976)MathSciNetCrossRefMATHGoogle Scholar - 27.Sun, D.F., Sun, J.: Semismooth matrix-valued functions. Math. Oper. Res.
**27**, 150–169 (2002)MathSciNetCrossRefMATHGoogle Scholar - 28.Sun, D.F., Toh, K.-C., Yang, L.: A convergent 3-block semiproximal alternating direction method of multipliers for conic programming with 4-type constraints. SIAM J. Optim.
**25**, 882–915 (2015)MathSciNetCrossRefMATHGoogle Scholar - 29.Sun, D.F., Toh, K.-C., Yang, L.: An efficient inexact ABCD method for least squares semidefinite programming. SIAM J. Optim.
**26**, 1072–1100 (2016)MathSciNetCrossRefMATHGoogle Scholar - 30.Sun, J., Zhang, S.: A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs. Eur. J. Oper. Res.
**207**, 1210–1220 (2010)MathSciNetCrossRefMATHGoogle Scholar - 31.Toh, K.-C.: An inexact primal-dual path following algorithm for convex quadratic SDP. Math. Program.
**112**, 221–254 (2008)MathSciNetCrossRefMATHGoogle Scholar - 32.Toh, K.-C., Tütüncü, R., Todd, M.: Inexact primal-dual path-following algorithms for a special class of convex quadratic SDP and related problems. Pac. J. Optim.
**3**, 135–164 (2007)MathSciNetMATHGoogle Scholar - 33.Yang, L., Sun, D.F., Toh, K.-C.: SDPNAL+: a majorized semismooth Newton-CG augmented Lagrangian method for semidefinite programming with nonnegative constraints. Math. Program. Comput.
**7**, 331–366 (2015)MathSciNetCrossRefMATHGoogle Scholar - 34.Zhao, X.Y.: A Semismooth Newton-CG Augmented Lagrangian Method for Large Scale Linear and Convex Quadratic SDPs, PhD thesis, Department of Mathematics, National University of Singapore (2009)Google Scholar
- 35.Zhao, X.Y., Sun, D.F., Toh, K.-C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim.
**20**, 1737–1765 (2010)MathSciNetCrossRefMATHGoogle Scholar