# QSDPNAL: a two-phase augmented Lagrangian method for convex quadratic semidefinite programming

- 3.5k Downloads
- 1 Citations

## Abstract

In this paper, we present a two-phase augmented Lagrangian method, called QSDPNAL, for solving convex quadratic semidefinite programming (QSDP) problems with constraints consisting of a large number of linear equality and inequality constraints, a simple convex polyhedral set constraint, and a positive semidefinite cone constraint. A first order algorithm which relies on the inexact Schur complement based decomposition technique is developed in QSDPNAL-Phase I with the aim of solving a QSDP problem to moderate accuracy or using it to generate a reasonably good initial point for the second phase. In QSDPNAL-Phase II, we design an augmented Lagrangian method (ALM) wherein the inner subproblem in each iteration is solved via inexact semismooth Newton based algorithms. Simple and implementable stopping criteria are designed for the ALM. Moreover, under mild conditions, we are able to establish the rate of convergence of the proposed algorithm and prove the R-(super)linear convergence of the KKT residual. In the implementation of QSDPNAL, we also develop efficient techniques for solving large scale linear systems of equations under certain subspace constraints. More specifically, simpler and yet better conditioned linear systems are carefully designed to replace the original linear systems and novel shadow sequences are constructed to alleviate the numerical difficulties brought about by the crucial subspace constraints. Extensive numerical results for various large scale QSDPs show that our two-phase algorithm is highly efficient and robust in obtaining accurate solutions. The software reviewed as part of this submission was given the DOI (Digital Object Identifier) https://doi.org/10.5281/zenodo.1206980.

## Keywords

Quadratic semidefinite programming Schur complement Augmented Lagrangian Inexact semismooth Newton method## Mathematics Subject Classification

90C06 90C20 90C22 90C25 65F10## References

- 1.Alfakih, A.Y., Khandani, A., Wolkowicz, H.: Solving Euclidean distance matrix completion problems via semidefinite programming. Comput. Optim. Appl.
**12**, 13–30 (1999)MathSciNetCrossRefGoogle Scholar - 2.Bauschke, H.H., Borwein, J.M., Li, W.: Strong conical hull intersection property, bounded linear regularity, Jameson’s property (G), and error bounds in convex optimization. Math. Program.
**86**, 135–160 (1999)MathSciNetCrossRefGoogle Scholar - 3.Biswas, P., Liang, T.C., Toh, K.-C., Wang, T.C., Ye, Y.: Semidefinite programming approaches for sensor network localization with noisy distance measurements. IEEE Trans. Autom. Sci. Eng.
**3**, 360–371 (2006)CrossRefGoogle Scholar - 4.Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2000)CrossRefGoogle Scholar
- 5.Burkard, R.E., Karisch, S.E., Rendl, F.: QAPLIB—a quadratic assignment problem library. J. Global Optim.
**10**, 391–403 (1997)MathSciNetCrossRefGoogle Scholar - 6.Chen, L., Sun, D.F., Toh, K.-C.: An efficient inexact symmetric Gauss–Seidel based majorized ADMM for high-dimensional convex composite conic programming. Math. Program.
**161**, 237–270 (2017)MathSciNetCrossRefGoogle Scholar - 7.Cui, Y., Sun, D.F., Toh, K.-C.: On the asymptotic superlinear convergence of the augmented Lagrangian method for semidefinite programming with multiple solutions, arXiv:1610.00875 (2016)
- 8.Clarke, F.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)zbMATHGoogle Scholar
- 9.Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer, New York (2003)zbMATHGoogle Scholar
- 10.Han, D., Sun, D., Zhang, L.: Linear rate convergence of the alternating direction method of multipliers for convex composite programming. Math. Oper. Res. (2017). https://doi.org/10.1287/moor.2017.0875 MathSciNetCrossRefGoogle Scholar
- 11.Higham, N.J.: Computing the nearest correlation matrix—a problem from finance. IMA J. Numer. Anal.
**22**, 329–343 (2002)MathSciNetCrossRefGoogle Scholar - 12.Hiriart-Urruty, J.-B., Strodiot, J.-J., Nguyen, V.H.: Generalized Hessian matrix and second-order optimality conditions for problems with \({C}^{1,1}\) data. Appl. Math. Optim.
**11**, 43–56 (1984)MathSciNetCrossRefGoogle Scholar - 13.Jiang, K., Sun, D.F., Toh, K.-C.: An inexact accelerated proximal gradient method for large scale linearly constrained convex SDP. SIAM J. Optim.
**22**, 1042–1064 (2012)MathSciNetCrossRefGoogle Scholar - 14.Jiang, K., Sun, D.F., Toh, K.-C.: A partial proximal point algorithm for nuclear norm regularized matrix least squares problems. Math. Program. Comput.
**6**, 281–325 (2014)MathSciNetCrossRefGoogle Scholar - 15.Krislock, N., Lang, J., Varah, J., Pai, D.K., Seidel, H.-P.: Local compliance estimation via positive semidefinite constrained least squares. IEEE Trans. Robot.
**20**, 1007–1011 (2004)CrossRefGoogle Scholar - 16.Li, L., Toh, K.-C.: An inexact interior point method for l1-regularized sparse covariance selection. Math. Program. Comput.
**2**, 291–315 (2010)MathSciNetCrossRefGoogle Scholar - 17.Li, X.D.: A Two-Phase Augmented Lagrangian Method for Convex Composite Quadratic Programming, PhD thesis, Department of Mathematics, National University of Singapore (2015)Google Scholar
- 18.Li, X.D., Sun, D.F., Toh, K.-C.: A Schur complement based semi-proximal ADMM for convex quadratic conic programming and extensions. Math. Program.
**155**, 333–373 (2016)MathSciNetCrossRefGoogle Scholar - 19.Nie, J.W., Yuan, Y.X.: A predictor-corrector algorithm for QSDP combining Dikin-type and Newton centering steps. Ann. Oper. Res.
**103**, 115–133 (2001)MathSciNetCrossRefGoogle Scholar - 20.Pang, J.-S., Sun, D.F., Sun, J.: Semismooth homeomorphisms and strong stability of semidefinite and Lorentz complementarity problems. Math. Oper. Res.
**28**, 39–63 (2003)MathSciNetCrossRefGoogle Scholar - 21.Povh, J., Rendl, F.: Copositive and semidefinite relaxations of the quadratic assignment problem. Discrete Optim.
**6**, 231–241 (2009)MathSciNetCrossRefGoogle Scholar - 22.Qi, H.D.: Local duality of nonlinear semidefinite programming. Math. Oper. Res.
**34**, 124–141 (2009)MathSciNetCrossRefGoogle Scholar - 23.Qi, H.D., Sun, D.F.: A quadratically convergent Newton method for computing the nearest correlation matrix. SIAM J. Matrix Anal. Appl.
**28**, 360–385 (2006)MathSciNetCrossRefGoogle Scholar - 24.Rockafellar, R.T.: Conjugate Duality and Optimization, CBMS-NSF Regional Conf. Ser. Appl. Math. vol. 16. SIAM, Philadelphia (1974)Google Scholar
- 25.Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim.
**14**, 877–898 (1976)MathSciNetCrossRefGoogle Scholar - 26.Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res.
**1**, 97–116 (1976)MathSciNetCrossRefGoogle Scholar - 27.Sun, D.F., Sun, J.: Semismooth matrix-valued functions. Math. Oper. Res.
**27**, 150–169 (2002)MathSciNetCrossRefGoogle Scholar - 28.Sun, D.F., Toh, K.-C., Yang, L.: A convergent 3-block semiproximal alternating direction method of multipliers for conic programming with 4-type constraints. SIAM J. Optim.
**25**, 882–915 (2015)MathSciNetCrossRefGoogle Scholar - 29.Sun, D.F., Toh, K.-C., Yang, L.: An efficient inexact ABCD method for least squares semidefinite programming. SIAM J. Optim.
**26**, 1072–1100 (2016)MathSciNetCrossRefGoogle Scholar - 30.Sun, J., Zhang, S.: A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs. Eur. J. Oper. Res.
**207**, 1210–1220 (2010)MathSciNetCrossRefGoogle Scholar - 31.Toh, K.-C.: An inexact primal-dual path following algorithm for convex quadratic SDP. Math. Program.
**112**, 221–254 (2008)MathSciNetCrossRefGoogle Scholar - 32.Toh, K.-C., Tütüncü, R., Todd, M.: Inexact primal-dual path-following algorithms for a special class of convex quadratic SDP and related problems. Pac. J. Optim.
**3**, 135–164 (2007)MathSciNetzbMATHGoogle Scholar - 33.Yang, L., Sun, D.F., Toh, K.-C.: SDPNAL+: a majorized semismooth Newton-CG augmented Lagrangian method for semidefinite programming with nonnegative constraints. Math. Program. Comput.
**7**, 331–366 (2015)MathSciNetCrossRefGoogle Scholar - 34.Zhao, X.Y.: A Semismooth Newton-CG Augmented Lagrangian Method for Large Scale Linear and Convex Quadratic SDPs, PhD thesis, Department of Mathematics, National University of Singapore (2009)Google Scholar
- 35.Zhao, X.Y., Sun, D.F., Toh, K.-C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim.
**20**, 1737–1765 (2010)MathSciNetCrossRefGoogle Scholar