Abstract
In this paper, we give a new penalized semidefinite programming approach for non-convex quadratically-constrained quadratic programs (QCQPs). We incorporate penalty terms into the objective of convex relaxations in order to retrieve feasible and near-optimal solutions for non-convex QCQPs. We introduce a generalized linear independence constraint qualification (GLICQ) criterion and prove that any GLICQ regular point that is sufficiently close to the feasible set can be used to construct an appropriate penalty term and recover a feasible solution. Inspired by these results, we develop a heuristic sequential procedure that preserves feasibility and aims to improve the objective value at each iteration. Numerical experiments on large-scale system identification problems as well as benchmark instances from the library of quadratic programming demonstrate the ability of the proposed penalized semidefinite programs in finding near-optimal solutions for non-convex QCQP.
Similar content being viewed by others
References
Ahmadi, A.A., Majumdar, A.: DSOS and SDSOS optimization: more tractable alternatives to sum of squares and semidefinite optimization. SIAM J. Appl. Algebr. Geom. 3(2), 193–230 (2019)
Aittomaki, T., Koivunen, V.: Beam pattern optimization by minimization of quartic polynomial. In: 2009 IEEE/SP 15th Workshop on Statistical Signal Processing, pp. 437–440. IEEE (2009)
Alizadeh, F., Goldfarb, D.: Second-order cone programming. Math. Program. 95(1), 3–51 (2003)
ApS M: The MOSEK optimization toolbox for MATLAB manual. Version 8.1. http://docs.mosek.com/8.1/toolbox/index.html (2017)
Ashraphijuo, M., Madani, R., Lavaei, J.: Characterization of rank-constrained feasibility problems via a finite number of convex programs. In: 2016 IEEE 55th Conference on Decision and Control (CDC), pp. 6544–6550. IEEE(2016)
Atamtürk, A., Narayanan, V.: Cuts for conic mixed-integer programming. In: Fischetti, M., Williamson, D.P. (eds.) Integer Programming and Combinatorial Optimization, pp. 16–29. Springer, Heidelberg (2007)
Aubry, A., De Maio, A., Jiang, B., Zhang, S.: Ambiguity function shaping for cognitive radar via complex quartic optimization. IEEE Trans. Signal Process. 61(22), 5603–5619 (2013)
Bandeira, A.S., Boumal, N., Singer, A.: Tightness of the maximum likelihood semidefinite relaxation for angular synchronization. arXiv preprint arXiv:1411.3272 (2014)
Bao, X., Sahinidis, N.V., Tawarmalani, M.: Semidefinite relaxations for quadratically constrained quadratic programming: a review and comparisons. Math. Program. 129, 129–157 (2011)
Belotti, P.: COUENNE: A user’s manual. Technical report, Lehigh University, Tech. rep. (2013)
Bienstock, D., Munoz, G.: LP formulations for polynomial optimization problems. SIAM J. Optim. 28(2), 1121–1150 (2018)
Burer, S., Vandenbussche, D.: A finite branch-and-bound algorithm for nonconvex quadratic programming via semidefinite relaxations. Math. Program. 113(2), 259–282 (2008)
Burer, S., Ye, Y.: Exact semidefinite formulations for a class of (random and non-random) nonconvex quadratic programs. arXiv preprint arXiv:1802.02688 (2018)
Burgdorf, S., Laurent, M., Piovesan, T.: On the closure of the completely positive semidefinite cone and linear approximations to quantum colorings. arXiv preprint arXiv:1502.02842 (2015)
Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)
Candes, E.J., Strohmer, T., Voroninski, V.: Phaselift: exact and stable signal recovery from magnitude measurements via convex programming. Commun. Pure Appl. Math. 66(8), 1241–1274 (2013)
Candes, E.J., Eldar, Y.C., Strohmer, T., Voroninski, V.: Phase retrieval via matrix completion. SIAM Rev. 57(2), 225–251 (2015)
Chen, C., Atamtürk, A., Oren, S.S.: A spatial branch-and-cut method for nonconvex QCQP with bounded complex variables. Math. Program. 165(2), 549–577 (2017)
Chen, C.Y., Vaidyanathan, P.: Mimo radar waveform optimization with prior information of the extended target and clutter. IEEE Trans. Signal Process. 57(9), 3533–3544 (2009)
Chen, J., Burer, S.: Globally solving nonconvex quadratic programming problems via completely positive programming. Math. Program. Comput. 4(1), 33–52 (2012)
Cid, C., Murphy, S., Robshaw, M.: Computational and algebraic aspects of the advanced encryption standard. In: Proceedings of the Seventh International Workshop on Computer Algebra in Scientific Computing-CASC, vol. 2004 (2004)
Cid, C., Murphy, S., Robshaw, M.J. (2005) Small scale variants of the AES. In: International Workshop on Fast Software Encryption, pp. 145–162. Springer
Courtois, N.T., Pieprzyk, J.: Cryptanalysis of block ciphers with overdefined systems of equations. In: International Conference on the Theory and Application of Cryptology and Information Security, pp. 267–287. Springer (2002)
Deza, M., Laurent, M.: Applications of cut polyhedra–ii. J. Comput. Appl. Math. 55(2), 217–247 (1994)
Fattahi, S., Sojoudi, S.: Data-driven sparse system identification. In: 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE (2018)
Fattahi, S., Fazelnia, G., Lavaei, J., Arcak, M.: Transformation of optimal centralized controllers into near-globally optimal static distributed controllers. IEEE Trans. Autom. Control 64(1), 66–80 (2018)
Fazelnia, G., Madani, R., Kalbat, A., Lavaei, J.: Convex relaxation for optimal distributed control problems. IEEE Trans. Autom. Control 62(1), 206–221 (2017)
Fogel, F., Waldspurger, I., dAspremont, A.: Phase retrieval for imaging problems. In: Mathematical Programming Computation, pp. 1–25 (2013)
Furini, F., Traversi, E., Belotti, P., Frangioni, A., Gleixner, A., Gould, N., Liberti, L., Lodi, A., Misener, R., Mittelmann, H., Sahinidis, N., Vigerske, S., Wiegele, A.: QPLIB: a library of quadratic programming instances. Math. Program. Comput. 11, 237–310 (2019)
GAMS Development Corporation: General Algebraic Modeling System (GAMS) Release 24.2.1. Washington, DC, USA, http://www.gams.com/ (2013)
Gershman, A.B., Sidiropoulos, N.D., Shahbazpanahi, S., Bengtsson, M., Ottersten, B.: Convex optimization-based beamforming: from receive to transmit and network designs. IEEE Signal Process. Mag. 27(3), 62–75 (2010)
Goemans, M.X., Williamson, D.P.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM (JACM) 42(6), 1115–1145 (1995)
He, S., Luo, Z., Nie, J., Zhang, S.: Semidefinite relaxation bounds for indefinite homogeneous quadratic optimization. SIAM J. Optim. 19, 503–523 (2008)
He, S., Li, Z., Zhang, S.: Approximation algorithms for homogeneous polynomial optimization with quadratic constraints. Math. Program. 125, 353–383 (2010)
Hilling, J.J., Sudbery, A.: The geometric measure of multipartite entanglement and the singular values of a hypermatrix. J. Math. Phys. 51(7), 072102 (2010)
Ibaraki, S., Tomizuka, M.: Rank minimization approach for solving BMI problems with random search. In: Proceedings of the 2001 American Control Conference. (Cat. No. 01CH37148), vol. 3, pp. 1870–1875. IEEE (2001)
Josz, C., Molzahn, D.K.: Lasserre hierarchy for large scale polynomial optimization in real and complex variables. SIAM J. Optim. 28(2), 1017–1048 (2018)
Kheirandishfard, M., Zohrizadeh, F., Adil, M., Madani, R. (2018a). Convex relaxation of bilinear matrix inequalities part II: applications to optimal control synthesis. In: IEEE 57th Annual Conference on Decision and Control (CDC)
Kheirandishfard, M., Zohrizadeh, F., Adil, M., Madani, R.: Convex relaxation of bilinear matrix inequalities part ii: applications to optimal control synthesis. In: 2018 IEEE Conference on Decision and Control (CDC), pp. 75–82. IEEE (2018b)
Kheirandishfard, M., Zohrizadeh, F., Madani, R.: Convex relaxation of bilinear matrix inequalities part I: theoretical results. In: IEEE 57th Annual Conference on Decision and Control (CDC) (2018c)
Kim, S., Kojima, M.: Exact solutions of some nonconvex quadratic optimization problems via SDP and SOCP relaxations. Comput. Optim. Appl. 26(2), 143–154 (2003)
Kim, S., Kojima, M., Yamashita, M.: Second order cone programming relaxation of a positive semidefinite constraint. Optim. Methods Softw. 18, 535–541 (2003)
Lasserre, J.B.: An explicit exact SDP relaxation for nonlinear 0-1 programs. In: Integer Programming and Combinatorial Optimization, pp. 293–303. Springer (2001a)
Lasserre, J.B.: Global optimization with polynomials and the problem of moments. SIAM J. Optim. 11, 796–817 (2001b)
Lasserre, J.B.: Convergent SDP-relaxations in polynomial optimization with sparsity. SIAM J. Optim. 17, 822–843 (2006)
Laurent, M., Piovesan, T.: Conic approach to quantum graph parameters using linear optimization over the completely positive semidefinite cone. SIAM J. Optim. 25(4), 2461–2493 (2015)
Li, Z., He, S., Zhang, S.: Approximation Methods for Polynomial Optimization: Models, Algorithms, and Applications. Springer, Berlin (2012)
Lovász, L., Schrijver, A.: Cones of matrices and set-functions and 0–1 optimization. SIAM J. Optim. 1(2), 166–190 (1991)
Luo, Z., Sidiropoulos, N., Tseng, P., Zhang, S.: Approximation bounds for quadratic optimization with homogeneous quadratic constraints. SIAM J. Optim. 18, 1–28 (2007)
Luo, Z.Q., Wk, Ma., So, A.M.C., Ye, Y., Zhang, S.: Semidefinite relaxation of quadratic optimization problems. IEEE Signal Process. Mag. 27(3), 20 (2010)
Madani, R., Fazelnia, G., Lavaei, J.: Rank-2 matrix solution for semidefinite relaxations of arbitrary polynomial optimization problems. Preprint (2014)
Madani, R., Lavaei, J., Baldick, R.: Convexification of power flow problem over arbitrary networks. In: 2015 54th IEEE Conference on Decision and Control (CDC), pp. 1–8. IEEE (2015a)
Madani, R., Sojoudi, S., Lavaei, J.: Convex relaxation for optimal power flow problem: Mesh networks. IEEE Trans. Power Syst. 30(1), 199–211 (2015b)
Madani, R., Ashraphijuo, M., Lavaei, J.: Promises of conic relaxation for contingency-constrained optimal power flow problem. IEEE Trans. Power Syst. 31(2), 1297–1307 (2016)
Madani, R., Atamtürk, A., Davoudi, A.: A scalable semidefinite relaxation approach to grid scheduling. arXiv preprint arXiv:1707.03541 (2017a)
Madani, R., Sojoudi, S., Fazelnia, G., Lavaei, J.: Finding low-rank solutions of sparse linear matrix inequalities using convex optimization. SIAM J. Optim. 27(2), 725–758 (2017b)
Majumdar, A., Ahmadi, A.A., Tedrake, R.: Control and verification of high-dimensional systems with DSOS and SDSOS programming. In: 2014 IEEE 53rd Annual Conference on Decision and Control (CDC), pp. 394–401. IEEE (2014)
Mariere, B., Luo, Z.Q., Davidson, T.N.: Blind constant modulus equalization via convex optimization. IEEE Trans. Signal Process. 51(3), 805–818 (2003)
Mohammad-Nezhad, A., Terlaky, T.: A rounding procedure for semidefinite optimization. Oper. Res. Lett. 47(1), 59–65 (2019)
Mu, C., Zhang, Y., Wright, J., Goldfarb, D.: Scalable robust matrix recovery: Frank-Wolfe meets proximal methods. SIAM J. Sci. Comput. 38(5), A3291–A3317 (2016)
Muramatsu, M., Suzuki, T.: A new second-order cone programming relaxation for max-cut problems. J. Oper. Res. Soc. Jpn. 46, 164–177 (2003)
Murphy, S., Robshaw, M.J.: Essential algebraic structure within the AES. In: Annual International Cryptology Conference, pp. 1–16. Springer (2002)
Natarajan. K., Shi, D., Toh, K.C.: A penalized quadratic convex reformulation method for random quadratic unconstrained binary optimization. Optimization Online (2013)
Nesterov, Y.: Semidefinite relaxation and nonconvex quadratic optimization. Optim. Methods Softw. 9, 141–160 (1998)
Nesterov, Y., Nemirovskii, A.: Interior-point polynomial algorithms in convex programming (Vol. 13). SIAM (1994)
Papp, D., Alizadeh, F.: Semidefinite characterization of sum-of-squares cones in algebras. SIAM J. Optim. 23(3), 1398–1423 (2013)
Pereira, J., Ibrahimi, M., Montanari, A.: Learning networks of stochastic differential equations. In: Advances in Neural Information Processing Systems, pp. 172–180 (2010)
Permenter, F., Parrilo, P.: Partial facial reduction: simplified, equivalent SDPS via approximations of the psd cone. Mathematical Programming, pp. 1–54 (2014)
Rotkowitz, M., Lall, S.: A characterization of convex problems in decentralized control. IEEE Trans. Autom. Control 50(12), 1984–1996 (2005)
Sarkar, T., Rakhlin, A.: How fast can linear dynamical systems be learned? arXiv preprint arXiv:1812.01251 (2018)
Sherali, H.D., Adams, W.P.: A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems. SIAM J. Discrete Math. 3(3), 411–430 (1990)
Sherali, H.D., Adams, W.P.: A reformulation-linearization technique for solving discrete and continuous nonconvex problems, vol. 31. Springer, Berlin (2013)
Singer, A.: Angular synchronization by eigenvectors and semidefinite programming. Appl. Comput. Harmonic Anal. 30(1), 20–36 (2011)
Sojoudi, S., Lavaei, J.: On the exactness of semidefinite relaxation for nonlinear optimization over graphs: Part I. In: 2013 IEEE 52nd Annual Conference on Decision and Control (CDC), pp. 1043–1050. IEEE (2013a)
Sojoudi, S., Lavaei, J.: On the exactness of semidefinite relaxation for nonlinear optimization over graphs: part II. In: 2013 IEEE 52nd Annual Conference on Decision and Control (CDC), pp. 1043–1050. IEEE (2013b)
Tawarmalani, M., Sahinidis, N.V.: A polyhedral branch-and-cut approach to global optimization. Math. Program. 103, 225–249 (2005)
Toker, O., Ozbay, H.: On the complexity of purely complex \(\mu \) computation and related problems in multidimensional systems. IEEE Trans. Autom. Control 43(3), 409–414 (1998)
Wang, Y.S., Matni, N., Doyle, J.C.: Separable and localized system-level synthesis for large-scale systems. IEEE Trans. Autom. Control 63(12), 4234–4249 (2018)
Wang, Y.S., Matni, N., Doyle, J.C.: A system-level approach to controller synthesis. IEEE Trans. Autom. Control 64(10), 4079–4093 (2019). https://doi.org/10.1109/TAC.2018.2890753
Ye, Y.: Approximating global quadratic optimization with convex quadratic constraints. J. Global Optim. 15, 1–17 (1999a)
Ye, Y.: Approximating quadratic programming with bound and quadratic constraints. Math. Program. 84, 219–226 (1999b)
Zhang, S.: Quadratic maximization and semidefinite relaxation. Math. Program. 87, 453–465 (2000)
Zhang, S., Huang, Y.: Complex quadratic optimization and semidefinite programming. SIAM J. Optim. 87, 871–890 (2006)
Zohrizadeh, F., Kheirandishfard, M., Nasir, A., Madani, R.: Sequential relaxation of unit commitment with AC transmission constraints. In: IEEE 57th Annual Conference on Decision and Control (CDC) (2018a)
Zohrizadeh, F., Kheirandishfard, M., Quarm, E., Madani, R.: Penalized parabolic relaxation for optimal power flow problem. In: IEEE 57th Annual Conference on Decision and Control (CDC) (2018b)
Acknowledgements
The authors are grateful to GAMS Development Corporation for providing them with unrestricted access to a full set of solvers throughout the project.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work is in part supported by the NSF Award 1809454. Javad Lavaei is supported by an AFOSR YIP Award and ONR N000141712933. Alper Atamtürk is supported, in part, by Grant FA9550-10-1-0168 from the Office of the Assistant Secretary of Defense for Research & Engineering, NSF Award 1807260, DOE ARPA-E Grant 260801540061, and DOD ONR Grant 12951270.
Appendices
Application to polynomial optimization
In this section, we show that the proposed penalized SDP approach can be used for polynomial optimization as well. A polynomial optimization problem is formulated as
for every \(k\in \{0\}\cup \mathcal {I}\cup \mathcal {E}\), where each function \(u_k:\mathbb {R}^n\rightarrow \mathbb {R}\) is a polynomial of arbitrary degree. Problem (48a)–(48c) can be reformulated as a QCQP of the form:
where \(\varvec{y}\in \mathbb {R}^{|\mathcal {O}|}\) is an auxiliary variable, and \(v_1,\ldots ,v_{|\mathcal {O}|}\) and \(w_0,w_1,\ldots ,w_{|\{0\}\cup \mathcal {I}\cup \mathcal {E}|}\) are quadratic functions with the following properties:
-
For every \(\varvec{x}\in \mathbb {R}^n\), the function \(\varvec{v}(\varvec{x},\cdot ):\mathbb {R}^{|\mathcal {O}|}\rightarrow \mathbb {R}^{|\mathcal {O}|}\) is invertible,
-
If \(\varvec{v}(\varvec{x},\varvec{y})=\varvec{0}_n\), then \(w_k(\varvec{x},\varvec{y}) = u_k(\varvec{x})\) for every \(k\in \{0\}\cup \mathcal {I}\cup \mathcal {E}\).
Based on the above properties, there is a one-to-one correspondence between the feasible sets of (48a)–(48c) and (49a)–(49d). Moreover, a feasible point is an optimal solution to the QCQP (49a)–(49d) if and only if is an optimal solution to the polynomial optimization problem (48a)–(48c).
Theorem 3
[51] Suppose that \(\{u_k\}_{k\in \{0\}\cup \mathcal {I}\cup \mathcal {E}}\) are polynomials of degree at most d, consisting of m monomials in total. There exists a QCQP reformulation of the polynomial optimization (48a)–(48c) in the form of (49a)–(49d), where \({|\mathcal {O}|}\le m n \left( \lfloor \log _2(d)\rfloor +1\right) \).
The next proposition shows that the LICQ regularity of a point \(\hat{\varvec{x}}\in \mathbb {R}^n\) is inherited by the corresponding point \((\hat{\varvec{x}},\hat{\varvec{y}})\in \mathbb {R}^n\times \mathbb {R}^o\) of the QCQP reformulation (49a)–(49d).
Proposition 1
Consider a pair of vectors \(\hat{\varvec{x}}\in \mathbb {R}^n\) and \(\hat{\varvec{y}}\in \mathbb {R}^{|\mathcal {O}|}\) satisfying \(\varvec{v}(\hat{\varvec{x}},\hat{\varvec{y}})=\varvec{0}_n\). The following two statements are equivalent:
-
1.
\(\hat{\varvec{x}}\) is feasible and satisfies the LICQ condition for the polynomial optimization problem (48a)–(48b).
-
2.
\((\hat{\varvec{x}},\hat{\varvec{y}})\) is feasible and satisfies the LICQ condition for the QCQP (49a)–(49d).
Proof
From \(\varvec{u}(\hat{\varvec{x}})=\varvec{w}(\hat{\varvec{x}},\hat{\varvec{y}})\) and the invertiblity assumption for \(\varvec{v}(\hat{\varvec{x}},\cdot )\), we have
Therefore, \(\mathcal {J}_{\mathrm {PO}}(\hat{\varvec{x}})=\frac{\partial \varvec{u}(\hat{\varvec{x}})}{\partial \varvec{x}}\) is equal to the Schur complement of
which is the Jacobian matrix of the QCQP (49a)–(49d) at the point \((\hat{\varvec{x}},\hat{\varvec{y}})\). As a result, the matrix \(\mathcal {J}_{\mathrm {PO}}(\hat{\varvec{x}})\) is singular if and only if \(\mathcal {J}_{\mathrm {QCQP}}(\hat{\varvec{x}},\hat{\varvec{y}})\) is singular. \(\square \)
Reformulation-linearization technique
This appendix covers the reformulation-linearization technique (RLT) of Sherali and Adams [72] as an approach to strengthen convex relaxations of the form (4a)–(4d) in the presence of affine constraints. Define \(\mathcal {L}\) as the set of affine constrains in the QCQP (1a)–(1c), i.e., \(\mathcal {L}\triangleq \{k\in \mathcal {I}\cup \mathcal {E}\;|\;\varvec{A}_k=\varvec{0}_{n\times n}\}\). Define also
where \(\varvec{B}\triangleq [\varvec{b}_1,\ldots ,\varvec{b}_{|\mathcal {I}\cap \mathcal {E}|}]^{\top }\) and \(\varvec{c}\triangleq [c_1,\ldots ,c_{|\mathcal {I}\cap \mathcal {E}|}]^{\top }\). Every \(\varvec{x}\in \mathcal {F}\) satisfies
and, as a result, all elements of the matrix
are non-negative if \(\varvec{x}\) is feasible. Hence, the inequality
holds true for every \(\varvec{x}\in \mathcal {F}\) and \((i,j)\in \mathcal {H}\times \mathcal {H}\), where \(\varvec{V}:\mathbb {R}^n\times \mathbb {S}_n\rightarrow \mathbb {S}_{|\mathcal {H}|}\) is defined as
\(\mathcal {H}\triangleq \{1,\ldots ,|\mathcal {L}\cap \mathcal {I}|+2|\mathcal {L}\cap \mathcal {E}|\}\), and \(\varvec{e}_1,\ldots ,\varvec{e}_{|\mathcal {H}|}\) denote the standard bases in \(\mathbb {R}^{|\mathcal {H}|}\).
This leads to a strengthened relaxation of QCQP (1a)–(1c):
where \(\mathcal {V}\subseteq \mathcal {H}\times \mathcal {H}\) is a selection of RLT inequalities.
Rights and permissions
About this article
Cite this article
Madani, R., Kheirandishfard, M., Lavaei, J. et al. Penalized semidefinite programming for quadratically-constrained quadratic optimization. J Glob Optim 78, 423–451 (2020). https://doi.org/10.1007/s10898-020-00918-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-020-00918-8