Penalized semidefinite programming for quadratically-constrained quadratic optimization

Abstract

In this paper, we give a new penalized semidefinite programming approach for non-convex quadratically-constrained quadratic programs (QCQPs). We incorporate penalty terms into the objective of convex relaxations in order to retrieve feasible and near-optimal solutions for non-convex QCQPs. We introduce a generalized linear independence constraint qualification (GLICQ) criterion and prove that any GLICQ regular point that is sufficiently close to the feasible set can be used to construct an appropriate penalty term and recover a feasible solution. Inspired by these results, we develop a heuristic sequential procedure that preserves feasibility and aims to improve the objective value at each iteration. Numerical experiments on large-scale system identification problems as well as benchmark instances from the library of quadratic programming demonstrate the ability of the proposed penalized semidefinite programs in finding near-optimal solutions for non-convex QCQP.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    Ahmadi, A.A., Majumdar, A.: DSOS and SDSOS optimization: more tractable alternatives to sum of squares and semidefinite optimization. SIAM J. Appl. Algebr. Geom. 3(2), 193–230 (2019)

    MathSciNet  MATH  Google Scholar 

  2. 2.

    Aittomaki, T., Koivunen, V.: Beam pattern optimization by minimization of quartic polynomial. In: 2009 IEEE/SP 15th Workshop on Statistical Signal Processing, pp. 437–440. IEEE (2009)

  3. 3.

    Alizadeh, F., Goldfarb, D.: Second-order cone programming. Math. Program. 95(1), 3–51 (2003)

    MathSciNet  MATH  Google Scholar 

  4. 4.

    ApS M: The MOSEK optimization toolbox for MATLAB manual. Version 8.1. http://docs.mosek.com/8.1/toolbox/index.html (2017)

  5. 5.

    Ashraphijuo, M., Madani, R., Lavaei, J.: Characterization of rank-constrained feasibility problems via a finite number of convex programs. In: 2016 IEEE 55th Conference on Decision and Control (CDC), pp. 6544–6550. IEEE(2016)

  6. 6.

    Atamtürk, A., Narayanan, V.: Cuts for conic mixed-integer programming. In: Fischetti, M., Williamson, D.P. (eds.) Integer Programming and Combinatorial Optimization, pp. 16–29. Springer, Heidelberg (2007)

    Google Scholar 

  7. 7.

    Aubry, A., De Maio, A., Jiang, B., Zhang, S.: Ambiguity function shaping for cognitive radar via complex quartic optimization. IEEE Trans. Signal Process. 61(22), 5603–5619 (2013)

    MathSciNet  MATH  Google Scholar 

  8. 8.

    Bandeira, A.S., Boumal, N., Singer, A.: Tightness of the maximum likelihood semidefinite relaxation for angular synchronization. arXiv preprint arXiv:1411.3272 (2014)

  9. 9.

    Bao, X., Sahinidis, N.V., Tawarmalani, M.: Semidefinite relaxations for quadratically constrained quadratic programming: a review and comparisons. Math. Program. 129, 129–157 (2011)

    MathSciNet  MATH  Google Scholar 

  10. 10.

    Belotti, P.: COUENNE: A user’s manual. Technical report, Lehigh University, Tech. rep. (2013)

    Google Scholar 

  11. 11.

    Bienstock, D., Munoz, G.: LP formulations for polynomial optimization problems. SIAM J. Optim. 28(2), 1121–1150 (2018)

    MathSciNet  MATH  Google Scholar 

  12. 12.

    Burer, S., Vandenbussche, D.: A finite branch-and-bound algorithm for nonconvex quadratic programming via semidefinite relaxations. Math. Program. 113(2), 259–282 (2008)

    MathSciNet  MATH  Google Scholar 

  13. 13.

    Burer, S., Ye, Y.: Exact semidefinite formulations for a class of (random and non-random) nonconvex quadratic programs. arXiv preprint arXiv:1802.02688 (2018)

  14. 14.

    Burgdorf, S., Laurent, M., Piovesan, T.: On the closure of the completely positive semidefinite cone and linear approximations to quantum colorings. arXiv preprint arXiv:1502.02842 (2015)

  15. 15.

    Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)

    MathSciNet  MATH  Google Scholar 

  16. 16.

    Candes, E.J., Strohmer, T., Voroninski, V.: Phaselift: exact and stable signal recovery from magnitude measurements via convex programming. Commun. Pure Appl. Math. 66(8), 1241–1274 (2013)

    MathSciNet  MATH  Google Scholar 

  17. 17.

    Candes, E.J., Eldar, Y.C., Strohmer, T., Voroninski, V.: Phase retrieval via matrix completion. SIAM Rev. 57(2), 225–251 (2015)

    MathSciNet  MATH  Google Scholar 

  18. 18.

    Chen, C., Atamtürk, A., Oren, S.S.: A spatial branch-and-cut method for nonconvex QCQP with bounded complex variables. Math. Program. 165(2), 549–577 (2017)

    MathSciNet  MATH  Google Scholar 

  19. 19.

    Chen, C.Y., Vaidyanathan, P.: Mimo radar waveform optimization with prior information of the extended target and clutter. IEEE Trans. Signal Process. 57(9), 3533–3544 (2009)

    MathSciNet  MATH  Google Scholar 

  20. 20.

    Chen, J., Burer, S.: Globally solving nonconvex quadratic programming problems via completely positive programming. Math. Program. Comput. 4(1), 33–52 (2012)

    MathSciNet  MATH  Google Scholar 

  21. 21.

    Cid, C., Murphy, S., Robshaw, M.: Computational and algebraic aspects of the advanced encryption standard. In: Proceedings of the Seventh International Workshop on Computer Algebra in Scientific Computing-CASC, vol. 2004 (2004)

  22. 22.

    Cid, C., Murphy, S., Robshaw, M.J. (2005) Small scale variants of the AES. In: International Workshop on Fast Software Encryption, pp. 145–162. Springer

  23. 23.

    Courtois, N.T., Pieprzyk, J.: Cryptanalysis of block ciphers with overdefined systems of equations. In: International Conference on the Theory and Application of Cryptology and Information Security, pp. 267–287. Springer (2002)

  24. 24.

    Deza, M., Laurent, M.: Applications of cut polyhedra–ii. J. Comput. Appl. Math. 55(2), 217–247 (1994)

    MathSciNet  MATH  Google Scholar 

  25. 25.

    Fattahi, S., Sojoudi, S.: Data-driven sparse system identification. In: 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE (2018)

  26. 26.

    Fattahi, S., Fazelnia, G., Lavaei, J., Arcak, M.: Transformation of optimal centralized controllers into near-globally optimal static distributed controllers. IEEE Trans. Autom. Control 64(1), 66–80 (2018)

    MathSciNet  MATH  Google Scholar 

  27. 27.

    Fazelnia, G., Madani, R., Kalbat, A., Lavaei, J.: Convex relaxation for optimal distributed control problems. IEEE Trans. Autom. Control 62(1), 206–221 (2017)

    MathSciNet  MATH  Google Scholar 

  28. 28.

    Fogel, F., Waldspurger, I., dAspremont, A.: Phase retrieval for imaging problems. In: Mathematical Programming Computation, pp. 1–25 (2013)

  29. 29.

    Furini, F., Traversi, E., Belotti, P., Frangioni, A., Gleixner, A., Gould, N., Liberti, L., Lodi, A., Misener, R., Mittelmann, H., Sahinidis, N., Vigerske, S., Wiegele, A.: QPLIB: a library of quadratic programming instances. Math. Program. Comput. 11, 237–310 (2019)

    MathSciNet  MATH  Google Scholar 

  30. 30.

    GAMS Development Corporation: General Algebraic Modeling System (GAMS) Release 24.2.1. Washington, DC, USA, http://www.gams.com/ (2013)

  31. 31.

    Gershman, A.B., Sidiropoulos, N.D., Shahbazpanahi, S., Bengtsson, M., Ottersten, B.: Convex optimization-based beamforming: from receive to transmit and network designs. IEEE Signal Process. Mag. 27(3), 62–75 (2010)

    Google Scholar 

  32. 32.

    Goemans, M.X., Williamson, D.P.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM (JACM) 42(6), 1115–1145 (1995)

    MathSciNet  MATH  Google Scholar 

  33. 33.

    He, S., Luo, Z., Nie, J., Zhang, S.: Semidefinite relaxation bounds for indefinite homogeneous quadratic optimization. SIAM J. Optim. 19, 503–523 (2008)

    MathSciNet  MATH  Google Scholar 

  34. 34.

    He, S., Li, Z., Zhang, S.: Approximation algorithms for homogeneous polynomial optimization with quadratic constraints. Math. Program. 125, 353–383 (2010)

    MathSciNet  MATH  Google Scholar 

  35. 35.

    Hilling, J.J., Sudbery, A.: The geometric measure of multipartite entanglement and the singular values of a hypermatrix. J. Math. Phys. 51(7), 072102 (2010)

    MathSciNet  MATH  Google Scholar 

  36. 36.

    Ibaraki, S., Tomizuka, M.: Rank minimization approach for solving BMI problems with random search. In: Proceedings of the 2001 American Control Conference. (Cat. No. 01CH37148), vol. 3, pp. 1870–1875. IEEE (2001)

  37. 37.

    Josz, C., Molzahn, D.K.: Lasserre hierarchy for large scale polynomial optimization in real and complex variables. SIAM J. Optim. 28(2), 1017–1048 (2018)

    MathSciNet  MATH  Google Scholar 

  38. 38.

    Kheirandishfard, M., Zohrizadeh, F., Adil, M., Madani, R. (2018a). Convex relaxation of bilinear matrix inequalities part II: applications to optimal control synthesis. In: IEEE 57th Annual Conference on Decision and Control (CDC)

  39. 39.

    Kheirandishfard, M., Zohrizadeh, F., Adil, M., Madani, R.: Convex relaxation of bilinear matrix inequalities part ii: applications to optimal control synthesis. In: 2018 IEEE Conference on Decision and Control (CDC), pp. 75–82. IEEE (2018b)

  40. 40.

    Kheirandishfard, M., Zohrizadeh, F., Madani, R.: Convex relaxation of bilinear matrix inequalities part I: theoretical results. In: IEEE 57th Annual Conference on Decision and Control (CDC) (2018c)

  41. 41.

    Kim, S., Kojima, M.: Exact solutions of some nonconvex quadratic optimization problems via SDP and SOCP relaxations. Comput. Optim. Appl. 26(2), 143–154 (2003)

    MathSciNet  MATH  Google Scholar 

  42. 42.

    Kim, S., Kojima, M., Yamashita, M.: Second order cone programming relaxation of a positive semidefinite constraint. Optim. Methods Softw. 18, 535–541 (2003)

    MathSciNet  MATH  Google Scholar 

  43. 43.

    Lasserre, J.B.: An explicit exact SDP relaxation for nonlinear 0-1 programs. In: Integer Programming and Combinatorial Optimization, pp. 293–303. Springer (2001a)

  44. 44.

    Lasserre, J.B.: Global optimization with polynomials and the problem of moments. SIAM J. Optim. 11, 796–817 (2001b)

    MathSciNet  MATH  Google Scholar 

  45. 45.

    Lasserre, J.B.: Convergent SDP-relaxations in polynomial optimization with sparsity. SIAM J. Optim. 17, 822–843 (2006)

    MathSciNet  MATH  Google Scholar 

  46. 46.

    Laurent, M., Piovesan, T.: Conic approach to quantum graph parameters using linear optimization over the completely positive semidefinite cone. SIAM J. Optim. 25(4), 2461–2493 (2015)

    MathSciNet  MATH  Google Scholar 

  47. 47.

    Li, Z., He, S., Zhang, S.: Approximation Methods for Polynomial Optimization: Models, Algorithms, and Applications. Springer, Berlin (2012)

    Google Scholar 

  48. 48.

    Lovász, L., Schrijver, A.: Cones of matrices and set-functions and 0–1 optimization. SIAM J. Optim. 1(2), 166–190 (1991)

    MathSciNet  MATH  Google Scholar 

  49. 49.

    Luo, Z., Sidiropoulos, N., Tseng, P., Zhang, S.: Approximation bounds for quadratic optimization with homogeneous quadratic constraints. SIAM J. Optim. 18, 1–28 (2007)

    MathSciNet  MATH  Google Scholar 

  50. 50.

    Luo, Z.Q., Wk, Ma., So, A.M.C., Ye, Y., Zhang, S.: Semidefinite relaxation of quadratic optimization problems. IEEE Signal Process. Mag. 27(3), 20 (2010)

    Google Scholar 

  51. 51.

    Madani, R., Fazelnia, G., Lavaei, J.: Rank-2 matrix solution for semidefinite relaxations of arbitrary polynomial optimization problems. Preprint (2014)

  52. 52.

    Madani, R., Lavaei, J., Baldick, R.: Convexification of power flow problem over arbitrary networks. In: 2015 54th IEEE Conference on Decision and Control (CDC), pp. 1–8. IEEE (2015a)

  53. 53.

    Madani, R., Sojoudi, S., Lavaei, J.: Convex relaxation for optimal power flow problem: Mesh networks. IEEE Trans. Power Syst. 30(1), 199–211 (2015b)

    Google Scholar 

  54. 54.

    Madani, R., Ashraphijuo, M., Lavaei, J.: Promises of conic relaxation for contingency-constrained optimal power flow problem. IEEE Trans. Power Syst. 31(2), 1297–1307 (2016)

    Google Scholar 

  55. 55.

    Madani, R., Atamtürk, A., Davoudi, A.: A scalable semidefinite relaxation approach to grid scheduling. arXiv preprint arXiv:1707.03541 (2017a)

  56. 56.

    Madani, R., Sojoudi, S., Fazelnia, G., Lavaei, J.: Finding low-rank solutions of sparse linear matrix inequalities using convex optimization. SIAM J. Optim. 27(2), 725–758 (2017b)

    MathSciNet  MATH  Google Scholar 

  57. 57.

    Majumdar, A., Ahmadi, A.A., Tedrake, R.: Control and verification of high-dimensional systems with DSOS and SDSOS programming. In: 2014 IEEE 53rd Annual Conference on Decision and Control (CDC), pp. 394–401. IEEE (2014)

  58. 58.

    Mariere, B., Luo, Z.Q., Davidson, T.N.: Blind constant modulus equalization via convex optimization. IEEE Trans. Signal Process. 51(3), 805–818 (2003)

    MathSciNet  MATH  Google Scholar 

  59. 59.

    Mohammad-Nezhad, A., Terlaky, T.: A rounding procedure for semidefinite optimization. Oper. Res. Lett. 47(1), 59–65 (2019)

    MathSciNet  MATH  Google Scholar 

  60. 60.

    Mu, C., Zhang, Y., Wright, J., Goldfarb, D.: Scalable robust matrix recovery: Frank-Wolfe meets proximal methods. SIAM J. Sci. Comput. 38(5), A3291–A3317 (2016)

    MathSciNet  MATH  Google Scholar 

  61. 61.

    Muramatsu, M., Suzuki, T.: A new second-order cone programming relaxation for max-cut problems. J. Oper. Res. Soc. Jpn. 46, 164–177 (2003)

    MathSciNet  MATH  Google Scholar 

  62. 62.

    Murphy, S., Robshaw, M.J.: Essential algebraic structure within the AES. In: Annual International Cryptology Conference, pp. 1–16. Springer (2002)

  63. 63.

    Natarajan. K., Shi, D., Toh, K.C.: A penalized quadratic convex reformulation method for random quadratic unconstrained binary optimization. Optimization Online (2013)

  64. 64.

    Nesterov, Y.: Semidefinite relaxation and nonconvex quadratic optimization. Optim. Methods Softw. 9, 141–160 (1998)

    MathSciNet  MATH  Google Scholar 

  65. 65.

    Nesterov, Y., Nemirovskii, A.: Interior-point polynomial algorithms in convex programming (Vol. 13). SIAM (1994)

  66. 66.

    Papp, D., Alizadeh, F.: Semidefinite characterization of sum-of-squares cones in algebras. SIAM J. Optim. 23(3), 1398–1423 (2013)

    MathSciNet  MATH  Google Scholar 

  67. 67.

    Pereira, J., Ibrahimi, M., Montanari, A.: Learning networks of stochastic differential equations. In: Advances in Neural Information Processing Systems, pp. 172–180 (2010)

  68. 68.

    Permenter, F., Parrilo, P.: Partial facial reduction: simplified, equivalent SDPS via approximations of the psd cone. Mathematical Programming, pp. 1–54 (2014)

  69. 69.

    Rotkowitz, M., Lall, S.: A characterization of convex problems in decentralized control. IEEE Trans. Autom. Control 50(12), 1984–1996 (2005)

    MathSciNet  MATH  Google Scholar 

  70. 70.

    Sarkar, T., Rakhlin, A.: How fast can linear dynamical systems be learned? arXiv preprint arXiv:1812.01251 (2018)

  71. 71.

    Sherali, H.D., Adams, W.P.: A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems. SIAM J. Discrete Math. 3(3), 411–430 (1990)

    MathSciNet  MATH  Google Scholar 

  72. 72.

    Sherali, H.D., Adams, W.P.: A reformulation-linearization technique for solving discrete and continuous nonconvex problems, vol. 31. Springer, Berlin (2013)

  73. 73.

    Singer, A.: Angular synchronization by eigenvectors and semidefinite programming. Appl. Comput. Harmonic Anal. 30(1), 20–36 (2011)

    MathSciNet  MATH  Google Scholar 

  74. 74.

    Sojoudi, S., Lavaei, J.: On the exactness of semidefinite relaxation for nonlinear optimization over graphs: Part I. In: 2013 IEEE 52nd Annual Conference on Decision and Control (CDC), pp. 1043–1050. IEEE (2013a)

  75. 75.

    Sojoudi, S., Lavaei, J.: On the exactness of semidefinite relaxation for nonlinear optimization over graphs: part II. In: 2013 IEEE 52nd Annual Conference on Decision and Control (CDC), pp. 1043–1050. IEEE (2013b)

  76. 76.

    Tawarmalani, M., Sahinidis, N.V.: A polyhedral branch-and-cut approach to global optimization. Math. Program. 103, 225–249 (2005)

    MathSciNet  MATH  Google Scholar 

  77. 77.

    Toker, O., Ozbay, H.: On the complexity of purely complex \(\mu \) computation and related problems in multidimensional systems. IEEE Trans. Autom. Control 43(3), 409–414 (1998)

    MathSciNet  MATH  Google Scholar 

  78. 78.

    Wang, Y.S., Matni, N., Doyle, J.C.: Separable and localized system-level synthesis for large-scale systems. IEEE Trans. Autom. Control 63(12), 4234–4249 (2018)

    MathSciNet  MATH  Google Scholar 

  79. 79.

    Wang, Y.S., Matni, N., Doyle, J.C.: A system-level approach to controller synthesis. IEEE Trans. Autom. Control 64(10), 4079–4093 (2019). https://doi.org/10.1109/TAC.2018.2890753

    MathSciNet  Article  MATH  Google Scholar 

  80. 80.

    Ye, Y.: Approximating global quadratic optimization with convex quadratic constraints. J. Global Optim. 15, 1–17 (1999a)

    MathSciNet  MATH  Google Scholar 

  81. 81.

    Ye, Y.: Approximating quadratic programming with bound and quadratic constraints. Math. Program. 84, 219–226 (1999b)

    MathSciNet  MATH  Google Scholar 

  82. 82.

    Zhang, S.: Quadratic maximization and semidefinite relaxation. Math. Program. 87, 453–465 (2000)

    MathSciNet  MATH  Google Scholar 

  83. 83.

    Zhang, S., Huang, Y.: Complex quadratic optimization and semidefinite programming. SIAM J. Optim. 87, 871–890 (2006)

    MathSciNet  MATH  Google Scholar 

  84. 84.

    Zohrizadeh, F., Kheirandishfard, M., Nasir, A., Madani, R.: Sequential relaxation of unit commitment with AC transmission constraints. In: IEEE 57th Annual Conference on Decision and Control (CDC) (2018a)

  85. 85.

    Zohrizadeh, F., Kheirandishfard, M., Quarm, E., Madani, R.: Penalized parabolic relaxation for optimal power flow problem. In: IEEE 57th Annual Conference on Decision and Control (CDC) (2018b)

Download references

Acknowledgements

The authors are grateful to GAMS Development Corporation for providing them with unrestricted access to a full set of solvers throughout the project.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Alper Atamtürk.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is in part supported by the NSF Award 1809454. Javad Lavaei is supported by an AFOSR YIP Award and ONR N000141712933. Alper Atamtürk is supported, in part, by Grant FA9550-10-1-0168 from the Office of the Assistant Secretary of Defense for Research & Engineering, NSF Award 1807260, DOE ARPA-E Grant 260801540061, and DOD ONR Grant 12951270.

Appendices

Application to polynomial optimization

In this section, we show that the proposed penalized SDP approach can be used for polynomial optimization as well. A polynomial optimization problem is formulated as

$$\begin{aligned} \underset{\varvec{x}\in \mathbb {R}^n}{\text {minimize}} \ \&u_0(\varvec{x})&\end{aligned}$$
(48a)
$$\begin{aligned} \ \ \ \text {s.t.} \ \ \&u_k(\varvec{x})\le 0, \quad \quad k\in \mathcal {I}\end{aligned}$$
(48b)
$$\begin{aligned}&u_k(\varvec{x})=0, \quad \quad k\in \mathcal {E}, \end{aligned}$$
(48c)

for every \(k\in \{0\}\cup \mathcal {I}\cup \mathcal {E}\), where each function \(u_k:\mathbb {R}^n\rightarrow \mathbb {R}\) is a polynomial of arbitrary degree. Problem (48a)–(48c) can be reformulated as a QCQP of the form:

(49a)
(49b)
(49c)
(49d)

where \(\varvec{y}\in \mathbb {R}^{|\mathcal {O}|}\) is an auxiliary variable, and \(v_1,\ldots ,v_{|\mathcal {O}|}\) and \(w_0,w_1,\ldots ,w_{|\{0\}\cup \mathcal {I}\cup \mathcal {E}|}\) are quadratic functions with the following properties:

  • For every \(\varvec{x}\in \mathbb {R}^n\), the function \(\varvec{v}(\varvec{x},\cdot ):\mathbb {R}^{|\mathcal {O}|}\rightarrow \mathbb {R}^{|\mathcal {O}|}\) is invertible,

  • If \(\varvec{v}(\varvec{x},\varvec{y})=\varvec{0}_n\), then \(w_k(\varvec{x},\varvec{y}) = u_k(\varvec{x})\) for every \(k\in \{0\}\cup \mathcal {I}\cup \mathcal {E}\).

Based on the above properties, there is a one-to-one correspondence between the feasible sets of (48a)–(48c) and (49a)–(49d). Moreover, a feasible point is an optimal solution to the QCQP (49a)–(49d) if and only if is an optimal solution to the polynomial optimization problem (48a)–(48c).

Theorem 3

[51] Suppose that \(\{u_k\}_{k\in \{0\}\cup \mathcal {I}\cup \mathcal {E}}\) are polynomials of degree at most d, consisting of m monomials in total. There exists a QCQP reformulation of the polynomial optimization (48a)–(48c) in the form of (49a)–(49d), where \({|\mathcal {O}|}\le m n \left( \lfloor \log _2(d)\rfloor +1\right) \).

The next proposition shows that the LICQ regularity of a point \(\hat{\varvec{x}}\in \mathbb {R}^n\) is inherited by the corresponding point \((\hat{\varvec{x}},\hat{\varvec{y}})\in \mathbb {R}^n\times \mathbb {R}^o\) of the QCQP reformulation (49a)–(49d).

Proposition 1

Consider a pair of vectors \(\hat{\varvec{x}}\in \mathbb {R}^n\) and \(\hat{\varvec{y}}\in \mathbb {R}^{|\mathcal {O}|}\) satisfying \(\varvec{v}(\hat{\varvec{x}},\hat{\varvec{y}})=\varvec{0}_n\). The following two statements are equivalent:

  1. 1.

    \(\hat{\varvec{x}}\) is feasible and satisfies the LICQ condition for the polynomial optimization problem (48a)–(48b).

  2. 2.

    \((\hat{\varvec{x}},\hat{\varvec{y}})\) is feasible and satisfies the LICQ condition for the QCQP (49a)–(49d).

Proof

From \(\varvec{u}(\hat{\varvec{x}})=\varvec{w}(\hat{\varvec{x}},\hat{\varvec{y}})\) and the invertiblity assumption for \(\varvec{v}(\hat{\varvec{x}},\cdot )\), we have

$$\begin{aligned} \frac{\partial \varvec{u}(\hat{\varvec{x}})}{\partial \varvec{x}}&= \begin{bmatrix} \frac{\partial \varvec{w}(\hat{\varvec{x}},\hat{\varvec{y}})}{\partial \varvec{x}}&\frac{\partial \varvec{w}(\hat{\varvec{x}},\hat{\varvec{y}})}{\partial \varvec{y}} \end{bmatrix} \begin{bmatrix} \varvec{I}&- \left( \frac{\partial \varvec{v}(\hat{\varvec{x}},\hat{\varvec{y}})}{\partial \varvec{y}} \right) ^{ -1} \frac{\partial \varvec{v}(\hat{\varvec{x}},\hat{\varvec{y}})}{\partial \varvec{x}} \end{bmatrix}^{ \top }\nonumber \\&= \frac{\partial \varvec{w}(\hat{\varvec{x}},\hat{\varvec{y}})}{\partial \varvec{x}} - \frac{\partial \varvec{w}(\hat{\varvec{x}},\hat{\varvec{y}})}{\partial \varvec{y}} \left( \frac{\partial \varvec{v}(\hat{\varvec{x}},\hat{\varvec{y}})}{\partial \varvec{y}} \right) ^{ -1} \frac{\partial \varvec{v}(\hat{\varvec{x}},\hat{\varvec{y}})}{\partial \varvec{x}}. \end{aligned}$$
(50)

Therefore, \(\mathcal {J}_{\mathrm {PO}}(\hat{\varvec{x}})=\frac{\partial \varvec{u}(\hat{\varvec{x}})}{\partial \varvec{x}}\) is equal to the Schur complement of

$$\begin{aligned} \mathcal {J}_{\mathrm {QCQP}}(\hat{\varvec{x}},\hat{\varvec{y}})= \begin{bmatrix} \frac{\partial \varvec{w}(\hat{\varvec{x}},\hat{\varvec{y}})}{\partial \varvec{x}} &{} \frac{\partial \varvec{w}(\hat{\varvec{x}},\hat{\varvec{y}})}{\partial \varvec{y}}\\ \frac{\partial \varvec{v}(\hat{\varvec{x}},\hat{\varvec{y}})}{\partial \varvec{x}} &{} \frac{\partial \varvec{v}(\hat{\varvec{x}},\hat{\varvec{y}})}{\partial \varvec{y}}\\ \end{bmatrix}, \end{aligned}$$
(51)

which is the Jacobian matrix of the QCQP (49a)–(49d) at the point \((\hat{\varvec{x}},\hat{\varvec{y}})\). As a result, the matrix \(\mathcal {J}_{\mathrm {PO}}(\hat{\varvec{x}})\) is singular if and only if \(\mathcal {J}_{\mathrm {QCQP}}(\hat{\varvec{x}},\hat{\varvec{y}})\) is singular. \(\square \)

Reformulation-linearization technique

This appendix covers the reformulation-linearization technique (RLT) of Sherali and Adams [72] as an approach to strengthen convex relaxations of the form (4a)–(4d) in the presence of affine constraints. Define \(\mathcal {L}\) as the set of affine constrains in the QCQP (1a)–(1c), i.e., \(\mathcal {L}\triangleq \{k\in \mathcal {I}\cup \mathcal {E}\;|\;\varvec{A}_k=\varvec{0}_{n\times n}\}\). Define also

(52a)
(52b)

where \(\varvec{B}\triangleq [\varvec{b}_1,\ldots ,\varvec{b}_{|\mathcal {I}\cap \mathcal {E}|}]^{\top }\) and \(\varvec{c}\triangleq [c_1,\ldots ,c_{|\mathcal {I}\cap \mathcal {E}|}]^{\top }\). Every \(\varvec{x}\in \mathcal {F}\) satisfies

$$\begin{aligned} \varvec{H}\varvec{x} + \varvec{h} \le 0, \end{aligned}$$
(53)

and, as a result, all elements of the matrix

$$\begin{aligned} \varvec{H}\varvec{x}\varvec{x}^{ \top } \varvec{H}^{\top }+ \varvec{h}\varvec{x}^{ \top } \varvec{H}^{ \top }+ \varvec{H}\varvec{x}\varvec{h}^{ \top }+ \varvec{h}\varvec{h}^{ \top } \end{aligned}$$
(54)

are non-negative if \(\varvec{x}\) is feasible. Hence, the inequality

$$\begin{aligned} \varvec{e}^{\top }_i\varvec{V}(\varvec{x},\varvec{x}\varvec{x}^{\top })\varvec{e}_j\ge 0 \end{aligned}$$
(55)

holds true for every \(\varvec{x}\in \mathcal {F}\) and \((i,j)\in \mathcal {H}\times \mathcal {H}\), where \(\varvec{V}:\mathbb {R}^n\times \mathbb {S}_n\rightarrow \mathbb {S}_{|\mathcal {H}|}\) is defined as

$$\begin{aligned} \varvec{V}(\varvec{x},\varvec{X})\triangleq \varvec{H}\varvec{X}\varvec{H}^{\top }+ \varvec{h}\varvec{x}^{\top } \varvec{H}^{\top }+ \varvec{H}\varvec{x}\varvec{h}^{ \top }+ \varvec{h}\varvec{h}^{\top }, \end{aligned}$$
(56)

\(\mathcal {H}\triangleq \{1,\ldots ,|\mathcal {L}\cap \mathcal {I}|+2|\mathcal {L}\cap \mathcal {E}|\}\), and \(\varvec{e}_1,\ldots ,\varvec{e}_{|\mathcal {H}|}\) denote the standard bases in \(\mathbb {R}^{|\mathcal {H}|}\).

This leads to a strengthened relaxation of QCQP (1a)–(1c):

$$\begin{aligned} \underset{\begin{subarray}{l} \varvec{x}\in \mathbb {R}^n, \varvec{X}\in \mathbb {S}_n\end{subarray}}{\text {minimize}} \ \&\bar{q}_0(\varvec{x},\varvec{X}) \end{aligned}$$
(57a)
$$\begin{aligned} \ \ \ \ \ \ \text {s.t.} \ \ \ \&\bar{q}_k(\varvec{x},\varvec{X}) \le 0,\qquad k\in \mathcal {I} \end{aligned}$$
(57b)
$$\begin{aligned}&\bar{q}_k(\varvec{x},\varvec{X}) = 0,\qquad k\in \mathcal {E} \end{aligned}$$
(57c)
$$\begin{aligned}&\varvec{X}-\varvec{x}\varvec{x}^\top \succeq _{\mathcal {C}_r}0 \end{aligned}$$
(57d)
$$\begin{aligned}&\varvec{e}^{\top }_i\varvec{V}(\varvec{x},\varvec{X})\varvec{e}_j\ge 0, \qquad \ (i,j)\in \mathcal {V} \end{aligned}$$
(57e)

where \(\mathcal {V}\subseteq \mathcal {H}\times \mathcal {H}\) is a selection of RLT inequalities.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Madani, R., Kheirandishfard, M., Lavaei, J. et al. Penalized semidefinite programming for quadratically-constrained quadratic optimization. J Glob Optim (2020). https://doi.org/10.1007/s10898-020-00918-8

Download citation

Keywords

  • Semidefinite programming
  • Non-convex optimization
  • Non-linear programming
  • Convex relaxation

Mathematics Subject Classification

  • 65K05
  • 90-08
  • 90C26
  • 90C22