Sparse semidefinite programs with guaranteed near-linear time complexity via dualized clique tree conversion

Abstract

Clique tree conversion solves large-scale semidefinite programs by splitting an \(n\times n\) matrix variable into up to n smaller matrix variables, each representing a principal submatrix of up to \(\omega \times \omega \). Its fundamental weakness is the need to introduce overlap constraints that enforce agreement between different matrix variables, because these can result in dense coupling. In this paper, we show that by dualizing the clique tree conversion, the coupling due to the overlap constraints is guaranteed to be sparse over dense blocks, with a block sparsity pattern that coincides with the adjacency matrix of a tree. We consider two classes of semidefinite programs with favorable sparsity patterns that encompass the MAXCUT and MAX k-CUT relaxations, the Lovasz Theta problem, and the AC optimal power flow relaxation. Assuming that \(\omega \ll n\), we prove that the per-iteration cost of an interior-point method is linearO(n) time and memory, so an \(\epsilon \)-accurate and \(\epsilon \)-feasible iterate is obtained after \(O(\sqrt{n}\log (1/\epsilon ))\) iterations in near-linear\(O(n^{1.5}\log (1/\epsilon ))\) time. We confirm our theoretical insights with numerical results on semidefinite programs as large as \(n=13{,}659\).

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    Throughout this paper, we denote the (ij)-th element of the matrix X as X[ij], and the submatrix of X formed by the rows in I and columns in J as X[IJ].

  2. 2.

    The symmetric matrices \(A_{1},A_{2},\dots ,A_{m}\) share an aggregate sparsity pattern E that contains at most \(\omega n\) nonzero elements (in the lower-triangular part). The set of symmetric matrices with sparsity pattern E is a linear subspace of \({\mathbb {R}}^{n\times n}\), with dimension at most \(\omega n\). Therefore, the number of linearly independent \(A_{1},A_{2},\dots ,A_{m}\) is at most \(\omega n\).

  3. 3.

    To keep our derivations simple, we perform the rank-1 update using the Sherman–Morrison—Woodbury (SMW) formula. In practice, the product-form Cholesky Factor (PFCF) formula of Goldfarb and Scheinberg [63] is more numerically stable and more widely used [59, 61]. Our complexity results remain valid in either cases because the PFCF is a constant factor of approximately two times slower than the SWM [63].

  4. 4.

    Since T is connected, we can always find a connected subset \(W'\) satisfying \(W\subseteq W'\subseteq V(T)\) and replace W by \(W'\).

References

  1. 1.

    Lovász, L.: On the Shannon capacity of a graph. IEEE Trans. Inf. Theory 25(1), 1–7 (1979)

    MathSciNet  MATH  Article  Google Scholar 

  2. 2.

    Goemans, M.X., Williamson, D.P.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM 42(6), 1115–1145 (1995)

    MathSciNet  MATH  Article  Google Scholar 

  3. 3.

    Sherali, H.D., Adams, W.P.: A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems. SIAM J. Discrete Math. 3(3), 411–430 (1990)

    MathSciNet  MATH  Article  Google Scholar 

  4. 4.

    Lovász, L., Schrijver, A.: Cones of matrices and set-functions and 0–1 optimization. SIAM J. Optim. 1(2), 166–190 (1991)

    MathSciNet  MATH  Article  Google Scholar 

  5. 5.

    Lasserre, J.B.: An explicit exact SDP relaxation for nonlinear 0-1 programs. In: International Conference on Integer Programming and Combinatorial Optimization, pp. 293–303, Springer, Berlin (2001)

    Google Scholar 

  6. 6.

    Laurent, M.: A comparison of the Sherali-Adams, Lovász-Schrijver, and Lasserre relaxations for 0–1 programming. Math. Oper. Res. 28(3), 470–496 (2003)

    MathSciNet  MATH  Article  Google Scholar 

  7. 7.

    Lasserre, J.B.: Global optimization with polynomials and the problem of moments. SIAM J. Optim. 11(3), 796–817 (2001)

    MathSciNet  MATH  Article  Google Scholar 

  8. 8.

    Parrilo, P.A.: Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization. Ph.D. thesis, California Institute of Technology (2000)

  9. 9.

    Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, vol. 87. Springer, Berlin (2013)

    Google Scholar 

  10. 10.

    Fukuda, M., Kojima, M., Murota, K., Nakata, K.: Exploiting sparsity in semidefinite programming via matrix completion I: general framework. SIAM J. Optim. 11(3), 647–674 (2001)

    MathSciNet  MATH  Article  Google Scholar 

  11. 11.

    Molzahn, D.K., Holzer, J.T., Lesieutre, B.C., DeMarco, C.L.: Implementation of a large-scale optimal power flow solver based on semidefinite programming. IEEE Trans. Power Syst. 28(4), 3987–3998 (2013)

    Article  Google Scholar 

  12. 12.

    Madani, R., Sojoudi, S., Lavaei, J.: Convex relaxation for optimal power flow problem: Mesh networks. IEEE Trans. Power Syst. 30(1), 199–211 (2015)

    Article  Google Scholar 

  13. 13.

    Madani, R., Ashraphijuo, M., Lavaei, J.: Promises of conic relaxation for contingency-constrained optimal power flow problem. IEEE Trans. Power Syst. 31(2), 1297–1307 (2016)

    Article  Google Scholar 

  14. 14.

    Eltved, A., Dahl, J., Andersen, M.S.: On the robustness and scalability of semidefinite relaxation for optimal power flow problems. Optim. Eng. 21, 375–392 (2020). https://doi.org/10.1007/s11081-019-09427-4

    MathSciNet  Article  Google Scholar 

  15. 15.

    Vandenberghe, L., Andersen, M.S., et al.: Chordal graphs and semidefinite optimization. Found. Trends Optim. 1(4), 241–433 (2015)

    Article  Google Scholar 

  16. 16.

    Sun, Y., Andersen, M.S., Vandenberghe, L.: Decomposition in conic optimization with partially separable structure. SIAM J. Optim. 24(2), 873–897 (2014)

    MathSciNet  MATH  Article  Google Scholar 

  17. 17.

    Andersen, M.S., Hansson, A., Vandenberghe, L.: Reduced-complexity semidefinite relaxations of optimal power flow problems. IEEE Trans. Power Syst. 29(4), 1855–1863 (2014)

    Article  Google Scholar 

  18. 18.

    Löfberg, J.: Dualize it: software for automatic primal and dual conversions of conic programs. Optim. Methods Softw. 24(3), 313–325 (2009)

    MathSciNet  MATH  Article  Google Scholar 

  19. 19.

    Griewank, A., Toint, P.L.: Partitioned variable metric updates for large structured optimization problems. Numerische Mathematik 39(1), 119–137 (1982)

    MathSciNet  MATH  Article  Google Scholar 

  20. 20.

    Andersen, M., Dahl, J., Vandenberghe, L.: CVXOPT: A Python package for convex optimization. abel.ee.ucla.edu/cvxopt. (2013)

  21. 21.

    Löfberg, J.: Yalmip: A toolbox for modeling and optimization in matlab. In: Proceedings of the CACSD Conference, 3. Taipei, Taiwan (2004)

  22. 22.

    Fujisawa, K., Kim, S., Kojima, M., Okamoto, Y., Yamashita, M.: User’s manual for SparseCoLO: Conversion methods for sparse conic-form linear optimization problems. Technical report, Department of Mathematical and Computing Sciences, Tokyo Institute of Technology, (2009). Research Report B-453

  23. 23.

    Kim, S., Kojima, M., Mevissen, M., Yamashita, M.: Exploiting sparsity in linear and nonlinear matrix inequalities via positive semidefinite matrix completion. Math. Program. 129(1), 33–68 (2011)

    MathSciNet  MATH  Article  Google Scholar 

  24. 24.

    Andersen, M.S.: Opfsdr v0.2.3 (2018)

  25. 25.

    Madani, R., Kalbat, A., Lavaei, J.: ADMM for sparse semidefinite programming with applications to optimal power flow problem. In: IEEE 54th Annual Conference on Decision and Control (CDC), pp. 5932–5939. IEEE (2015)

  26. 26.

    Zheng, Y., Fantuzzi, G., Papachristodoulou, A., Goulart, P., Wynn, A.: Chordal decomposition in operator-splitting methods for sparse semidefinite programs. Math. Program. 180, 489–532 (2020). https://doi.org/10.1007/s10107-019-01366-3

    MathSciNet  MATH  Article  Google Scholar 

  27. 27.

    Annergren, M., Pakazad, S.K., Hansson, A., Wahlberg, B.: A distributed primal-dual interior-point method for loosely coupled problems using ADMM. arXiv preprint arXiv:1406.2192 (2014)

  28. 28.

    Khoshfetrat Pakazad, S., Hansson, A., Andersen, M.S.: Distributed interior-point method for loosely coupled problems. IFAC Proc. Volumes 47(3), 9587–9592 (2014)

    Article  Google Scholar 

  29. 29.

    Zhang, R.Y., White, J.K.: Gmres-accelerated ADMM for quadratic objectives. SIAM J. Optim. 28(4), 3025–3056 (2018)

    MathSciNet  MATH  Article  Google Scholar 

  30. 30.

    Andersen, M.S., Dahl, J., Vandenberghe, L.: Implementation of nonsymmetric interior-point methods for linear optimization over sparse matrix cones. Math. Program. Comput. 2(3), 167–201 (2010)

    MathSciNet  MATH  Article  Google Scholar 

  31. 31.

    Duff, I.S., Reid, J.K.: The multifrontal solution of indefinite sparse symmetric linear. ACM Trans. Math. Softw. (TOMS) 9(3), 302–325 (1983)

    MathSciNet  MATH  Article  Google Scholar 

  32. 32.

    Liu, J.W.: The multifrontal method for sparse matrix solution: theory and practice. SIAM Rev. 34(1), 82–109 (1992)

    MathSciNet  MATH  Article  Google Scholar 

  33. 33.

    Pakazad, S.Khoshfetrat, Hansson, A., Andersen, M.S., Nielsen, I.: Distributed primal-dual interior-point methods for solving tree-structured coupled convex problems using message-passing. Optim. Methods Softw. 32(3), 401–435 (2017)

    MathSciNet  MATH  Article  Google Scholar 

  34. 34.

    Khoshfetrat Pakazad, S., Hansson, A., Andersen, M.S., Rantzer, A.: Distributed semidefinite programming with application to large-scale system analysis. IEEE Trans. Autom. Control 63(4), 1045–1058 (2017)

    MathSciNet  MATH  Article  Google Scholar 

  35. 35.

    Alizadeh, F., Haeberly, J.-P.A., Overton, M.L.: Complementarity and nondegeneracy in semidefinite programming. Math. Program. 77(1), 111–128 (1997)

    MathSciNet  MATH  Article  Google Scholar 

  36. 36.

    Arnborg, S., Corneil, D.G., Proskurowski, A.: Complexity of finding embeddings in a k-tree. SIAM J. Algebraic Discrete Methods 8(2), 277–284 (1987)

    MathSciNet  MATH  Article  Google Scholar 

  37. 37.

    Ye, Y., Todd, M.J., Mizuno, S.: An \(O(\sqrt{nL})\)-iteration homogeneous and self-dual linear programming algorithm. Math. Oper. Res. 19(1), 53–67 (1994)

    MathSciNet  MATH  Article  Google Scholar 

  38. 38.

    Wolkowicz, H., Saigal, R., Vandenberghe, L. (eds.): Handbook of Semidefinite Programming: Theory, Algorithms, and Applications. Springer Science & Business Media (2012)

  39. 39.

    Permenter, F., Friberg, H.A., Andersen, E.D.: Solving conic optimization problems via self-dual embedding and facial reduction: a unified approach. SIAM J. Optim. 27(3), 1257–1282 (2017)

    MathSciNet  MATH  Article  Google Scholar 

  40. 40.

    Frieze, A., Jerrum, M.: Improved approximation algorithms for MAX k-CUT and MAX BISECTION. Algorithmica 18(1), 67–81 (1997)

    MathSciNet  MATH  Article  Google Scholar 

  41. 41.

    Pataki, G., Stefan S.: The DIMACS library of semidefinite-quadratic-linear programs. Technical Report. Preliminary draft, Computational Optimization Research Center, Columbia University, New York (2002)

  42. 42.

    Borchers, B.: Sdplib 1.2, a library of semidefinite programming test problems. Optim. Methods Softw. 11(1–4), 683–690 (1999)

    MathSciNet  MATH  Article  Google Scholar 

  43. 43.

    Sun, Y.: Decomposition methods for semidefinite optimization. Ph.D. thesis, UCLA (2015)

  44. 44.

    Liu, J.W.: The role of elimination trees in sparse factorization. SIAM J. Matrix Anal. Appl. 11(1), 134–172 (1990)

    MathSciNet  MATH  Article  Google Scholar 

  45. 45.

    Bodlaender, H.L., Gilbert, J.R., Hafsteinsson, H., Kloks, T.: Approximating treewidth, pathwidth, frontsize, and shortest elimination tree. J. Algorithms 18(2), 238–255 (1995)

    MathSciNet  MATH  Article  Google Scholar 

  46. 46.

    Leighton, T., Rao, S.: An approximate max-flow min-cut theorem for uniform multicommodity flow problems with applications to approximation algorithms. In: Proceedings of the 29th Annual Symposium on Foundations of Computer Science, pp. 422–431. IEEE (1988)

  47. 47.

    Klein, P., Stein, C., Tardos, E.: Leighton-Rao might be practical: faster approximation algorithms for concurrent flow with uniform capacities. In: Proceedings of the twenty-second annual ACM symposium on Theory of computing, pp. 310–321. ACM (1990)

  48. 48.

    George, A., Liu, J.W.: The evolution of the minimum degree ordering algorithm. SIAM Rev. 31(1), 1–19 (1989)

    MathSciNet  MATH  Article  Google Scholar 

  49. 49.

    Lipton, R.J., Rose, D.J., Tarjan, R.E.: Generalized nested dissection. SIAM J. Numer. Anal. 16(2), 346–358 (1979)

    MathSciNet  MATH  Article  Google Scholar 

  50. 50.

    Rose, D.J., Tarjan, R.E., Lueker, G.S.: Algorithmic aspects of vertex elimination on graphs. SIAM J. Comput. 5(2), 266–283 (1976)

    MathSciNet  MATH  Article  Google Scholar 

  51. 51.

    George, A., Liu, J.W.: Computer Solution of Large Sparse Positive Definite Systems. Prentice Hall, Englewood Cliffs, NJ (1981)

    Google Scholar 

  52. 52.

    Grone, R., Johnson, C.R., Sá, E.M., Wolkowicz, H.: Positive definite completions of partial Hermitian matrices. Linear Algebra Appl. 58, 109–124 (1984)

    MathSciNet  MATH  Article  Google Scholar 

  53. 53.

    Dancis, J.: Positive semidefinite completions of partial hermitian matrices. Linear Algebra Appl. 175, 97–114 (1992)

    MathSciNet  MATH  Article  Google Scholar 

  54. 54.

    Laurent, M., Varvitsiotis, A.: A new graph parameter related to bounded rank positive semidefinite matrix completions. Math. Program. 145(1–2), 291–325 (2014)

    MathSciNet  MATH  Article  Google Scholar 

  55. 55.

    Madani, R., Sojoudi, S., Fazelnia, G., Lavaei, J.: Finding low-rank solutions of sparse linear matrix inequalities using convex optimization. SIAM J. Optim. 27(2), 725–758 (2017)

    MathSciNet  MATH  Article  Google Scholar 

  56. 56.

    Jiang, X.: Minimum rank positive semidefinite matrix completion with chordal sparsity pattern. Master’s thesis, UCLA (2017)

  57. 57.

    Kobayashi, K., Kim, S., Kojima, M.: Correlative sparsity in primal-dual interior-point methods for LP, SDP, and SOCP. Appl. Math. Optim. 58(1), 69–88 (2008)

    MathSciNet  MATH  Article  Google Scholar 

  58. 58.

    Parter, S.: The use of linear graphs in Gauss elimination. SIAM Rev. 3(2), 119–130 (1961)

    MathSciNet  MATH  Article  Google Scholar 

  59. 59.

    Sturm, J.F.: Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optim. Methods Softw. 11(1–4), 625–653 (1999)

    MathSciNet  MATH  Article  Google Scholar 

  60. 60.

    Andersen, E.D.: Handling free variables in primal-dual interior-point methods using a quadratic cone. In: Proceedings of the SIAM Conference on Optimization, Toronto (2002)

  61. 61.

    Sturm, J.F.: Implementation of interior point methods for mixed semidefinite and second order cone optimization problems. Optim. Methods Softw. 17(6), 1105–1154 (2002)

    MathSciNet  MATH  Article  Google Scholar 

  62. 62.

    Andersen, E.D., Roos, C., Terlaky, T.: On implementing a primal-dual interior-point method for conic quadratic optimization. Math. Program. 95(2), 249–277 (2003)

    MathSciNet  MATH  Article  Google Scholar 

  63. 63.

    Goldfarb, D., Scheinberg, K.: Product-form Cholesky factorization in interior point methods for second-order cone programming. Math. Program. 103(1), 153–179 (2005)

    MathSciNet  MATH  Article  Google Scholar 

  64. 64.

    Guo, J., Niedermeier, R.: Exact algorithms and applications for Tree-like Weighted Set Cover. J. Discrete Algorithms 4(4), 608–622 (2006)

    MathSciNet  MATH  Article  Google Scholar 

  65. 65.

    Lewis, J.G., Peyton, B.W., Pothen, A.: A fast algorithm for reordering sparse matrices for parallel factorization. SIAM J. Sci. Stat. Comput. 10(6), 1146–1173 (1989)

    MathSciNet  MATH  Article  Google Scholar 

  66. 66.

    Pothen, A., Sun, C.: Compact clique tree data structures in sparse matrix factorizations. In: Coleman, T.F., Li, Y. (eds.) Large-Scale Numerical Optimization, pp. 180–204. SIAM (1990)

  67. 67.

    Andersen, M.S., Dahl, J., Vandenberghe, L.: Logarithmic barriers for sparse matrix cones. Optim. Methods Softw. 28(3), 396–423 (2013)

    MathSciNet  MATH  Article  Google Scholar 

  68. 68.

    George, A., Gilbert, J.R., Liu, J.W.H. (eds.): Graph Theory and Sparse Matrix Computation. Springer Science & Business Media (2012)

  69. 69.

    Zimmerman, R.D., Murillo-Sánchez, C.E., Thomas, R.J.: MATPOWER: Steady-state operations, planning, and analysis tools for power systems research and education. IEEE Trans. Power Syst. 26(1), 12–19 (2011)

    Article  Google Scholar 

  70. 70.

    Josz, C., Fliscounakis, S., Maeght, J., Panciatici, P.: AC power flow data in MATPOWER and QCQP format: iTesla, RTE snapshots, and PEGASE. arXiv preprint arXiv:1603.01533 (2016)

  71. 71.

    Mittelmann, H.D.: An independent benchmarking of SDP and SOCP solvers. Math. Program. 95(2), 407–430 (2003)

    MathSciNet  MATH  Article  Google Scholar 

  72. 72.

    Frenk, H., Roos, K., Terlaky, T., Zhang, S. (eds.): High Performance Optimiztion. Springer Science & Business Media (2013)

  73. 73.

    Amestoy, P.R., Davis, T.A., Duff, I.S.: Algorithm 837: AMD, an approximate minimum degree ordering algorithm. ACM Trans. Math. Softw. (TOMS) 30(3), 381–388 (2004)

    MathSciNet  MATH  Article  Google Scholar 

  74. 74.

    Lavaei, J., Low, S.H.: Zero duality gap in optimal power flow problem. IEEE Trans. Power Syst. 27(1), 92 (2012)

    Article  Google Scholar 

  75. 75.

    Nakata, K., Fujisawa, K., Fukuda, M., Kojima, M., Murota, K.: Exploiting sparsity in semidefinite programming via matrix completion II: implementation and numerical results. Math. Program. 95(2), 303–327 (2003)

    MathSciNet  MATH  Article  Google Scholar 

  76. 76.

    Agler, J., Helton, W., McCullough, S., Rodman, L.: Positive semidefinite matrices with a given sparsity pattern. Linear Algebra Appl. 107, 101–149 (1988)

    MathSciNet  MATH  Article  Google Scholar 

  77. 77.

    Vanderbei, R.J.: Linear Programming: Foundations and Extensions. Springer, Berlin (2015)

    Google Scholar 

  78. 78.

    Nesterov, Y.E., Todd, M.J.: Primal-dual interior-point methods for self-scaled cones. SIAM J. Optim. 8(2), 324–364 (1998)

    MathSciNet  MATH  Article  Google Scholar 

  79. 79.

    Sturm, J.F., Zhang, S.: Symmetric primal-dual path-following algorithms for semidefinite programming. Appl. Numer. Math. 29(3), 301–315 (1999)

    MathSciNet  MATH  Article  Google Scholar 

  80. 80.

    Sturm, J.F., Zhang, S.: On a wide region of centers and primal-dual interior point algorithms for linear programming. Math. Oper. Res. 22(2), 408–431 (1997)

    MathSciNet  MATH  Article  Google Scholar 

  81. 81.

    Todd, M.J., Toh, K.-C., Tütüncü, R.H.: On the nesterov-todd direction in semidefinite programming. SIAM J. Optim. 8(3), 769–796 (1998)

    MathSciNet  MATH  Article  Google Scholar 

  82. 82.

    Alizadeh, F., Haeberly, J.-P.A., Overton, M.L.: Primal-dual interior-point methods for semidefinite programming: convergence rates, stability and numerical results. SIAM J. Optim. 8(3), 746–768 (1998)

    MathSciNet  MATH  Article  Google Scholar 

Download references

Acknowledgements

The authors are grateful to Daniel Bienstock, Salar Fattahi, Cédric Josz, and Yi Ouyang for insightful discussions and helpful comments on earlier versions of this manuscript. We thank Frank Permenter for clarifications on various aspects of the homogeneous self-dual embedding for SDPs. Finally, we thank the Associate Editor and Reviewer 2 for meticulous and detailed comments that led to a significantly improved paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Javad Lavaei.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the ONR YIP Award, DARPA YFA Award, AFOSR YIP Award, NSF CAREER Award, and ONR N000141712933.

Appendices

A Linear independence and Slater’s conditions

In this section, we prove that (CTC) inherits the assumptions of linear independence and Slater’s conditions from (SDP). We begin with two important technical lemmas.

Lemma 10

The matrix \({\mathbf {N}}\) in (13) has full row rank, that is \(\det ({\mathbf {N}}{\mathbf {N}}^{T})\ne 0\).

Proof

We make \({\mathbf {N}}=[{\mathbf {N}}_{i,j}]_{i,j=1}^{\ell }\) upper-triangular by ordering its blocks topologically on T: each nonempty block row \({\mathbf {N}}_{j}\) contains a nonzero block at \({\mathbf {N}}_{j,j}\) and a nonzero block at \({\mathbf {N}}_{j,p(j)}\) where the parent node \(p(j)>j\) is ordered after j. Then, the claim follows because each diagonal block \({\mathbf {N}}_{j,j}\) implements a surjection and must therefore have full row-rank. \(\square \)

Lemma 11

(Orthogonal complement) Let \(d=\frac{1}{2}\sum _{j=1}^{\ell }|J_{j}|(|J_{j}|+1)\). Implicitly define the \(d\times \frac{1}{2}n(n+1)\) matrix \({\mathbf {P}}\) to satisfy

$$\begin{aligned} {\mathbf {P}}\,\mathrm {svec}\,(X)&=[\mathrm {svec}\,(X[J_{1},J_{1}]);\ldots ;\mathrm {svec}\,(X[J_{\ell },J_{\ell }])]\qquad \forall X\in {\mathbb {S}}^{n}. \end{aligned}$$

Then, (i) \({\mathbf {N}}{\mathbf {P}}=0\); (ii) every \(x\in {\mathbb {R}}^{d}\) can be decomposed as \(x={\mathbf {N}}^{T}u+{\mathbf {P}}v\).

Proof

For each \(x=[\mathrm {svec}\,(X_{j})]_{j=1}^{\ell }\in {\mathbb {R}}^{d}\), Theorem 4 says that there exists a Z satisfying \({\mathbf {P}}\,\mathrm {svec}\,(Z)=x\) if and only if \({\mathbf {N}}x=0\). Equivalently, \(x\in {\mathrm {span}}({\mathbf {P}})\) if and only if \(x\perp {\mathrm {span}}({\mathbf {N}}^{T})\). The “only if” part implies (i), while the “if” part implies (ii).

\(\square \)

Define the \(m\times \frac{1}{2}n(n+1)\) matrix \({\mathbf {M}}\) as the vectorization of the linear constraints in (SDP), as in

$$\begin{aligned} {\mathbf {M}}\,\mathrm {svec}\,(X)=\begin{bmatrix}\mathrm {svec}\,(A_{1})^{T}\\ \vdots \\ \mathrm {svec}\,(A_{m})^{T} \end{bmatrix}\mathrm {svec}\,(X)=\begin{bmatrix}\mathrm {svec}\,(A_{1})^{T}\mathrm {svec}\,(X)\\ \vdots \\ \mathrm {svec}\,(A_{m})^{T}\mathrm {svec}\,(X) \end{bmatrix}=\begin{bmatrix}A_{1}\bullet X\\ \vdots \\ A_{m}\bullet X \end{bmatrix}. \end{aligned}$$

In reformulating (SDP) into (CTC), the splitting conditions (10) can be rewritten as the following

$$\begin{aligned} c^{T}{\mathbf {P}}&=\mathrm {svec}\,(C)^{T},&{\mathbf {A}}{\mathbf {P}}&={\mathbf {M}}, \end{aligned}$$
(38)

where \(c=[\mathrm {svec}\,(C_{j})]_{j=1}^{\ell }\) and \({\mathbf {A}}=[\mathrm {svec}\,(A_{i,j})^{T}]_{i,j=1}^{m,\ell }\) are the data for the vectorized verison of (CTC).

Proof of Lemma 1

We will prove that

$$\begin{aligned} \text {exists }[u;v]\ne 0,\quad {\mathbf {A}}^{T}u+{\mathbf {N}}^{T}v=0\qquad \iff \qquad \text {exists }y\ne 0,\quad \sum _{i=1}^{m}y_{i}A_{i}=0. \end{aligned}$$

(\(\implies \)) We must have \(u\ne 0\), because \({\mathbf {N}}\) has full row rank by Lemma 10, and so \({\mathbf {A}}^{T}0+{\mathbf {N}}^{T}v=0\) if and only if \(v=0\). Multiplying by \({\mathbf {P}}\) yields \({\mathbf {P}}^{T}({\mathbf {A}}^{T}u+{\mathbf {N}}^{T}v)={\mathbf {M}}^{T}u+0=0\) and so setting \(y=u\ne 0\) yields \({\mathbf {M}}^{T}y=0\). () We use Lemma 11 to decompose \({\mathbf {A}}^{T}y={\mathbf {P}}z+{\mathbf {N}}^{T}v\). If \(\mathrm {svec}\,(\sum _{i}y_{i}A_{i})={\mathbf {M}}^{T}y={\mathbf {P}}^{T}{\mathbf {A}}^{T}y=0,\) then \({\mathbf {P}}^{T}{\mathbf {P}}z={\mathbf {P}}^{T}{\mathbf {A}}^{T}y-{\mathbf {P}}^{T}{\mathbf {N}}^{T}v=0\) and so \({\mathbf {P}}z=0\). Setting \(u=-y\ne 0\) yields \({\mathbf {A}}^{T}u+{\mathbf {N}}^{T}v=0\). \(\square \)

Proof of Lemma 2

We will prove that

$$\begin{aligned} \text {exists }x\in \mathrm {Int}({\mathcal {K}}),\quad \begin{bmatrix}{\mathbf {A}}\\ {\mathbf {N}}\end{bmatrix}x=\begin{bmatrix}b\\ 0 \end{bmatrix}\qquad \iff \qquad \text {exists }X\succeq 0,\quad {\mathbf {M}}\,\mathrm {svec}\,(X)=b. \end{aligned}$$

Define the chordal completion F as in (8). Observe that \({\mathbf {M}}\,\mathrm {svec}\,(X)={\mathbf {M}}\,\mathrm {svec}\,(Z)\) holds for all pairs of \(P_{F}(X)=Z\), because each \(A_{i}\in {\mathbb {S}}_{F}^{n}\) satisfies \(A_{i}\bullet X=A_{i}\bullet P_{F}(X)\). Additionally, the positive definite version of Theorem 3 is written

$$\begin{aligned} \text {exists }X\succ 0,\quad P_{F}(X)=Z\qquad \iff \qquad {\mathbf {P}}\,\mathrm {svec}\,(Z)\in \mathrm {Int}({\mathcal {K}}). \end{aligned}$$
(39)

This result was first established by Grone et al. [52]; a succinct proof can be found in [15, Theorem 10.1]. (\(\implies \)) For every x satisfying \({\mathbf {N}}x=0\), there exists Z such that \({\mathbf {P}}\,\mathrm {svec}\,(Z)=x\) due to Lemma 11. If additionally \(x\in \mathrm {Int}({\mathcal {K}})\), then there exists \(X\succ 0\) satisfying \(Z=P_{F}(X)\) due to (39). We can verify that \({\mathbf {M}}\,\mathrm {svec}\,(X)={\mathbf {M}}\,\mathrm {svec}\,(Z)={\mathbf {A}}{\mathbf {P}}\,\mathrm {svec}\,(Z)={\mathbf {A}}x=b\). () For every \(X\succ 0,\) there exists Z satisfying \(Z=P_{F}(X)\) and \({\mathbf {P}}\,\mathrm {svec}\,(Z)\in \mathrm {Int}({\mathcal {K}})\) due to (39). Set \(u={\mathbf {P}}\,\mathrm {svec}\,(Z)\) and observe that \(u\in \mathrm {Int}({\mathcal {K}})\) and \({\mathbf {N}}u={\mathbf {N}}{\mathbf {P}}\,\mathrm {svec}\,(Z)=0\). If additionally \({\mathbf {M}}\,\mathrm {svec}\,(X)=b\), then \({\mathbf {A}}u={\mathbf {A}}{\mathbf {P}}\,\mathrm {svec}\,(Z)={\mathbf {M}}\,\mathrm {svec}\,(Z)=b\). \(\square \)

Proof of Lemma 2

We will prove that

$$\begin{aligned} \text {exists }u,v,\quad c-{\mathbf {A}}^{T}u-{\mathbf {N}}^{T}v\in \mathrm {Int}({\mathcal {K}}_{*})\qquad \iff \qquad \text {exists }y,\quad C-\sum _{i}y_{i}A_{i}\succ 0. \end{aligned}$$

Define the chordal completion F as in (8). Theorem 3 in (39) has a dual theorem

$$\begin{aligned} \text {exists }S\succ 0,\quad S\in {\mathbb {S}}_{F}^{n}\qquad \iff \qquad \text {exists }h\in \mathrm {Int}({\mathcal {K}}_{*}),\quad \mathrm {svec}\,(S)={\mathbf {P}}^{T}h.\qquad \quad \end{aligned}$$
(40)

This result readily follows from the positive semidefinite version proved by Alger et al. [76]; see also [15, Theorem 9.2]. (\(\implies \)) For each \(h=c-{\mathbf {A}}^{T}u-{\mathbf {N}}^{T}v\), define \(S=C-\sum _{i}u_{i}A_{i}\) and observe that

$$\begin{aligned} {\mathbf {P}}^{T}h={\mathbf {P}}^{T}(c-{\mathbf {A}}^{T}u-{\mathbf {N}}^{T}v)=\mathrm {svec}\,(C)-{\mathbf {M}}^{T}u-0=\mathrm {svec}\,(S). \end{aligned}$$

If additionally \(h\in {\mathcal {K}}_{*}\), then \(S\succ 0\) due to (40). () For each \(S=C-\sum _{i}y_{i}A_{i}\succ 0\), there exists an \(h\in \mathrm {Int}({\mathcal {K}}_{*})\) satisfying \(\mathrm {svec}\,(S)={\mathbf {P}}^{T}h\) due to (40). We use Lemma 11 to decompose \(h={\mathbf {P}}u+{\mathbf {N}}^{T}v\). Given that \(\mathrm {svec}\,(S)={\mathbf {P}}^{T}h={\mathbf {P}}^{T}{\mathbf {P}}u+0\), we must actually have \({\mathbf {P}}u=c-{\mathbf {A}}^{T}y\) since \({\mathbf {P}}^{T}(c-{\mathbf {A}}^{T}y)=\mathrm {svec}\,(C)-{\mathbf {M}}^{T}y=\mathrm {svec}\,(S)\). Hence \(h=c-{\mathbf {A}}^{T}y+{\mathbf {N}}^{T}v\) and \(h\in \mathrm {Int}({\mathcal {K}}_{*})\). \(\square \)

B Extension to inequality constraints

Consider the modifying the equality constraint in (11) into an inequality constraint, as in

$$\begin{aligned} \text {minimize }c^{T}x\quad \text {subject to }\quad {\mathbf {A}}x\le b,\quad {\mathbf {N}}x=0,\quad x\in {\mathcal {K}}. \end{aligned}$$

The corresponding dualization reads

$$\begin{aligned} \text {maximize }-c^{T}y\quad \text {subject to }\quad \begin{bmatrix}{\mathbf {A}}\\ {\mathbf {N}}\\ -I \end{bmatrix}y+s=\begin{bmatrix}b\\ 0\\ 0 \end{bmatrix},\quad s\in {\mathbb {R}}_{+}^{m}\times \{0\}^{f}\times {\mathcal {K}}, \end{aligned}$$

where m denotes the number of rows in \({\mathbf {A}}\) and f now denotes the number of rows in \({\mathbf {N}}\). Embedding the equality constraint into a second-order cone, the associated normal equation takes the form

$$\begin{aligned} \left( {\mathbf {D}}_{s}+{\mathbf {A}}^{T}{\mathbf {D}}_{l}{\mathbf {A}}+{\mathbf {N}}^{T}{\mathbf {D}}_{f}{\mathbf {N}}\right) \varDelta y=r, \end{aligned}$$

where \({\mathbf {D}}_{s}\) and \({\mathbf {D}}_{f}\) are comparable as before in (15) and (23), and \({\mathbf {D}}_{l}\) is a diagonal matrix with positive diagonal elements. This matrix has the same sparse-plus-rank-1 structure as (22), and can therefore be solved using the same rank-1 update

$$\begin{aligned} \varDelta y=({\mathbf {H}}+{\mathbf {q}}{\mathbf {q}}^{T})^{-1}r=\left( I-\frac{({\mathbf {H}}^{-1}{\mathbf {q}}){\mathbf {q}}^{T}}{1+{\mathbf {q}}^{T}({\mathbf {H}}^{-1}{\mathbf {q}})}\right) {\mathbf {H}}^{-1}r, \end{aligned}$$

where \({\mathbf {H}}\) and \({\mathbf {q}}\) now read

$$\begin{aligned} {\mathbf {H}}&={\mathbf {D}}_{s}+{\mathbf {A}}^{T}{\mathbf {D}}_{l}{\mathbf {A}}+\sigma {\mathbf {N}}^{T}{\mathbf {N}},&{\mathbf {q}}&={\mathbf {N}}^{T}w. \end{aligned}$$

The matrix \({\mathbf {H}}\) has the same block sparsity graph as the tree graph T, so we can evoke Lemma 5 to show that the cost of computing \(\varDelta y\) is again \(O(\omega ^{6}n)\) time and \(O(\omega ^{4}n)\) memory.

C Interior-point method complexity analysis

We solve the dualized problem (21) by solving its extended homogeneous self-dual embedding

$$\begin{aligned}&\text {min.}\quad (\nu +1)\theta \end{aligned}$$
(41a)
$$\begin{aligned}&\text {s.t.} \quad \begin{bmatrix}0 &{}\quad +{\mathbf {M}}^{T} &{}\quad -{\mathbf {c}}&{}\quad -r_{d}\\ -{\mathbf {M}}&{}\quad 0 &{}\quad +{\mathbf {b}}&{}\quad -r_{p}\\ +{\mathbf {c}}^{T} &{}\quad -{\mathbf {b}}^{T} &{}\quad 0 &{}\quad -r_{c}\\ r_{d}^{T} &{}\quad r_{p}^{T} &{}\quad r_{c} &{}\quad 0 \end{bmatrix}\begin{bmatrix}x\\ y\\ \tau \\ \theta \end{bmatrix}+\begin{bmatrix}s\\ 0\\ \kappa \\ 0 \end{bmatrix} =\begin{bmatrix}0\\ 0\\ 0\\ \nu +1 \end{bmatrix} \end{aligned}$$
(41b)
$$\begin{aligned}&x,s\in {\mathcal {C}}\quad \kappa ,\tau \quad \ge 0, \end{aligned}$$
(41c)

where the data is given in standard form

$$\begin{aligned} {\mathbf {M}}=\begin{bmatrix}0&-\begin{bmatrix}{\mathbf {A}}^{T}&{\mathbf {N}}^{T}\end{bmatrix}&+I\end{bmatrix},\quad {\mathbf {c}}^{T}=\begin{bmatrix}0&\begin{bmatrix}b^{T}&0\end{bmatrix}&0\end{bmatrix},\quad {\mathbf {b}}=-c,\end{aligned}$$
(41d)
$$\begin{aligned} {\mathcal {C}}=\{(x_{0},x_{1}\}\in {\mathbb {R}}^{p+1}:\Vert x_{1}\Vert \le x_{0}\}\times {\mathcal {K}}, \end{aligned}$$
(41e)

and the residual vectors are defined

$$\begin{aligned} r_{d}={\mathbf {1}}_{{\mathcal {C}}}-{\mathbf {c}},\quad r_{p}=-{\mathbf {M}}{\mathbf {1}}_{{\mathcal {C}}}+{\mathbf {b}},\quad r_{c}=1+{\mathbf {c}}^{T}{\mathbf {1}}_{{\mathcal {C}}}. \end{aligned}$$
(41f)

Here, \(\nu \) is the order of the cone \({\mathcal {C}}\), and \({\mathbf {1}}_{{\mathcal {C}}}\) is its identity element

$$\begin{aligned} \nu&=1+|J_{1}|+\cdots +|J_{\ell }|,&{\mathbf {1}}_{{\mathcal {C}}}&=[1;0;\ldots ;0;\mathrm {svec}\,(I_{|J_{1}|});\ldots ;\mathrm {svec}\,(I_{|J_{\ell }|})]. \end{aligned}$$
(42)

Problem (41) has optimal value \(\theta ^{\star }=0\). Under the primal-dual Slater’s conditions (Assumption 2), an interior-point method is guaranteed to converge to an \(\epsilon \)-accurate solution with \(\tau >0\), and this yields an \(\epsilon \)-feasible and \(\epsilon \)-accurate solution to the dualized problem (21) by rescaling \(x/\tau \) and \(y=y/\tau \) and \(s=s/\tau \). The following result is adapted from [38, Lemma 5.7.2] and [77, Theorem 22.7].

Lemma 12

(\(\epsilon \)-accurate and \(\epsilon \)-feasible) If \((x,y,s,\tau ,\theta ,\kappa )\) satisfies (41b) and (41c) and

$$\begin{aligned} \mu =\frac{x^{T}s+\tau \kappa }{\nu +1}&\le \epsilon ,&\tau \kappa \ge&\gamma \mu \end{aligned}$$

for constants \(\epsilon ,\gamma >0\), then the rescaled point \((x/\tau ,y/\tau ,s/\tau )\) satisfies

$$\begin{aligned} \Vert {\mathbf {M}}(x/\tau )-{\mathbf {b}}\Vert&\le K\epsilon ,&\Vert {\mathbf {M}}^{T}(y/\tau )+(s/\tau )-{\mathbf {c}}\Vert&\le K\epsilon ,&\frac{(x/\tau )^{T}(s/\tau )}{\nu +1}\le K\epsilon \end{aligned}$$

where K is a constant.

Proof

Note that (41b) implies \(\mu =\theta \) and

$$\begin{aligned} \Vert {\mathbf {M}}(x/\tau )-{\mathbf {b}}\Vert&=\frac{\Vert r_{p}\Vert \mu }{\tau },&\Vert {\mathbf {M}}^{T}(y/\tau )+(s/\tau )-{\mathbf {c}}\Vert&=\frac{\Vert r_{d}\Vert \mu }{\tau },&\\ \frac{(x/\tau )^{T}(s/\tau )}{\nu +1}&=\frac{\mu }{\tau ^{2}}. \end{aligned}$$

Hence, we obtain our desired result by upper-bounding \(1/\tau \). Let \((x^{\star },y^{\star },s^{\star },\tau ^{\star },\theta ^{\star },\kappa ^{\star })\) be a solution of (41), and note that for every \((x,y,s,\tau ,\theta ,\kappa )\) satisfying (41b) and (41c), we have the following via the skew-symmetry of (41b)

$$\begin{aligned} (x-x^{\star })^{T}(s-s^{\star })+(\tau -\tau ^{\star })(\kappa -\kappa ^{\star })=0. \end{aligned}$$

Rearranging yields

$$\begin{aligned} (\nu +1)\mu =x^{T}s+\tau \kappa =(x^{\star })^{T}s +x^{T}(s^{\star })+\tau \kappa ^{\star }+\tau ^{\star }\kappa \ge \tau ^{\star }\kappa \end{aligned}$$

and hence

$$\begin{aligned} \kappa \tau ^{\star }\le \mu (\nu +1),\quad \tau \kappa \ge \gamma \mu \qquad \implies \qquad \tau \ge \frac{\gamma }{\nu +1}\tau ^{\star }. \end{aligned}$$

If (SDP) satisfies the primal-dual Slater’s condition, then (CTC) also satisfies the primal-dual Slater’s condition (Lemmas 23). Therefore, the vectorized version (11) of (CTC) attains a solution \(({\hat{x}},{\hat{y}},{\hat{s}})\) with \({\hat{x}}^{T}{\hat{s}}=0\), and the following

$$\begin{aligned} x^{\star }=\tau ^{\star }\begin{bmatrix}\Vert {\hat{y}}\Vert \\ {\hat{y}}\\ {\hat{s}} \end{bmatrix},\quad y^{\star }=\tau ^{\star }{\hat{x}},\quad s^{\star }=\tau ^{\star }\begin{bmatrix}0\\ 0\\ {\hat{x}} \end{bmatrix},\quad \tau ^{\star }=\frac{\nu +1}{\Vert {\hat{y}}\Vert +{\mathbf {1}}_{{\mathcal {K}}}^{T}{\hat{s}}+{\mathbf {1}}_{{\mathcal {K}}}^{T}{\hat{x}}+1}, \end{aligned}$$

with \(\theta ^{\star }=\kappa ^{\star }=0\) is a solution to (41). This proves the following upper-bound

$$\begin{aligned} \frac{1}{\tau }\le K_{\tau }=\frac{1}{\gamma }\cdot \min _{{\hat{x}},{\hat{y}},{\hat{s}}}\{\Vert {\hat{y}}\Vert +{\mathbf {1}}_{{\mathcal {K}}}^{T}{\hat{s}}+{\mathbf {1}}_{{\mathcal {K}}}^{T}{\hat{x}}+1:({\hat{x}},{\hat{y}},{\hat{s}}) \text { solves }(11)\text { with }{\hat{x}}^{T}{\hat{s}}=0\}. \end{aligned}$$

Setting \(K=\max \{\Vert r_{p}\Vert K_{\tau },\Vert r_{d}\Vert K_{\tau },K_{\tau }^{2}\}\) yields our desired result. \(\square \)

We solve the homogeneous self-dual embedding (41) using the short-step method of Nesterov and Todd [78, Algorithm 6.1] (and also Sturm and S. Zhang [79, Section 5.1]), noting that SeDuMi reduces to it in the worst case; see [61] and [80]. Beginning at the following strictly feasible, perfectly centered point

$$\begin{aligned} \theta ^{(0)}=\tau ^{(0)}=\kappa ^{(0)}=1,\quad y^{(0)}=0,\quad x^{(0)}=s^{(0)}={\mathbf {1}}_{{\mathcal {C}}}, \end{aligned}$$
(43)

with barrier parameter \(\mu =1\), we take the following steps

$$\begin{aligned} \mu ^{+}&=\left( 1-\frac{1}{15\sqrt{\nu +1}}\right) \cdot \frac{x^{T}s+\tau \kappa }{\nu +1},\nonumber \\ (x^{+},y^{+},s^{+},\tau ^{+},\theta ^{+},\kappa ^{+})&=(x,y,s,\tau ,\theta ,\kappa )+(\varDelta x,\varDelta y,\varDelta s,\varDelta \tau ,\varDelta \theta ,\varDelta \kappa ). \end{aligned}$$
(44a)

along the search direction defined by the linear system [81, Eqn. 9]

$$\begin{aligned} \begin{bmatrix}0 &{}\quad +{\mathbf {M}}^{T} &{}\quad -{\mathbf {c}}&{}\quad -r_{p}\\ -{\mathbf {M}}&{}\quad 0 &{}\quad +{\mathbf {b}}&{} \quad -r_{d}\\ +{\mathbf {c}}^{T} &{}\quad -{\mathbf {b}}^{T} &{}\quad 0 &{}\quad -r_{c}\\ +r_{p}^{T} &{}\quad +r_{d}^{T} &{}\quad +r_{c} &{}\quad 0 \end{bmatrix}\begin{bmatrix}\varDelta x\\ \varDelta y\\ \varDelta \tau \\ \varDelta \theta \end{bmatrix}+\begin{bmatrix}\varDelta s\\ 0\\ \varDelta \kappa \\ 0 \end{bmatrix}&=0,\end{aligned}$$
(45a)
$$\begin{aligned} s+\varDelta s+\nabla ^{2}F(w)\varDelta x+\mu ^{+}\nabla F(x)&=0,\end{aligned}$$
(45b)
$$\begin{aligned} \kappa +\varDelta \kappa +(\kappa /\tau )\varDelta \tau -\mu ^{+}\tau ^{-1}&=0. \end{aligned}$$
(45c)

Here, F is the usual self-concordant barrier function on \({\mathcal {C}}\)

$$\begin{aligned} F([x_{0};x_{1};\mathrm {svec}\,(X_{1});\ldots ;\mathrm {svec}\,(X_{\ell })])=&-\log \left( \frac{1}{2}x_{0}^{2}-\frac{1}{2}x_{1}^{T}x_{1}\right) -\sum _{j=1}^{\ell }\log \det (X_{j}) \end{aligned}$$
(46)

and \(w\in \mathrm {Int}({\mathcal {C}})\) is the unique scaling point satisfying \(\nabla ^{2}F(w)x=s\), which can be computed from x and s in closed-form. The following iteration bound is an immediate consequence of [78, Theorem 6.4]; see also [79, Theorem 5.1].

Lemma 13

(Short-Step Method) The sequence in (44) arrives at an iterate \((x,y,s,\tau ,\theta ,\kappa )\) satisfying the conditions of Lemma 12 with \(\gamma =9/10\) in at most \(O(\sqrt{\nu }\log (1/\epsilon ))\) iterations.

The cost of each interior-point iteration is dominated by the cost of computing the search direction in (45). Using elementary but tedious linear algebra, we can show that if

$$\begin{aligned} ({\mathbf {M}}{\mathbf {D}}^{-1}{\mathbf {M}}^{T})\begin{bmatrix}v_{1}&v_{2}&v_{3}\end{bmatrix}=\begin{bmatrix}0&-{\mathbf {b}}&r_{p}\end{bmatrix}-{\mathbf {M}}{\mathbf {D}}^{-1}\begin{bmatrix}d&{\mathbf {c}}&r_{d}\end{bmatrix} \end{aligned}$$
(47a)

where \({\mathbf {D}}=\nabla ^{2}F(w)\) and \(d=-s-\mu ^{+}\nabla F(x)\), and

$$\begin{aligned} \begin{bmatrix}u_{1}&u_{2}&u_{3}\end{bmatrix}={\mathbf {D}}^{-1}(\begin{bmatrix}d&c&r_{d}\end{bmatrix}+{\mathbf {M}}^{T}\begin{bmatrix}v_{1}&v_{2}&v_{3}\end{bmatrix}), \end{aligned}$$
(47b)

then

$$\begin{aligned} \left( \begin{bmatrix}-{\mathbf {D}}_{0} &{} -r_{c}\\ r_{c} &{} 0 \end{bmatrix}-\begin{bmatrix}{\mathbf {c}}&{} r_{d}\\ -{\mathbf {b}}&{} r_{p} \end{bmatrix}^{T}\begin{bmatrix}u_{2} &{} u_{3}\\ v_{2} &{} v_{3} \end{bmatrix}\right) \begin{bmatrix}\varDelta \tau \\ \varDelta \theta \end{bmatrix}&=\begin{bmatrix}-d_{0}\\ 0 \end{bmatrix}-\begin{bmatrix}{\mathbf {c}}&{} r_{d}\\ -{\mathbf {b}}&{} r_{p} \end{bmatrix}^{T}\begin{bmatrix}u_{1}\\ v_{1} \end{bmatrix},\end{aligned}$$
(47c)
$$\begin{aligned} \begin{bmatrix}\varDelta x\\ \varDelta y \end{bmatrix}&=\begin{bmatrix}u_{1}\\ v_{1} \end{bmatrix}-\begin{bmatrix}u_{1} &{} u_{2}\\ v_{1} &{} v_{2} \end{bmatrix}\begin{bmatrix}\varDelta \tau \\ \varDelta \theta \end{bmatrix},\end{aligned}$$
(47d)
$$\begin{aligned} \varDelta s&=d-{\mathbf {D}}\varDelta x,\end{aligned}$$
(47e)
$$\begin{aligned} \varDelta \kappa&=d_{0}-{\mathbf {D}}_{0}\varDelta \tau , \end{aligned}$$
(47f)

where \({\mathbf {D}}_{0}=\kappa /\tau \) and \(d_{0}=-\kappa +\mu ^{+}\tau ^{-1}\). Hence, the cost of computing the search direction is dominated by the cost of solving the normal equation for three different right-hand sides. Here, the normal matrix is written

$$\begin{aligned} {\mathbf {M}}{\mathbf {D}}^{-1}{\mathbf {M}}^{T}=\mathrm {diag}(W_{1}\otimes _{s}W_{1},\ldots ,W_{\ell } \otimes _{s}W_{\ell })+\begin{bmatrix}{\mathbf {A}}\\ {\mathbf {N}}\end{bmatrix}^{T}(w_{1}w_{1}^{T}+\sigma I)\begin{bmatrix}{\mathbf {A}}\\ {\mathbf {N}}\end{bmatrix}, \end{aligned}$$

where \(\sigma =\frac{1}{2}(w_{0}^{2}-w_{1}^{T}w_{1})>0\) and \(\otimes _{s}\) denotes the symmetric Kronecker product [82] implicitly defined to satisfy

$$\begin{aligned} (A\otimes _{s}B)\mathrm {svec}\,(X)=\frac{1}{2}\mathrm {svec}\,(AXB^{T}+BXA^{T})\qquad \text {for all }X=X^{T}. \end{aligned}$$

Under the hypothesis on \({\mathbf {A}}\) stated in Theorem 5, the normal matrix satisfies the assumptions of Lemma 5, and can therefore be solved in linear O(n) time and memory.

Proof of Theorem 5

Combining Lemmas 12 and 13 shows that the desired \(\epsilon \)-accurate, \(\epsilon \)-feasible iterate is obtained after \(O(\sqrt{\nu }\log (1/\epsilon ))\) interior-point iterations. At each iteration we perform the following steps: (1) compute the scaling point w; (2) solve the normal equation (47a) for three right-hand sides; (3) back-substitute (47b)–(47f) for the search direction and take the step in (44). Note from the proof of Lemma 5 that the matrix \([{\mathbf {A}};{\mathbf {N}}]\) has at most \(O(\omega ^{2}n)\) rows under Assumption 1, and therefore \(\mathrm {nnz}\,({\mathbf {M}})=O(\omega ^{4}n)\) under the hypothesis of Theorem 5. Below, we show that the cost of each step is bounded by \(O(\omega ^{6}n)\) time and \(O(\omega ^{4}n)\) memory.

Scaling point We partition \(x=[x_{0};x_{1};\mathrm {svec}\,(X_{1});\ldots ;\mathrm {svec}\,(X_{\ell })]\) and similarly for s. Then, the scaling point w is given in closed-form [61, Section 5]

$$\begin{aligned}&\begin{bmatrix}u_{0}\\ u_{1} \end{bmatrix}=2^{-3/4}\begin{bmatrix}1 &{} 1\\ -s_{1}/\Vert s_{1}\Vert &{} s_{1}/\Vert s_{1}\Vert \end{bmatrix}\begin{bmatrix}(s_{0}-\Vert s_{1}\Vert )^{1/2}\\ (s_{0}+\Vert s_{1}\Vert )^{1/2} \end{bmatrix},\quad \begin{bmatrix}v_{0}\\ v_{1} \end{bmatrix}=\begin{bmatrix}u_{0} &{} u_{1}^{T}\\ u_{1} &{} \frac{1}{2}(u_{0}^{2}-u_{1}^{T}u_{1})I \end{bmatrix}\begin{bmatrix}x_{0}\\ x_{1} \end{bmatrix},\\&\begin{bmatrix}w_{0}\\ w_{1} \end{bmatrix}=2^{-1/4}\begin{bmatrix}u_{0} &{} u_{1}^{T}\\ u_{1} &{} \frac{1}{2}(u_{0}^{2}-u_{1}^{T}u_{1})I \end{bmatrix}\begin{bmatrix}1 &{} 1\\ -v_{1}/\Vert v_{1}\Vert &{} v_{1}/\Vert v_{1}\Vert \end{bmatrix}\begin{bmatrix}(v_{0}-\Vert v_{1}\Vert )^{-1/2}\\ (v_{0}+\Vert v_{1}\Vert )^{-1/2} \end{bmatrix},\\&W_{j}=S_{j}^{1/2}(S_{j}^{1/2}X_{j}S_{j}^{1/2})^{-1/2}S_{j}^{1/2}\qquad \text {for all }j\in \{1,\ldots ,\ell \}. \end{aligned}$$

Noting that \(\mathrm {nnz}\,(w_{1})\le O(\omega ^{2}n)\), \(\ell \le n\) and each \(W_{j}\) is at most \(\omega \times \omega \), the cost of forming \(w=[w_{0};w_{1};\mathrm {svec}\,(W_{1});\ldots ;\mathrm {svec}\,(W_{\ell })]\) is at most \(O(\omega ^{3}n)\) time and \(O(\omega ^{2}n)\) memory. Also, since

$$\begin{aligned} {\mathbf {D}}=\nabla ^{2}F(w)=\mathrm {diag}\left( \begin{bmatrix}w_{0} &{}\quad w_{1}^{T}\\ w_{1} &{}\quad \frac{1}{2}(w_{0}^{2}-w_{1}^{T}w_{1})I \end{bmatrix},W_{1}\otimes _{s}W_{1},\ldots ,W_{\ell }\otimes _{s}W_{\ell }\right) ^{-1}, \end{aligned}$$

the cost of each matrix-vector product with \({\mathbf {D}}\) and \({\mathbf {D}}^{-1}\) is also \(O(\omega ^{3}n)\) time and \(O(\omega ^{2}n)\) memory.

Normal equation The cost of matrix-vector products with \({\mathbf {M}}\) and \({\mathbf {M}}^{T}\) is \(\mathrm {nnz}\,({\mathbf {M}})=O(\omega ^{4}n)\) time and memory. Using Lemma 5, we form the right-hand sides and solve the three normal equations in (47a) in \(O(\omega ^{6}n)\) time and \(O(\omega ^{4}n)\) memory.

Back-substitution The cost of back substituting (47b)–(47f) and making the step (44) is dominated by matrix-vector products with \({\mathbf {D}}\), \({\mathbf {D}}^{-1}\), \({\mathbf {M}}\), and \({\mathbf {M}}^{T}\) at \(O(\omega ^{4}n)\) time and memory.\(\square \)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, R.Y., Lavaei, J. Sparse semidefinite programs with guaranteed near-linear time complexity via dualized clique tree conversion. Math. Program. (2020). https://doi.org/10.1007/s10107-020-01516-y

Download citation

Mathematics Subject Classification

  • 90C22 Semidefinite programming
  • 90C35 Programming involving graphs or networds
  • 90C51 Interior-point methiods
  • 90C06 Large-scale problems in mathematica programming