A Convex Relaxation Approach to the Affine Subspace Clustering Problem

  • Francesco Silvestri
  • Gerhard Reinelt
  • Christoph Schnörr
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9358)


Prototypical data clustering is known to suffer from poor initializations. Recently, a semidefinite relaxation has been proposed to overcome this issue and to enable the use of convex programming instead of ad-hoc procedures. Unfortunately, this relaxation does not extend to the more involved case where clusters are defined by parametric models, and where the computation of means has to be replaced by parametric regression. In this paper, we provide a novel convex relaxation approach to this more involved problem class that is relevant to many scenarios of unsupervised data analysis. Our approach applies, in particular, to data sets where assumptions of model recovery through sparse regularization, like the independent subspace model, do not hold. Our mathematical analysis enables to distinguish scenarios where the relaxation is tight enough and scenarios where the approach breaks down.



Authors gratefully acknowledge support by the DFG, grant GRK 1653.


  1. 1.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)CrossRefzbMATHGoogle Scholar
  2. 2.
    Berman, A., Shaked-Monderer, N.: Completely Positive Matrices. World Scientific Publishing, New York (2003)CrossRefzbMATHGoogle Scholar
  3. 3.
    Carin, L., Baraniuk, R., Cevher, V., Dunson, V., Jordan, M., Sapiro, G., Wakin, M.: Learning low-dimensional signal models. IEEE Signal Proc. Mag. 28(2), 39–51 (2011)CrossRefGoogle Scholar
  4. 4.
    Chen, G., Lerman, G.: Foundations of a multi-way spectral clustering framework for hybrid linear modeling. Found. Comp. Math. 9, 517–558 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Dickinson, P., Gijben, L.: On the computational complexity of membership problems for the completely positive cone and its dual. Comput. Optim. Appl. 57(2), 403–415 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Patt. Anal. Mach. Intell. 35(11), 2765–2781 (2013)CrossRefGoogle Scholar
  7. 7.
    Grossmann, I.E., Lee, S.: Generalized convex disjunctive programming: nonlinear convex hull relaxation. Comput. Optim. Appl. 26(1), 83–100 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Hofman, T., Buhmann, J.: Pairwise data clustering by deterministic annealing. IEEE Trans. Patt. Anal. Mach. Intell. 19(1), 1–14 (1997)CrossRefGoogle Scholar
  9. 9.
    Ma, Y., Yang, A., Derksen, H., Fossum, R.: Estimation of subspace arrangements with applications in modeling and segmenting mixed data. SIAM Rev. 50(3), 413–458 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    du Merle, O., Hansen, P., Jaumard, B., Mladenović, N.: An interior points algorithm for minimum sum-of-squares clustering. SIAM J. Sci. Comput. 21(4), 1485–1505 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Peng, J., Wei, Y.: Approximating \({K}\)-means-type clustering via semidefinite programming. SIAM J. Optim. 18(1), 186–205 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Rockafellar, R., Wets, R.J.B.: Variational Analysis, 2nd edn. Springer, New York (2009)zbMATHGoogle Scholar
  13. 13.
    Rose, K.: Deterministic annealing for clustering, compression, classification, regression, and related optimization problems. Proc. IEEE 86(11), 2210–2239 (1998)CrossRefGoogle Scholar
  14. 14.
    Singh, V., Mukherjee, L., Peng, J., Xu, J.: Ensemble clustering using semidefinite programming with applications. Mach. Learn. 79(1–2), 177–200 (2010)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Toh, K.C., Todd, M.J., Tütüncü, R.H.: SDPT3 – a MATLAB software package for semidefinite programming, December 1996Google Scholar
  16. 16.
    Tütüncü, R.H., Toh, K.C., Todd, M.J.: Solving semidefinite-quadratic-linear programs using SDPT3. Math. Program. 95(2), 189–217 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Xing, E., Jordan, M.: On Semidefinite relaxation for normalized k-cut and connections to spectral clustering. Technical report UCB/CSD-03-1265, EECS Department, University of California, Berkeley, June 2003Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Open Access This chapter is distributed under the terms of the Creative Commons Attribution Noncommercial License, which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Authors and Affiliations

  • Francesco Silvestri
    • 1
    • 2
  • Gerhard Reinelt
    • 1
  • Christoph Schnörr
    • 2
  1. 1.Discrete and Combinatorial Optimization GroupHeidelberg UniversityHeidelbergGermany
  2. 2.IPA and HCIHeidelberg UniversityHeidelbergGermany

Personalised recommendations