Advertisement

A Convex Approach to K-Means Clustering and Image Segmentation

  • Laurent CondatEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10746)

Abstract

A new convex formulation of data clustering and image segmentation is proposed, with fixed number K of regions and possible penalization of the region perimeters. So, this problem is a spatially regularized version of the K-means problem, a.k.a. piecewise constant Mumford–Shah problem. The proposed approach relies on a discretization of the search space; that is, a finite number of candidates must be specified, from which the K centroids are determined. After reformulation as an assignment problem, a convex relaxation is proposed, which involves a kind of \(l_{1,\infty }\) norm ball. A splitting of it is proposed, so as to avoid the costly projection onto this set. Some examples illustrate the efficiency of the approach.

Keywords

Image segmentation Piecewise constant Mumford–Shah problem K-means Convex relaxation 

References

  1. 1.
    Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM Comput. Surv. 31, 264–323 (1999)CrossRefGoogle Scholar
  2. 2.
    Gan, G., Ma, C., Wu, J.: Data Clustering: Theory, Algorithms, and Applications. ASA-SIAM Series on Statistics and Applied Probability. SIAM, Philadelphia (2007)CrossRefzbMATHGoogle Scholar
  3. 3.
    Steinley, D.: K-means clustering: a half-century synthesis. Br. J. Math. Stat. Psychol. 59(1), 1–34 (2006)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Wu, X.: Optimal quantization by matrix searching. J. Algorithms 12(4), 663–673 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Soong, F.K., Juang, B.H.: Optimal quantization of LSP parameters. IEEE Trans. Speech Audio Process. 1(1), 15–24 (1993)CrossRefGoogle Scholar
  6. 6.
    Aloise, D., Deshpande, A., Hansen, P., Popat, P.: NP-hardness of Euclidean sum-of-squares clustering. Mach. Learn. 75(2), 245–248 (2009)CrossRefzbMATHGoogle Scholar
  7. 7.
    Mahajan, M., Nimbhorkar, P., Varadarajan, K.: The planar k-means problem is NP-hard. Theor. Comput. Sci. 442, 13–21 (2012). Special Issue on the Workshop on Algorithms and Computation (WALCOM 2009)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Brun, L., Trémeau, A.: Color quantization. In: Digital Color Imaging Handbook, pp. 589–638. CRC Press (2012)Google Scholar
  9. 9.
    Celebi, M.E.: Improving the performance of k-means for color quantization. Image Vis. Comput. 29(4), 260–271 (2011)CrossRefGoogle Scholar
  10. 10.
    Cremers, D., Rousson, M., Deriche, R.: A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape. Int. J. Comput. Vis. 72, 195–215 (2007)CrossRefGoogle Scholar
  11. 11.
    Bar, L., Chan, T.F., Chung, G., Jung, M., Kiryati, N., Sochen, N., Vese, L.A.: Mumford and Shah model and its applications to image segmentation and image restoration. In: Scherzer, O. (ed.) Handbook of Mathematical Methods in Imaging, pp. 1095–1157. Springer, New York (2015).  https://doi.org/10.1007/978-0-387-92920-0_25 Google Scholar
  12. 12.
    Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42, 577–685 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Condat, L.: Discrete total variation: new definition and minimization. SIAM J. Imaging Sci. 10(3), 1258–1290 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Chambolle, A., Cremers, D., Pock, T.: A convex approach to minimal partitions. SIAM J. Imaging Sci. 5(4), 1113–1158 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Pustelnik, N., Condat, L.: Proximity operator of a sum of functions; application to depth map estimation. IEEE Sig. Process. Lett. 24(12), 1827–1831 (2017)CrossRefGoogle Scholar
  16. 16.
    Yuan, J., Bae, E., Tai, X.-C., Boykov, Y.: A continuous max-flow approach to potts model. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 379–392. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15567-3_28 CrossRefGoogle Scholar
  17. 17.
    Zach, C., Häne, C., Pollefeys, M.: What is optimized in convex relaxations for multilabel problems: connecting discrete and continuously inspired MAP inference. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 157–170 (2014)CrossRefGoogle Scholar
  18. 18.
    Lloyd, S.: Least squares quantization in PCM. IEEE Trans. Inform. Theory 28(2), 129–136 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Brown, E.S., Chan, T.F., Bresson, X.: Completely convex formulation of the Chan-Vese image segmentation model. Int. J. Comput. Vis. 98(1), 103–121 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Bae, E., Yuan, J., Tai, X.-C.: Simultaneous convex optimization of regions and region parameters in image segmentation models. In: Breuß, M., Bruckstein, A., Maragos, P. (eds.) Innovations for Shape Analysis: Models and Algorithms. Mathematics and Visualization, pp. 421–438. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-34141-0_19 CrossRefGoogle Scholar
  21. 21.
    Bae, E., Tai, X.-C.: Efficient global minimization methods for image segmentation models with four regions. J. Math. Imaging Vis. 51(1), 71–97 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Pock, T., Cremers, D., Bischof, H., Chambolle, A.: Global solutions of variational models with convex regularization. SIAM J. Imaging Sci. 3(4), 1122–1145 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Condat, L.: Fast projection onto the simplex and the l1 ball. Math. Program. Ser. A 158(1), 575–585 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Quattoni, A., Carreras, X., Collins, M., Darrell, T.: An efficient projection for l1, \(\infty \) regularization. In: Proceedings of ICML, Montreal, Canada, June 2009, pp. 857–864 (2009)Google Scholar
  25. 25.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011).  https://doi.org/10.1007/978-3-319-48311-5 CrossRefzbMATHGoogle Scholar
  26. 26.
    Yuan, J., Bae, E., Boykov, Y., Tai, X.-C.: A continuous max-flow approach to minimal partitions with label cost prior. In: Bruckstein, A.M., ter Haar Romeny, B.M., Bronstein, A.M., Bronstein, M.M. (eds.) SSVM 2011. LNCS, vol. 6667, pp. 279–290. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-24785-9_24 CrossRefGoogle Scholar
  27. 27.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Condat, L.: A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Reese, J.: Solution methods for the p-median problem: an annotated bibliography. Networks 48(3), 125–142 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Balinski, M.L.: On finding integer solutions to linear programs. In: Proceedings of the I.B.M. Scientific Computing Symposium on Combinatorial Problems, pp. 225–248 (1966)Google Scholar
  31. 31.
    Li, S., Svensson, O.: Approximating k-median via pseudo-approximation. In: Proceedings of the forty-Fifth Annual ACM Symposium on Theory of Computing (STOC 2013), Palo Alto, California, USA, June 2013, pp. 901–910 (2013)Google Scholar
  32. 32.
    Van der Laan, M., Pollard, K., Bryan, J.: A new partitioning around medoids algorithm. J. Stat. Comput. Simul. 73(8), 575–584 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Peng, J., Xia, Y.: A new theoretical framework for K-means-type clustering. In: Chu, W., Young Lin, T. (eds.) Foundations and Advances in Data Mining. Studies in Fuzziness and Soft Computing, vol. 180, pp. 79–96. Springer, Heidelberg (2005).  https://doi.org/10.1007/11362197_4 CrossRefGoogle Scholar
  34. 34.
    Peng, J., Wei, Y.: Approximating K-means-type clustering via semidefinite programming. SIAM J. Optim. 18(1), 186–205 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Awasthi, P., Bandeira, A.S., Charikar, M., Krishnaswamy, R., Villar, S., Ward, R.: Relax, no need to round: integrality of clustering formulations. In: Proceedings of the 2015 Conference on Innovations in Theoretical Computer Science (ITCS), Rehovot, Israel, January 2015, pp. 191–200 (2015)Google Scholar
  36. 36.
    Pelckmans, K., De Brabanter, J., Suykens, J.A.K., De Moor, B.: Convex clustering shrinkage. In: Proceedings of Workshop on Statistics and Optimization of Clustering Workshop (PASCAL), London, UK, July 2005Google Scholar
  37. 37.
    Hocking, T., Vert, J.-P., Bach, F., Joulin, A.: Clusterpath: an algorithm for clustering using convex fusion penalties. In: Proceeding of the 28th International Conference on Machine Learning (ICML), Bellevue, USA, June 2011, pp. 745–752 (2011)Google Scholar
  38. 38.
    Lindsten, F., Ohlsson, H., Ljung, L.: Clustering using sum-of-norms regularization: with application to particle filter output computation. In: Proceedings of Statistical Signal Processing Workshop (SSP), Nice, France, June 2011, pp. 201–204 (2011)Google Scholar
  39. 39.
    Zhu, C., Xu, H., Leng, C., Yan, S.: Convex optimization procedure for clustering: theoretical revisit. In: Proceedings of NIPS, Montreal, Canada, December 2014, pp. 1619–1627 (2014)Google Scholar
  40. 40.
    Chi, E.C., Lange, K.: Splitting methods for convex clustering. J. Comput. Graph. Stat. 24(4), 994–1013 (2015)MathSciNetCrossRefGoogle Scholar
  41. 41.
    Kärkkäinen, I., Fränti, P.: Dynamic local search algorithm for the clustering problem. Technical report A-2002-6, Department of Computer Science, University of Joensuu, Joensuu, Finland (2002)Google Scholar
  42. 42.
    Barnes, E.S., Sloane, N.J.A.: The optimal lattice quantizer in three dimensions. SIAM J. Algebr. Discret. Methods 4(1), 30–41 (1983)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.CNRS, GIPSA-LabUniv. Grenoble AlpesGrenobleFrance

Personalised recommendations