Skip to main content
SpringerLink
Log in

A Convex Approach to K-Means Clustering and Image Segmentation

  • Conference paper
  • First Online:
Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR 2017)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10746))

Abstract

A new convex formulation of data clustering and image segmentation is proposed, with fixed number K of regions and possible penalization of the region perimeters. So, this problem is a spatially regularized version of the K-means problem, a.k.a. piecewise constant Mumford–Shah problem. The proposed approach relies on a discretization of the search space; that is, a finite number of candidates must be specified, from which the K centroids are determined. After reformulation as an assignment problem, a convex relaxation is proposed, which involves a kind of \(l_{1,\infty }\) norm ball. A splitting of it is proposed, so as to avoid the costly projection onto this set. Some examples illustrate the efficiency of the approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The number of regions is actually at most K, and not exactly K, because some regions \(\mathrm {\Omega }_k\) could be empty. This is never the case in practical applications.

  2. 2.

    We assume symmetric boundary conditions, so the boundary of the domain \(\mathrm {\Omega }\) is not counted in the perimeter.

  3. 3.

    In this paper, we make an abuse of the terms \(l_{1,\infty }\) norm and ball: the elements of z are nonnegative, so there is no need to take their absolute values.

References

  1. Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM Comput. Surv. 31, 264–323 (1999)

    Article  Google Scholar 

  2. Gan, G., Ma, C., Wu, J.: Data Clustering: Theory, Algorithms, and Applications. ASA-SIAM Series on Statistics and Applied Probability. SIAM, Philadelphia (2007)

    Book  MATH  Google Scholar 

  3. Steinley, D.: K-means clustering: a half-century synthesis. Br. J. Math. Stat. Psychol. 59(1), 1–34 (2006)

    Article  MathSciNet  Google Scholar 

  4. Wu, X.: Optimal quantization by matrix searching. J. Algorithms 12(4), 663–673 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  5. Soong, F.K., Juang, B.H.: Optimal quantization of LSP parameters. IEEE Trans. Speech Audio Process. 1(1), 15–24 (1993)

    Article  Google Scholar 

  6. Aloise, D., Deshpande, A., Hansen, P., Popat, P.: NP-hardness of Euclidean sum-of-squares clustering. Mach. Learn. 75(2), 245–248 (2009)

    Article  MATH  Google Scholar 

  7. Mahajan, M., Nimbhorkar, P., Varadarajan, K.: The planar k-means problem is NP-hard. Theor. Comput. Sci. 442, 13–21 (2012). Special Issue on the Workshop on Algorithms and Computation (WALCOM 2009)

    Article  MathSciNet  MATH  Google Scholar 

  8. Brun, L., Trémeau, A.: Color quantization. In: Digital Color Imaging Handbook, pp. 589–638. CRC Press (2012)

    Google Scholar 

  9. Celebi, M.E.: Improving the performance of k-means for color quantization. Image Vis. Comput. 29(4), 260–271 (2011)

    Article  Google Scholar 

  10. Cremers, D., Rousson, M., Deriche, R.: A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape. Int. J. Comput. Vis. 72, 195–215 (2007)

    Article  Google Scholar 

  11. Bar, L., Chan, T.F., Chung, G., Jung, M., Kiryati, N., Sochen, N., Vese, L.A.: Mumford and Shah model and its applications to image segmentation and image restoration. In: Scherzer, O. (ed.) Handbook of Mathematical Methods in Imaging, pp. 1095–1157. Springer, New York (2015). https://doi.org/10.1007/978-0-387-92920-0_25

    Google Scholar 

  12. Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42, 577–685 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  13. Condat, L.: Discrete total variation: new definition and minimization. SIAM J. Imaging Sci. 10(3), 1258–1290 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  14. Chambolle, A., Cremers, D., Pock, T.: A convex approach to minimal partitions. SIAM J. Imaging Sci. 5(4), 1113–1158 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  15. Pustelnik, N., Condat, L.: Proximity operator of a sum of functions; application to depth map estimation. IEEE Sig. Process. Lett. 24(12), 1827–1831 (2017)

    Article  Google Scholar 

  16. Yuan, J., Bae, E., Tai, X.-C., Boykov, Y.: A continuous max-flow approach to potts model. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 379–392. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15567-3_28

    Chapter  Google Scholar 

  17. Zach, C., Häne, C., Pollefeys, M.: What is optimized in convex relaxations for multilabel problems: connecting discrete and continuously inspired MAP inference. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 157–170 (2014)

    Article  Google Scholar 

  18. Lloyd, S.: Least squares quantization in PCM. IEEE Trans. Inform. Theory 28(2), 129–136 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  19. Brown, E.S., Chan, T.F., Bresson, X.: Completely convex formulation of the Chan-Vese image segmentation model. Int. J. Comput. Vis. 98(1), 103–121 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  20. Bae, E., Yuan, J., Tai, X.-C.: Simultaneous convex optimization of regions and region parameters in image segmentation models. In: Breuß, M., Bruckstein, A., Maragos, P. (eds.) Innovations for Shape Analysis: Models and Algorithms. Mathematics and Visualization, pp. 421–438. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-34141-0_19

    Chapter  Google Scholar 

  21. Bae, E., Tai, X.-C.: Efficient global minimization methods for image segmentation models with four regions. J. Math. Imaging Vis. 51(1), 71–97 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  22. Pock, T., Cremers, D., Bischof, H., Chambolle, A.: Global solutions of variational models with convex regularization. SIAM J. Imaging Sci. 3(4), 1122–1145 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  23. Condat, L.: Fast projection onto the simplex and the l1 ball. Math. Program. Ser. A 158(1), 575–585 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  24. Quattoni, A., Carreras, X., Collins, M., Darrell, T.: An efficient projection for l1, \(\infty \) regularization. In: Proceedings of ICML, Montreal, Canada, June 2009, pp. 857–864 (2009)

    Google Scholar 

  25. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011). https://doi.org/10.1007/978-3-319-48311-5

    Book  MATH  Google Scholar 

  26. Yuan, J., Bae, E., Boykov, Y., Tai, X.-C.: A continuous max-flow approach to minimal partitions with label cost prior. In: Bruckstein, A.M., ter Haar Romeny, B.M., Bronstein, A.M., Bronstein, M.M. (eds.) SSVM 2011. LNCS, vol. 6667, pp. 279–290. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-24785-9_24

    Chapter  Google Scholar 

  27. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  28. Condat, L.: A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  29. Reese, J.: Solution methods for the p-median problem: an annotated bibliography. Networks 48(3), 125–142 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  30. Balinski, M.L.: On finding integer solutions to linear programs. In: Proceedings of the I.B.M. Scientific Computing Symposium on Combinatorial Problems, pp. 225–248 (1966)

    Google Scholar 

  31. Li, S., Svensson, O.: Approximating k-median via pseudo-approximation. In: Proceedings of the forty-Fifth Annual ACM Symposium on Theory of Computing (STOC 2013), Palo Alto, California, USA, June 2013, pp. 901–910 (2013)

    Google Scholar 

  32. Van der Laan, M., Pollard, K., Bryan, J.: A new partitioning around medoids algorithm. J. Stat. Comput. Simul. 73(8), 575–584 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  33. Peng, J., Xia, Y.: A new theoretical framework for K-means-type clustering. In: Chu, W., Young Lin, T. (eds.) Foundations and Advances in Data Mining. Studies in Fuzziness and Soft Computing, vol. 180, pp. 79–96. Springer, Heidelberg (2005). https://doi.org/10.1007/11362197_4

    Chapter  Google Scholar 

  34. Peng, J., Wei, Y.: Approximating K-means-type clustering via semidefinite programming. SIAM J. Optim. 18(1), 186–205 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  35. Awasthi, P., Bandeira, A.S., Charikar, M., Krishnaswamy, R., Villar, S., Ward, R.: Relax, no need to round: integrality of clustering formulations. In: Proceedings of the 2015 Conference on Innovations in Theoretical Computer Science (ITCS), Rehovot, Israel, January 2015, pp. 191–200 (2015)

    Google Scholar 

  36. Pelckmans, K., De Brabanter, J., Suykens, J.A.K., De Moor, B.: Convex clustering shrinkage. In: Proceedings of Workshop on Statistics and Optimization of Clustering Workshop (PASCAL), London, UK, July 2005

    Google Scholar 

  37. Hocking, T., Vert, J.-P., Bach, F., Joulin, A.: Clusterpath: an algorithm for clustering using convex fusion penalties. In: Proceeding of the 28th International Conference on Machine Learning (ICML), Bellevue, USA, June 2011, pp. 745–752 (2011)

    Google Scholar 

  38. Lindsten, F., Ohlsson, H., Ljung, L.: Clustering using sum-of-norms regularization: with application to particle filter output computation. In: Proceedings of Statistical Signal Processing Workshop (SSP), Nice, France, June 2011, pp. 201–204 (2011)

    Google Scholar 

  39. Zhu, C., Xu, H., Leng, C., Yan, S.: Convex optimization procedure for clustering: theoretical revisit. In: Proceedings of NIPS, Montreal, Canada, December 2014, pp. 1619–1627 (2014)

    Google Scholar 

  40. Chi, E.C., Lange, K.: Splitting methods for convex clustering. J. Comput. Graph. Stat. 24(4), 994–1013 (2015)

    Article  MathSciNet  Google Scholar 

  41. Kärkkäinen, I., Fränti, P.: Dynamic local search algorithm for the clustering problem. Technical report A-2002-6, Department of Computer Science, University of Joensuu, Joensuu, Finland (2002)

    Google Scholar 

  42. Barnes, E.S., Sloane, N.J.A.: The optimal lattice quantizer in three dimensions. SIAM J. Algebr. Discret. Methods 4(1), 30–41 (1983)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CNRS, GIPSA-Lab, Univ. Grenoble Alpes, 38000, Grenoble, France

    Laurent Condat

Authors
  1. Laurent Condat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laurent Condat .

Editor information

Editors and Affiliations

  1. Ca’ Foscari University of Venice, Venice, Italy

    Marcello Pelillo

  2. University of York, York, United Kingdom

    Edwin Hancock

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Condat, L. (2018). A Convex Approach to K-Means Clustering and Image Segmentation. In: Pelillo, M., Hancock, E. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2017. Lecture Notes in Computer Science(), vol 10746. Springer, Cham. https://doi.org/10.1007/978-3-319-78199-0_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-78199-0_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-78198-3

  • Online ISBN: 978-3-319-78199-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation