Skip to main content
SpringerLink
Log in

Comparison of K-means Clustering Initialization Approaches with Brute-Force Initialization

  • Chapter
  • First Online:
Advanced Computing and Systems for Security

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 567))

Abstract

Data clustering is a basic data mining discipline that has been in center of interest of many research groups. This paper describes the formulation of the basic NP-hard optimization problem in data clustering which is approximated by many heuristic methods. The famous k-means clustering algorithm and its initialization is of a particular interest in this paper. A summary of the k-means variants and various initialization strategies is presented. Many initialization heuristics tend to search only through a fraction of the initial centroid space. The final clustering result is usually compared only to some other heuristic strategy. In this paper we compare the result to the solution provided by a brute-force experiment. Many instances of the k-means can be executed in parallel on the high performance computing infrastructure, which makes brute-force search for the best initial centroids possible. Solutions obtained by exact solvers [2, 11] of the clustering problem are used for verification of the brute-force approach. We present progress of the function optimization during the experiment for several benchmark data sets, including sparse document-term matrices.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aloise, D., Deshpande, A., Hansen, P., Popat, P.: Np-hardness of euclidean sum-of-squares clustering. Mach. Learn. 75(2), 245–248 (2009)

    Article  Google Scholar 

  2. Aloise, D., Hansen, P., Liberti, L.: An improved column generation algorithm for minimum sum-of-squares clustering. Math. Program. 131(1–2), 195–220 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  3. Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1027–1035. Society for Industrial and Applied Mathematics (2007)

    Google Scholar 

  4. Bagirov, A.M.: Modified global k-means algorithm for minimum sum-of-squares clustering problems. Pattern Recogn. 41(10), 3192–3199 (2008)

    Article  MATH  Google Scholar 

  5. Bagirov, A.M., Ugon, J., Webb, D.: Fast modified global k-means algorithm for incremental cluster construction. Pattern Recogn. 44(4), 866–876 (2011)

    Article  MATH  Google Scholar 

  6. Bradley, P.S., Fayyad, U.M.: Refining initial points for k-means clustering. In: ICML. vol. 98, pp. 91–99. Citeseer (1998)

    Google Scholar 

  7. Brusco, M.J.: A repetitive branch-and-bound procedure for minimum within-cluster sums of squares partitioning. Psychometrika 71(2), 347–363 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  8. Celebi, M.E., Kingravi, H.A., Vela, P.A.: A comparative study of efficient initialization methods for the k-means clustering algorithm. Expert Syst. Appl. 40(1), 200–210 (2013)

    Article  Google Scholar 

  9. Deng, G., Tao, J., Zhou, M., Xu, Y.: Improved k-means algorithm with better clustering centers based on density and variance. In: Industrial, Mechanical and Manufacturing Science: Proceedings of the 2014 International Conference on Industrial, Mechanical and Manufacturing Science (ICIMMS 2014), 12–13 June 2014, Tianjin, China. vol. 1, p. 147. CRC Press (2015)

    Google Scholar 

  10. Dhillon, I.S., Modha, D.S.: Concept decompositions for large sparse text data using clustering. Mach. Learn. 42(1–2), 143–175 (2001)

    Article  MATH  Google Scholar 

  11. Du Merle, O., Hansen, P., Jaumard, B., Mladenovic, N.: An interior point algorithm for minimum sum-of-squares clustering. SIAM J. Sci. Comput. 21(4), 1485–1505 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  12. Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Eugen. 7(2), 179–188 (1936)

    Article  Google Scholar 

  13. Forgy, E.W.: Cluster analysis of multivariate data: efficiency versus interpretability of classifications. Biometrics 21, 768–769 (1965)

    Google Scholar 

  14. Hansen, P., Ngai, E., Cheung, B.K., Mladenovic, N.: Analysis of global k-means, an incremental heuristic for minimum sum-of-squares clustering. J. Classif. 22(2), 287–310 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  15. Hartigan, J.A., Wong, M.A.: Algorithm as 136: a k-means clustering algorithm. J. Roy. Stat. Soc. Ser. C (Appl. Stat.) 28(1), 100–108 (1979)

    MATH  Google Scholar 

  16. Hornik, K., Feinerer, I., Kober, M., Buchta, C.: Spherical k-means clustering. J. Stat. Softw. 50(10), 1–22 (2012)

    Article  Google Scholar 

  17. Jain, A.K.: Data clustering: 50 years beyond k-means. Pattern Recogn. Lett. 31(8), 651–666 (2010)

    Article  Google Scholar 

  18. Likas, A., Vlassis, N., Verbeek, J.J.: The global k-means clustering algorithm. Pattern Recogn. 36(2), 451–461 (2003)

    Article  Google Scholar 

  19. Liu, H., Fang, C., Wu, Y., Xu, K., Dai, T.: Improved k-means algorithm with the pretreatment of PCA dimension reduction. Int. J. Hybrid Inf. Technol. 8(6), 195–204 (2015)

    Article  Google Scholar 

  20. Lloyd, S.: Least squares quantization in PCM. IEEE Trans. Inf. Theory 28(2), 129–137 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  21. MacQueen, J., et al.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. vol. 1, pp. 281–297, Oakland, CA, USA (1967)

    Google Scholar 

  22. R Core Team R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2015). http://www.R-project.org/

  23. Ruspini, E.H.: Numerical methods for fuzzy clustering. Inf. Sci. 2(3), 319–350 (1970)

    Article  MATH  Google Scholar 

  24. Steinhaus, H.: Sur la division des corp materiels en parties. Bull. Acad. Polon. Sci 1(804), 801 (1956)

    Google Scholar 

  25. Steinley, D.: K-means clustering: a half-century synthesis. Br. J. Math. Stat. Psychol. 59(1), 1–34 (2006)

    Article  MathSciNet  Google Scholar 

  26. West, L.J., Hankin, R.K.: Set partitions in r. J. Stat. Softw. 23(02) (2007)

    Google Scholar 

Download references

Acknowledgements

This work was supported by The Ministry of Education, Youth and Sports from the National Programme of Sustainability (NPU II) project ‘IT4Innovations excellence in science—LQ1602’ and co-financed by the internal grant agency of VŠB—Technical University of Ostrava, Czech Republic, under the projects no. SP2016/179 ‘HPC Usage for Transport Optimisation based on Dynamic Routing II’.

Author information

Authors and Affiliations

  1. IT4Innovations, VŠB – Technical University of Ostrava, 17. Listopadu 15/2172, 708 33, Poruba, Ostrava, Czech Republic

    Martin Golasowski, Jan Martinovič & Kateřina Slaninová

Authors
  1. Martin Golasowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Martinovič
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kateřina Slaninová
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin Golasowski .

Editor information

Editors and Affiliations

  1. A.K. Choudhury School of Information Tec, Associate Prof., University of Calcutta A.K. Choudhury School of Information Tec, Kolkata, West Bengal, India

    Rituparna Chaki

  2. Faculty of Computer Science, Bialystok University of Technology Faculty of Computer Science, Bialystok, Poland

    Khalid Saeed

  3. DAIS—Università Ca' Foscari, Full Professor of Computer Science DAIS—Università Ca' Foscari, Mestre, Venice, Venezia, Italy

    Agostino Cortesi

  4. Computer Science & Engineering, University of Calcutta Computer Science & Engineering, Kolkata, West Bengal, India

    Nabendu Chaki

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Nature Singapore Pte Ltd.

About this chapter

Cite this chapter

Golasowski, M., Martinovič, J., Slaninová, K. (2017). Comparison of K-means Clustering Initialization Approaches with Brute-Force Initialization. In: Chaki, R., Saeed, K., Cortesi, A., Chaki, N. (eds) Advanced Computing and Systems for Security. Advances in Intelligent Systems and Computing, vol 567. Springer, Singapore. https://doi.org/10.1007/978-981-10-3409-1_7

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-3409-1_7

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-3408-4

  • Online ISBN: 978-981-10-3409-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation