Advertisement

Parallel SLINK for big data

  • Poonam GoyalEmail author
  • Sonal Kumari
  • Sumit Sharma
  • Sundar Balasubramaniam
  • Navneet Goyal
Regular Paper
  • 4 Downloads

Abstract

The major strength of hierarchical clustering algorithms is that it allows visual interpretations of clusters through dendrograms. Users can cut the dendrogram at different levels to get desired number of clusters. A major problem with hierarchical algorithms is their quadratic runtime complexity, which limits the amount of data that can be clustered in reasonable amount of time. Also, due to its agglomerative merging process, each iteration depends on the data of all previous iterations, making it difficult to parallelize. Thus, there is a need for an efficient parallel implementation of SLINK algorithm which can scale to big data. We present a parallel SLINK algorithm, sGridSLINK, for shared memory architectures. sGridSLINK produces exactly the same dendrogram as the classical SLINK algorithm. We also present, hGridSLINK, a parallel algorithm which fully exploits a multi-core cluster system. To the best of our knowledge, there is no hybrid parallel algorithm for SLINK available in the literature. The proposed algorithms exploit spatial locality of data to reduce the number of distance calculations. Adaptive gridding is used to counter skewness in data and to ensure load balancing. Extensive experiments are carried out to establish the efficiency and scalability of proposed parallel algorithms. sGridSLINK is approximately 840 times faster than the state-of-the-art algorithm using 55 threads on a 48-core machine on a real dataset having 6 million data points. It also achieves a speedup of 47.93 over the best known sequential SLINK, GridSLINK, on a real dataset using 48 threads on a 48-core machine. hGridSLINK achieves a maximum speedup of 68.26 on a 32-node cluster (\(32\times 4\) processing elements) with respect to GridSLINK. The hGridSLINK algorithm is able to cluster 200 million data points in only 1317 s (less than 22 min). No existing parallel SLINK algorithm is capable of such efficient clustering of Big Data.

Keywords

Parallel clustering algorithms Big data SLINK R-tree Adaptive gridding 

Notes

Acknowledgements

This work was supported by a Research Grant from Department of Electronics and Information Technology (DeitY), Government of India.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

References

  1. 1.
    (2013) Uci Machine Learning Repository. http://archive.ics.uci.edu/ml. Accessed on 13 Oct 2013
  2. 2.
    Aggarwal, C.C., Reddy, C.K.: Data Clustering: Algorithms and Applications, 1st edn. CRC Press, Boca Raton (2013)CrossRefzbMATHGoogle Scholar
  3. 3.
    Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bentley, J.L.: A parallel algorithm for constructing minimum spanning trees. J. Algorithms 1(1), 51–59 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Bertone, S., De Lucia, G., Thomas, P.A.: The recycling of gas and metals in galaxy formation: predictions of a dynamical feedback model. Mon. Not. R. Astron. Soc. 379(3), 1143–1154 (2007)CrossRefGoogle Scholar
  6. 6.
    Bower, R.G., Benson, A.J., Malbon, R.K., Helly, J.C., Frenk, C.S., Baugh, C.M., Cole, S., Lacey, C.G.: Breaking the hierarchy of galaxy formation. Mon. Not. R. Astron. Soc. 370(2), 645–655 (2006)CrossRefGoogle Scholar
  7. 7.
    Brunst, H., Hackenberg, D., Juckeland, G., Rohling, H.: Comprehensive performance tracking with vampir 7. Tools for High Performance Computing, pp. 17–29. Springer, Berlin (2010)Google Scholar
  8. 8.
    Challa, J.S., Goyal, P., Nikhil, S., Mangla, A., Balasubramaniam, S.S., Goyal, N.: Dd-rtree: a dynamic distributed data structure for efficient data distribution among cluster nodes for spatial data mining algorithms. In: 2016 IEEE International Conference on Big Data (Big Data), IEEE Computer Society, Washington DC, USA, pp. 27–36 (2016)Google Scholar
  9. 9.
    Chapman, B., Jost, G., Rvd, P.: Using OpenMP: Portable Shared Memory Parallel Programming (Scientific and Engineering Computation). The MIT Press, Cambridge (2007)Google Scholar
  10. 10.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 3rd edn. The MIT Press, Cambridge (2009)zbMATHGoogle Scholar
  11. 11.
    Dahlhaus, E.: Parallel algorithms for hierarchical clustering and applications to split decomposition and parity graph recognition. J. Algorithms 36(2), 205–240 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Dash, M., Liu, H., Scheuermann, P., Tan, K.L.: Fast hierarchical clustering and its validation. Data Knowl. Eng. 44(1), 109–138 (2003)CrossRefzbMATHGoogle Scholar
  13. 13.
    Dash, M., Petrutiu, S., Scheuermann, P.: ppop: fast yet accurate parallel hierarchical clustering using partitioning. Data Knowl. Eng. 61(3), 563–578 (2007)CrossRefGoogle Scholar
  14. 14.
    De Lucia, G., Blaizot, J.: The hierarchical formation of the brightest cluster galaxies. Mon. Not. R. Astron. Soc. 375, 2–14 (2007)CrossRefGoogle Scholar
  15. 15.
    Du, Z., Lin, F.: A novel parallelization approach for hierarchical clustering. Parallel Comput. 31(5), 523–527 (2005)CrossRefGoogle Scholar
  16. 16.
    Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters a density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, AAAI Press, KDD’96, pp. 226–231 (1996)Google Scholar
  17. 17.
    Fatta, G.D., Pettinger, D.: Dynamic load balancing in parallel kd-tree k-means. In: 2010 10th IEEE International Conference on Computer and Information Technology, IEEE Computer Society, Washington DC, USA, pp. 2478–2485 (2010)Google Scholar
  18. 18.
    Forum, M.P.: Mpi: A Message-passing Interface Standard. University of Tennessee, Knoxville, TN, USA, Technical Report (1994)Google Scholar
  19. 19.
    Fouedjio, F.: A spectral clustering approach for multivariate geostatistical data. Int. J. Data Sci. Anal. 4(4), 301–312 (2017)CrossRefGoogle Scholar
  20. 20.
    Gagolewski, M., Bartoszuk, M., Cena, A.: Genie: a new, fast, and outlier-resistant hierarchical clustering algorithm. Inf. Sci. 363, 8–23 (2016)CrossRefGoogle Scholar
  21. 21.
    Goil, S., Nagesh, H., Choudhary, A.: Efficient and scalable subspace clustering for very large data sets. In: Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, New York, USA, pp. 443–452 (1999)Google Scholar
  22. 22.
    Goyal, P., Kumari, S., Sharma, S., Kishore, V., Goyal, N., Balasubramaniam, S.S.: Spatial locality aware, fast, and scalable slink algorithm for commodity clusters. In: 2016 IEEE International Conference on Cluster Computing (CLUSTER), IEEE Computer Society, Washington DC, USA, pp. 158–159 (2016)Google Scholar
  23. 23.
    Goyal, P., Kumari, S., Sharma, S., Kumar, D., Kishore, V., Balasubramaniam, S., Goyal, N.: A fast, scalable slink algorithm for commodity cluster computing exploiting spatial locality. In: 2016 IEEE 18th International Conference on High Performance Computing and Communications, IEEE Computer Society, Washington DC, USA, pp. 268–275 (2016)Google Scholar
  24. 24.
    Guttman, A.: R-trees: a dynamic index structure for spatial searching. SIGMOD Rec. 14(2), 47–57 (1984)CrossRefGoogle Scholar
  25. 25.
    Hendrix, W., Ali Patwary, M., Agrawal, A., Liao, W.K., Choudhary, A.: Parallel Hierarchical Clustering Code (2012). http://cucis.ece.northwestern.edu/projects/Clustering/. Accessed 10 Aug 2017
  26. 26.
    Hendrix, W., Patwary, M.M.A., Agrawal, A., Liao, W., Choudhary, A.: Parallel hierarchical clustering on shared memory platforms. In: 2012 19th International Conference on High Performance Computing, IEEE Computer Society, Washington DC, USA, pp. 1–9 (2012)Google Scholar
  27. 27.
    Hendrix, W., Palsetia, D., Patwary, M.M.A., Agrawal, A., Liao, W., Choudhary, A.: A scalable algorithm for single-linkage hierarchical clustering on distributed-memory architectures. In: 2013 IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV), IEEE Computer Society, Washington DC, USA, pp. 7–13 (2013)Google Scholar
  28. 28.
    Jeon, Y., Yoon, S.: Multi-threaded hierarchical clustering by parallel nearest-neighbor chaining. IEEE Trans. Parallel Distrib. Syst. 26(9), 2534–2548 (2015)CrossRefGoogle Scholar
  29. 29.
    Jin, C., Patwary, M., Agarwal, A., Hendrix, W., Liao, W., Choudhary, A.: A distributed single-linkage hierarchical clustering algorithm using mapreduce. In: Proceedings of the 4th International SC Workshop on Data Intensive Computing in the Clouds, ACM, New York, USA, pp. 418–426 (2013)Google Scholar
  30. 30.
    Jin, C., Chen, Z., Hendrix, W., Agrawal, A., Choudhary, A.: Incremental, distributed single-linkage hierarchical clustering algorithm using mapreduce. In: Proceedings of the Symposium on High Performance Computing, Society for Computer Simulation International, San Diego, CA, USA, HPC ’15, pp. 83–92 (2015)Google Scholar
  31. 31.
    Jin, C., Liu, R., Chen, Z., Hendrix, W., Agrawal, A., Choudhary, A.: A scalable hierarchical clustering algorithm using spark. In: 2015 IEEE First International Conference on Big Data Computing Service and Applications, IEEE Computer Society, Washington DC, USA, pp. 418–426 (2015)Google Scholar
  32. 32.
    Johnson, E.L., Kargupta, H.: Collective, hierarchical clustering from distributed, heterogeneous data. In: Revised Papers from Large-Scale Parallel Data Mining, SIGKDD, Springer-Verlag, Berlin, Heidelberg, Workshop on Large-Scale Parallel KDD Systems, pp. 221–244 (2000)Google Scholar
  33. 33.
    Kaul, M., Yang, B., Jensen, C.S.: Building accurate 3d spatial networks to enable next generation intelligent transportation systems. In: 2013 IEEE 14th International Conference on Mobile Data Management, vol. 1, pp. 137–146 (2013)Google Scholar
  34. 34.
    Kruskal, J.: On the shortest spanning subtree of a graph and the traveling salesman problem. Proc. Am. Math. Soc. 7(1), 48–50 (1956)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Kumari, S., Maurya, S., Goyal, P., Balasubramaniam, S.S., Goyal, N.: Scalable parallel algorithms for shared nearest neighbor clustering. In: 2016 IEEE 23rd International Conference on High Performance Computing (HiPC), pp. 72–81 (2016)Google Scholar
  36. 36.
    Kurban, H., Jenne, M., Dalkilic, M.M.: Using data to build a better em: Em* for big data. Int. J. Data Sci. Anal. 4(2), 83–97 (2017)CrossRefGoogle Scholar
  37. 37.
    Li, X.: Parallel algorithms for hierarchical clustering and cluster validity. IEEE Trans. Pattern Anal. Mach. Intell. 12(11), 1088–1092 (1990)CrossRefGoogle Scholar
  38. 38.
    Liao, W.K., Ying, L., Choudhary, A.: A grid-based clustering algorithm using adaptive mesh refinement. In: Proceedings of the 7th Workshop on Mining Scientific and Engineering Data Sets, pp. 1–9 (2004)Google Scholar
  39. 39.
    Mazzeo, G.M., Zaniolo, C.: The parallelization of a complex hierarchical clustering algorithm: faster unsupervised learning on larger data sets. University of California, Los Angeles, Technical Report (2016)Google Scholar
  40. 40.
    Murtágh, F.: Multidimensional Clustering Algorithms. Physica-Verlag, Heidelberg (1985)zbMATHGoogle Scholar
  41. 41.
    Olman, V., Mao, F., Wu, H., Xu, Y.: Parallel clustering algorithm for large data sets with applications in bioinformatics. IEEE/ACM Trans. Comput. Biol. Bioinform. 6(2), 344–352 (2009)CrossRefGoogle Scholar
  42. 42.
    Olson, C.F.: Parallel algorithms for hierarchical clustering. Parallel Comput. 21(8), 1313–1325 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Patwary, M.A., Palsetia, D., Agrawal, A., Liao, W.k., Manne, F., Choudhary, A.: A new scalable parallel dbscan algorithm using the disjoint-set data structure. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, IEEE Computer Society Press, Los Alamitos, CA, USA, SC ’12, pp. 62:1–62:11 (2012)Google Scholar
  44. 44.
    Patwary, M.M.A., Blair, J., Manne, F.: Experiments on union-find algorithms for the disjoint-set data structure. In: Proceedings of the 9th International Conference on Experimental Algorithms, Springer, Berlin, Heidelberg, SEA’10, pp. 411–423 (2010)Google Scholar
  45. 45.
    Patwary, M.M.A., Byna, S., Satish, N.R., Sundaram, N., Lukić, Z., Roytershteyn, V., Anderson, M.J., Yao, Y., Prabhat, Dubey P.: Bd-cats: big data clustering at trillion particle scale. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, ACM, New York, NY, USA, SC ’15, pp. 6:1–6:12 (2015)Google Scholar
  46. 46.
    Prim, R.C.: Shortest connection networks and some generalizations. Bell Syst. Tech. J. 36(6), 1389–1401 (1957)CrossRefGoogle Scholar
  47. 47.
    Rajasekaran, S.: Efficient parallel hierarchical clustering algorithms. IEEE Trans. Parallel Distrib. Syst. 16(6), 497–502 (2005)MathSciNetCrossRefGoogle Scholar
  48. 48.
    Sibson, R.: Slink: an optimally efficient algorithm for the single-link cluster method. Comput. J. 16(1), 30–34 (1973)MathSciNetCrossRefGoogle Scholar
  49. 49.
    Springel, V., White, S.D.M., Jenkins, A., Frenk, C.S., Yoshida, N., Gao, L., Navarro, J., Thacker, R., Croton, D., Helly, J., Peacock, J.A., Cole, S., Thomas, P., Couchman, H., Evrard, A., Colberg, J., Pearce, F.: Simulations of the formation, evolution and clustering of galaxies and quasars. Nature 435, 629–636 (2005)CrossRefGoogle Scholar
  50. 50.
    Tan, P.N., Steinbach, M., Kumar, V.: Introduction to Data Mining, 1st edn. Addison-Wesley Longman Publishing Co., Inc., Boston (2005)Google Scholar
  51. 51.
    Teffer, D., Srinivasan, R., Ghosh, J.: Adahash: hashing-based scalable, adaptive hierarchical clustering of streaming data on mapreduce frameworks. Int. J. Data Sci. Anal. 2018, 1–11 (2018)Google Scholar
  52. 52.
    Wu, C.H., Horng, S.J., Tsai, H.R.: Efficient parallel algorithms for hierarchical clustering on arrays with reconfigurable optical buses. J. Parallel Distrib. Comput. 60(9), 1137–1153 (2000)CrossRefzbMATHGoogle Scholar
  53. 53.
    Zaki Jr., M.J., Meira, W., Meira, W.: Data Mining and Analysis: Fundamental Concepts and Algorithms. Cambridge University Press, New York (2014)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.ADAPT LabBirla Institute of Technology and SciencePilaniIndia
  2. 2.Department of Computer Science and Information SystemsPilaniIndia

Personalised recommendations