Skip to main content
SpringerLink
Log in

An Introduction to Deep Clustering

  • Chapter
  • First Online:
Clustering Methods for Big Data Analytics

Part of the book series: Unsupervised and Semi-Supervised Learning ((UNSESUL))

Abstract

Driven by the explosive growth in available data and decreasing costs of computation, Deep Learning (DL) has found much of its fame in problems involving classification tasks which are considered supervised learning. Deep learning has also been widely used to learn richer and better data representations from big data, without relying too much on human engineered features. Even though it started mostly within the realm of supervised learning, deep learning’s success has recently inspired several deep learning-based developments in clustering algorithms which sit squarely within unsupervised learning. Most DL-based clustering approaches result in both deep representations and (either as an explicit aim or as a byproduct) clustering outputs, hence we refer to all these approaches as Deep Clustering. In this chapter, we present a simplified taxonomy of Deep Clustering methods, based mainly on the overall procedural structure or design which helps beginning readers quickly grasp how almost all approaches are designed, while allowing more advanced readers to learn how to design increasingly sophisticated deep clustering pipelines that fit their own machine learning problem-solving aims. Like DL, Deep Clustering promises to leave an impact on diverse application domains ranging from computer vision and speech recognition to recommender systems and natural language processing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. E. Aljalbout, V. Golkov, Y. Siddiqui, D. Cremers, Clustering with deep learning: taxonomy and new methods (2018). Arxiv preprint arXiv:1801.07648

    Google Scholar 

  2. P. Baldi, Autoencoders, unsupervised learning, and deep architectures, in Proceedings of ICML Workshop on Unsupervised and Transfer Learning (2012)

    Google Scholar 

  3. J. Bruna, S. Mallat, Invariant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1872–1886 (2013)

    Article  Google Scholar 

  4. D. Cai, X. He, J. Han, Locally consistent concept factorization for document clustering. IEEE Trans. Knowl. Data Eng. 23(6), 902–913 (2011)

    Article  Google Scholar 

  5. R. Collobert, J. Weston, A unified architecture for natural language processing: deep neural networks with multitask learning, in Proceedings of the 25th International Conference on Machine Learning (ACM, New York, 2008), pp. 160–167

    Google Scholar 

  6. D. Erhan, Y. Bengio, A. Courville, P.-A. Manzagol, P. Vincent, S. Bengio, Why does unsupervised pre-training help deep learning? J. Mach. Learn. Res. 11, 625–660 (2010)

    MathSciNet  MATH  Google Scholar 

  7. A.K. Farahat, A. Elgohary, A. Ghodsi, M.S. Kamel, Greedy column subset selection for large-scale data sets. Knowl. Inf. Syst. 45(1), 1–34 (2015)

    Article  Google Scholar 

  8. I. Goodfellow, Y. Bengio, A. Courville, Y. Bengio, Deep Learning, vol. 1 (MIT Press, Cambridge, 2016)

    MATH  Google Scholar 

  9. X. He, L. Liao, H. Zhang, L. Nie, X. Hu, T.-S. Chua, Neural collaborative filtering, in Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee (2017), pp. 173–182

    Google Scholar 

  10. J.R. Hershey, Z. Chen, J. Le Roux, S. Watanabe, Deep clustering: discriminative embeddings for segmentation and separation, in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, Piscataway, 2016), pp. 31–35

    Google Scholar 

  11. G.E. Hinton, A practical guide to training restricted Boltzmann machines, in Neural Networks: Tricks of the Trade (Springer, Berlin, 2012), pp. 599–619

    Book  Google Scholar 

  12. G.E. Hinton, R.R. Salakhutdinov, Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  Google Scholar 

  13. G. Hinton, L. Deng, D. Yu, G.E. Dahl, A.-R. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T.N. Sainath et al., Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process. Mag. 29(6), 82–97 (2012)

    Article  Google Scholar 

  14. A.K. Jain, Data clustering: 50 years beyond k-means. Pattern Recogn. Lett. 31(8), 651–666 (2010)

    Article  Google Scholar 

  15. A. Krizhevsky, I. Sutskever, G.E. Hinton, ImageNet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems (2012), pp. 1097–1105

    Google Scholar 

  16. S. Kullback, R.A. Leibler, On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)

    Article  MathSciNet  Google Scholar 

  17. Y. LeCun, Y. Bengio, G. Hinton, Deep learning. Nature 521(7553), 436 (2015)

    Article  Google Scholar 

  18. D.D. Lee, H.S. Seung, Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788 (1999)

    Article  Google Scholar 

  19. F. Li, H. Qiao, B. Zhang, X. Xi, Discriminatively boosted image clustering with fully convolutional auto-encoders (2017). Arxiv preprint arXiv:1703.07980

    Google Scholar 

  20. S. Lloyd, Least squares quantization in PCM. IEEE Trans. Inf. Theory 28(2), 129–137 (1982)

    Article  MathSciNet  Google Scholar 

  21. G. Montavon, K.-R. Müller, Better representations: invariant, disentangled and reusable, in Neural Networks: Tricks of the Trade (Springer, Berlin, 2012), pp. 559–560

    Google Scholar 

  22. A.Y. Ng, M.I. Jordan, Y. Weiss, On spectral clustering: analysis and an algorithm, in Advances in Neural Information Processing Systems (2002), pp. 849–856

    Google Scholar 

  23. V.M. Patel, H. Van Nguyen, R. Vidal, Latent space sparse subspace clustering, in 2013 IEEE International Conference on Computer Vision (ICCV) (IEEE, Piscataway, 2013), pp. 225–232

    Book  Google Scholar 

  24. X. Peng, J. Feng, S. Xiao, J. Lu, Z. Yi, S. Yan, Deep sparse subspace clustering (2017). ArXiv preprint arXiv:1709.08374

    Google Scholar 

  25. M. Quadrana, A. Karatzoglou, B. Hidasi, P. Cremonesi, Personalizing session-based recommendations with hierarchical recurrent neural networks, in Proceedings of the Eleventh ACM Conference on Recommender Systems (ACM, New York, 2017), pp. 130–137

    Book  Google Scholar 

  26. S. Saito, L. Wei, L. Hu, K. Nagano, H. Li, Photorealistic facial texture inference using deep neural networks, in IEEE Conference on Computer Vision and Pattern Recognition, CVPR, vol. 3 (2017)

    Google Scholar 

  27. R. Salakhutdinov, A. Mnih, G. Hinton, Restricted Boltzmann machines for collaborative filtering, in Proceedings of the 24th International Conference on Machine Learning (ACM, New York, 2007), pp. 791–798

    Google Scholar 

  28. F. Schroff, D. Kalenichenko, J. Philbin, FaceNet: a unified embedding for face recognition and clustering, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  29. C.-J. Tsai, Y.-W. Tsai, S.-L. Hsu, Y.-C. Wu, Synthetic training of deep CNN for 3d hand gesture identification, in 2017 International Conference on Control, Artificial Intelligence, Robotics & Optimization (ICCAIRO) (IEEE, Piscataway, 2017), pp. 165–170

    Google Scholar 

  30. A. Van den Oord, S. Dieleman, B. Schrauwen, Deep content-based music recommendation, in Advances in Neural Information Processing Systems (2013), pp. 2643–2651

    Google Scholar 

  31. P. Vincent, H. Larochelle, Y. Bengio, P.-A. Manzagol, Extracting and composing robust features with denoising autoencoders, in Proceedings of the 25th International Conference on Machine Learning (ACM, New York, 2008), pp. 1096–1103

    Google Scholar 

  32. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)

    MathSciNet  MATH  Google Scholar 

  33. U. Von Luxburg, A tutorial on spectral clustering. Stat. Comput. 17(4), 395–416 (2007)

    Article  MathSciNet  Google Scholar 

  34. J. Xie, R. Girshick, A. Farhadi, Unsupervised deep embedding for clustering analysis, in International Conference on Machine Learning (2016), pp. 478–487

    Google Scholar 

  35. R. Xu, Measuring explained variation in linear mixed effects models. Stat. Med. 22(22), 3527–3541 (2003)

    Article  Google Scholar 

  36. D. Xu, Y. Tian, A comprehensive survey of clustering algorithms. Ann. Data Sci. 2(2), 165–193 (2015)

    Article  MathSciNet  Google Scholar 

  37. Y. Yang, M.C. Gupta, K.L. Dudley, Towards cost-efficient EMI shielding materials using carbon nanostructure-based nanocomposites. Nanotechnology 18(34), 345701 (2007)

    Article  Google Scholar 

  38. A.Y. Yang, S.S. Sastry, A. Ganesh, Y. Ma, Fast ℓ 1-minimization algorithms and an application in robust face recognition: a review, in 2010 17th IEEE International Conference on Image Processing (ICIP) (IEEE, Piscataway, 2010), pp. 1849–1852

    Google Scholar 

  39. B. Yang, X. Fu, N.D. Sidiropoulos, M. Hong, Towards k-means-friendly spaces: simultaneous deep learning and clustering (2016). Arxiv preprint arXiv:1610.04794

    Google Scholar 

  40. J. Yang, D. Parikh, D. Batra, Joint unsupervised learning of deep representations and image clusters, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 5147–5156

    Google Scholar 

  41. M. Zheng, J. Bu, C. Chen, C. Wang, L. Zhang, G. Qiu, D. Cai, Graph regularized sparse coding for image representation. IEEE Trans. Image Process. 20(5), 1327–1336 (2011)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Knowledge Discovery & Web Mining Lab, University of Louisville, Louisville, KY, USA

    Gopi Chand Nutakki, Behnoush Abdollahi & Wenlong Sun

  2. Department of Computer Engineering and Computer Science, University of Louisville, Louisville, KY, USA

    Olfa Nasraoui

Authors
  1. Gopi Chand Nutakki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Behnoush Abdollahi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenlong Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Olfa Nasraoui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Olfa Nasraoui .

Editor information

Editors and Affiliations

  1. Department of Computer Engineering and Computer Science, University of Louisville, Louisville, KY, USA

    Olfa Nasraoui

  2. University of Jeddah, Jeddah, Saudi Arabia

    Chiheb-Eddine Ben N'Cir

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Nutakki, G.C., Abdollahi, B., Sun, W., Nasraoui, O. (2019). An Introduction to Deep Clustering. In: Nasraoui, O., Ben N'Cir, CE. (eds) Clustering Methods for Big Data Analytics. Unsupervised and Semi-Supervised Learning. Springer, Cham. https://doi.org/10.1007/978-3-319-97864-2_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-97864-2_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-97863-5

  • Online ISBN: 978-3-319-97864-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation