Multimedia Tools and Applications

, Volume 77, Issue 17, pp 22281–22297 | Cite as

Entropy-based active sparse subspace clustering

  • Yanbei Liu
  • Kaihua Liu
  • Changqing Zhang
  • Xiao Wang
  • Shaona Wang
  • Zhitao XiaoEmail author


Sparse Subspace Clustering (SSC) is widely used in data mining and machine learning. Some studies have been developed to add pairwise constraints as side information to improve the clustering results. However, most of these algorithms are “passive” in the sense that the side information is provided beforehand. In this paper, we propose a novel extension for SSC with active learning framework, in which we aim to select the most informative pairwise constraints to guide the SSC for accurate clustering results. Specifically, in the first step, an entropy-based query strategy is proposed to select the most uncertain pairwise constraints. Next, constrained sparse subspace clustering algorithms are followed to integrate the selected pairwise constraints and obtain the final clustering results. Two steps are effectively performed in an iterative manner until satisfactory results are achieved. Experimental results on two face datasets clustering well demonstrate the effectiveness of the proposed method.


Active learning Sparse subspace clustering Constrained clustering Entropy-based query strategy 



This work was supported in part by Major Program of National Natural Science Foundation of China (Grant no. 13&ZD162), Applied Basic Research Programs of China National Textile and Apparel Council (Grant no. J201509), National Natural Science Foundation of China(Grant no. 61601325), and Plan Program of Tianjin Educational Science and Research (Grant no. 2017KJ087).


  1. 1.
    Basu S, Banerjee A, Mooney R (2004) Active semi-supervision for pairwise constrained clustering. In: International conference on data mining (ICDM), pp 333–344Google Scholar
  2. 2.
    Biswas A, Jacobs D (2011) Large scale image clustering with active pairwise constraints. In: International conference in machine learning workshop on combining learning strategies to reduce label costGoogle Scholar
  3. 3.
    Biswas A, Jacobs D (2012) Active image clustering: seeking constraints from humans to complement algorithms. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2152–2159Google Scholar
  4. 4.
    Cai D, He X, Hu Y, Han J, Huang T (2007) Learning a spatially smooth subspace for face recognition. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1–7Google Scholar
  5. 5.
    Chen G, Lerman G (2009) Spectral curvature clustering. Int J Comput Vis 81(3):317–330CrossRefGoogle Scholar
  6. 6.
    Donoho DL (2006) For most large underdetermined systems of linear equations the minimal 1-norm solution is also the sparsest solution. Commun Pure Appl Math 59(6):797–829MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Elhamifar E, Vidal R (2009) Sparse subspace clustering. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2790–2797Google Scholar
  8. 8.
    Elhamifar E, Vidal R (2013) Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans Pattern Anal Mach Intell 35(11):2765–2781CrossRefGoogle Scholar
  9. 9.
    Eriksson B, Dasarathy G, Singh A, Nowak R (2011) Active clustering: robust and efficient hierarchical clustering using adaptively selected similarities. arXiv:1102.3887
  10. 10.
    Gao Z, Zhang H, Xu G P, Xue Y B, Hauptmann A G (2014) Multi-view discriminative and structured dictionary learning with group sparsity for human action recognition. Signal Process 112(C):83–97Google Scholar
  11. 11.
    Gao Z, Zhang L F, Chen M Y, Hauptmann A, Zhang H, Cai A N (2014) Enhanced and hierarchical structure algorithm for data imbalance problem in semantic extraction under massive video dataset. Multimed Tools Appl 68(3):641–657CrossRefGoogle Scholar
  12. 12.
    Georghiades AS, Belhumeur P N, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660CrossRefGoogle Scholar
  13. 13.
    Greene D, Cunningham P (2007) Constraint selection by committee: an ensemble approach to identifying informative constraints for semi-supervised clustering, pp 140–151Google Scholar
  14. 14.
    Grira N, Crucianu M, Boujemaa N (2008) Active semi-supervised fuzzy clustering. Pattern Recogn 41(5):1834–1844CrossRefzbMATHGoogle Scholar
  15. 15.
    Guha T, Ward R K (2012) Learning sparse representations for human action recognition. IEEE Trans Pattern Anal Mach Intell 34(8):1576–1588CrossRefGoogle Scholar
  16. 16.
    Hoi SCH, Jin R (2008) Active kernel learning. In: International conference on machine learning (ICML), pp 400–407Google Scholar
  17. 17.
    Hu Q, Che X, Zhang L, Zhang D, Guo M, Yu D (2012) Rank entropy-based decision trees for monotonic classification. IEEE Trans Knowl Data Eng 24(11):2052–2064CrossRefGoogle Scholar
  18. 18.
    Huang R, Lam W (2007) Semi-supervised document clustering via active learning with pairwise constraints, pp 517–522Google Scholar
  19. 19.
    Kenley E, Cho Y-R (2011) Entropy-based graph clustering: application to biological and social networks. In: IEEE international conference on data mining (ICDM), pp 1116–1121Google Scholar
  20. 20.
    Kim T, Ghosh J (2017) Relaxed oracles for semi-supervised clustering. arXiv:1711.07433
  21. 21.
    Kim T, Ghosh J (2017) Semi-supervised active clustering with weak oracles. arXiv:1709.03202
  22. 22.
    Klein D, Kamvar SD, Manning CD (2002) From instance-level constraints to space-level constraints: making the most of prior knowledge in data clustering. In: International conference on machine learning (ICML), pp 307–314Google Scholar
  23. 23.
    Koh K, Kim S-J, Boyd SP (2007) An interior-point method for large-scale 1-regularized logistic regression. J Mach Learn Res 8(8):1519–1555MathSciNetzbMATHGoogle Scholar
  24. 24.
    Krishnamurthy A, Balakrishnan S, Xu M, Singh A (2012) Efficient active algorithms for hierarchical clustering. arXiv:1206.4672
  25. 25.
    Kulis B, Basu S, Dhillon I, Mooney R (2009) Semi-supervised graph clustering: a kernel approach. Mach Learn 74(1):1–22CrossRefGoogle Scholar
  26. 26.
    Li T, Ma S, Ogihara M (2004) Entropy-based criterion in categorical clustering. In: International conference on machine learning (ICML), pp 68–75Google Scholar
  27. 27.
    Li T, Ding C, Jordan M et al (2007) Solving consensus and semi-supervised clustering problems using nonnegative matrix factorization. In: IEEE international conference on data mining (ICDM), pp 577–582Google Scholar
  28. 28.
    Mallapragada P K, Jin R, Jain AK (2008) Active query selection for semi-supervised clustering. In: International conference on pattern recognition (ICPR), pp 1–4Google Scholar
  29. 29.
    Nogueira BM, Jorge AM, Rezende SO (2012) Hierarchical confidence-based active clustering. In: ACM symposium on applied computing. ACM, pp 216–219Google Scholar
  30. 30.
    Vu V-V, Labroche N, Bouchon-Meunier B (2012) Improving constrained clustering with active query selection. Pattern Recogn 45(4):1749–1758CrossRefGoogle Scholar
  31. 31.
    Wang X, Davidson I (2010) Active spectral clustering. In: IEEE international conference on data mining (ICDM), pp 561–568Google Scholar
  32. 32.
    Wauthier FL, Jojic N, Jordan M I (2012) Active spectral clustering via iterative uncertainty reduction. In: International conference on knowledge discovery and data mining, pp 1339–1347Google Scholar
  33. 33.
    Xiong C, Johnson D, Corso JJ (2012) Spectral active clustering via purification of the k-nearest neighbor graph. In: European conference on data mining, pp 1–9Google Scholar
  34. 34.
    Xiong C, Johnson DM, Corso JJ (2017) Active clustering with model-based uncertainty reduction. IEEE Trans Pattern Anal Mach Intell 39(1):5–17CrossRefGoogle Scholar
  35. 35.
    Xu Q, Wagstaff K L et al (2005) Active constrained clustering by examining spectral eigenvectors. In: Discovery science, pp 294–307Google Scholar
  36. 36.
    Yang Y, Xu D, Nie F, Yan S, Zhuang Y (2010) Image clustering using local discriminant models and global integration. IEEE Trans Image Process 19 (10):2761–2773MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Yang Y, Nie F, Xu D, Luo J, Zhuang Y, Pan Y (2012) A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Trans Pattern Anal Mach Intell 34(4):723CrossRefGoogle Scholar
  38. 38.
    Zhang H, Shang X, Luan H, Wang M, Chua T S (2016) Learning from collective intelligence: feature learning using social images and tags. ACM Trans Multimed Comput Commun Appl 13(1):1CrossRefGoogle Scholar
  39. 39.
    Zhang H, Shang X, Yang W, Xu H, Luan H, Chua T S (2016) Online collaborative learning for open-vocabulary visual classifiers. In: Computer vision and pattern recognition, pp 2809–2817Google Scholar
  40. 40.
    Zhou C, Zhang C, Li X, Shi G, Cao X (2014) Video face clustering via constrained sparse representation. In: IEEE international conference on multimedia and expo, pp 1–6Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Yanbei Liu
    • 1
    • 2
  • Kaihua Liu
    • 3
  • Changqing Zhang
    • 4
  • Xiao Wang
    • 5
  • Shaona Wang
    • 1
    • 2
  • Zhitao Xiao
    • 1
    • 2
    Email author
  1. 1.Tianjin Key Laboratory of Optoelectronic Detection Technology and SystemsTianjinChina
  2. 2.School of Electronics and Information EngineeringTianjin Polytechnic UniversityTianjinChina
  3. 3.School of Electronic Information EngineeringTianjin UniversityTianjinChina
  4. 4.School of Computer and Science TechnologyTianjin UniversityTianjinChina
  5. 5.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina

Personalised recommendations