Multimedia Tools and Applications

, Volume 72, Issue 2, pp 1441–1463 | Cite as

Semi-supervised non-negative matrix factorization for image clustering with graph Laplacian



Non-negative matrix factorization (NMF) plays an important role in multivariate data analysis, and has been widely applied in information retrieval, computer vision, and pattern recognition. NMF is an effective method to capture the underlying structure of the data in the parts-based low dimensional representation space. However, NMF is actually an unsupervised method without making use of supervisory information of data. In recent years, semi-supervised learning has received a lot of attentions, because partial label information can significantly improve learning quality of the algorithms. In this paper, we propose a novel semi-supervised non-negative matrix factorization (SEMINMF) algorithm, which not only utilizes the local structure of the data characterized by the graph Laplacian, but also incorporates the label information as the fitting constraints to learn. Hence, it can learn from labeled and unlabeled data. By this means our SEMINMF can obtain a more discriminative powerful representation space. Experimental results show the effectiveness of our proposed novel method in comparison to the state-of-the-art algorithms on several real world applications.


Non-negative matrix factorization Clustering Semi-supervised learning Image clustering 



This work was supported in part by the National Basic Research Program of China (973 program) under Grant 2009CB320901, NSFC (no. 61272247), the National High Technology Research and Development Program of China (863 program) under Grant 2008AA02Z310, the National Natural Science Foundation of China under Grant 60873133, and the Innovation Ability Special Fund of Shanghai Jiao Tong University under Grant Z030026.


  1. 1.
    Bach F, Jordan M (2003) Learning spectral clustering. Computer Science Division, University of CaliforniaGoogle Scholar
  2. 2.
    Basu S, Bilenko M, Mooney R et al (2004) A probabilistic framework for semi-supervised clustering. In: Proceedings of the 10th ACM SIGKDD international conference on knowledge discovery and data mining, pp 22–25Google Scholar
  3. 3.
    Cai D, He X, Han J, Huang T (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560CrossRefGoogle Scholar
  4. 4.
    Chapelle O, Schölkopf B, Zien A et al (2006) Semi-supervised learning, vol 2. MIT Press, Cambridge, MACrossRefGoogle Scholar
  5. 5.
    Chen Y, Rege M, Dong M, Hua J (2008) Non-negative matrix factorization for semi-supervised data clustering. Knowl Inf Syst 17(3):355–379CrossRefGoogle Scholar
  6. 6.
    Chung F (1997) Spectral graph theory, no 92. American Mathematical SocietyGoogle Scholar
  7. 7.
    Cormen T (2001) Introduction to algorithms. The MIT PressGoogle Scholar
  8. 8.
    Das Gupta M, Xiao J (2011) Non-negative matrix factorization as a feature selection tool for maximum margin classifiers. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2841–2848Google Scholar
  9. 9.
    De la Torre F, Kanade, T (2006) Discriminative cluster analysis. In: Proceedings of the 23rd international conference on machine learning. ACM, pp 241–248Google Scholar
  10. 10.
    Dempster A, Laird N, Rubin D (1977) Maximum likelihood from incomplete data via the em algorithm. J Roy Stat Soc Ser B (Methodological) 39:1–38MATHMathSciNetGoogle Scholar
  11. 11.
    Ding C, Li T (2007) Adaptive dimension reduction using discriminant analysis and k-means clustering. In: ACM international conference proceeding series, vol 227, pp 521–528Google Scholar
  12. 12.
    Ding C, Li T, Jordan M (2008) Nonnegative matrix factorization for combinatorial optimization: spectral clustering, graph matching and clique finding. In: 8th IEEE International Conference on Data Mining, 2008. ICDM’08. IEEE, pp 183–192Google Scholar
  13. 13.
    Guillamet D, Vitria J (2002) Classifying faces with nonnegative matrix factorization. In: Proc. 5th Catalan conference for artificial intelligenceGoogle Scholar
  14. 14.
    Hoyer P (2004) Non-negative matrix factorization with sparseness constraints. J Mach Learn Res 5:1457–1469MATHMathSciNetGoogle Scholar
  15. 15.
    Kim J, Park, H (2008) Sparse nonnegative matrix factorization for clustering.Google Scholar
  16. 16.
    Lee D, Seung H (2001) Algorithms for non-negative matrix factorization. Adv Neural Inf Process Syst 13:556–562Google Scholar
  17. 17.
    Lee H, Yoo J, Choi S (2010) Semi-supervised nonnegative matrix factorization. IEEE Signal Process Lett 17(1):4–7Google Scholar
  18. 18.
    Li T, Ding C (2006) The relationships among various nonnegative matrix factorization methods for clustering. In: 6th International conference on data mining, 2006. ICDM’06. IEEE, pp 362–371Google Scholar
  19. 19.
    Lin TC, Huang HC, Liao BY, Pan JS (2007) An optimized approach on applying genetic algorithm to adaptive cluster validity index. Int J Comput Sci Eng Syst 1(4):253–257MATHGoogle Scholar
  20. 20.
    Liu H, Wu Z (2010) Non-negative matrix factorization with constraints. In: 24th AAAI conference on artificial intelligenceGoogle Scholar
  21. 21.
    Liu H, Wu Z, Li X, Cai D, Huang TS (2012) Constrained nonnegative matrix factorization for image representation. IEEE Trans Pattern Anal Mach Intell 34(7):1299–1311CrossRefGoogle Scholar
  22. 22.
    Lovász L, Plummer M (1986) Matching theory, no 121. Elsevier Science LtdGoogle Scholar
  23. 23.
    Ma Z, Yang Y, Nie F, Uijlings J, Sebe N (2011) Exploiting the entire feature space with sparsity for automatic image annotation. In: Proceedings of the 19th ACM international conference on multimedia. ACM, pp 283–292Google Scholar
  24. 24.
    Perona P, Zelnik-Manor L (2004) Self-tuning spectral clustering. Adv Neural Inf Process Syst 17:1601–1608Google Scholar
  25. 25.
    Philbin J, Chum O, Isard M, Sivic J, Zisserman A (1986) Object retrieval with large vocabularies and fast spatial matching. In: IEEE conference on computer vision and pattern recognition, 2007. CVPR’07. IEEE, pp 1–8Google Scholar
  26. 26.
    Saul, L., Pereira, F (1997) Aggregate and mixed-order markov models for statistical language processing. In: Proceedings of the 2nd conference on empirical methods in natural language processing. Association for Computational Linguistics, Somerset, New Jersey, pp 81–89Google Scholar
  27. 27.
    Shashua A, Hazan T (2005) Non-negative tensor factorization with applications to statistics and computer vision. In: Proceedings of the 22nd international conference on machine learning. ACM, pp 792–799Google Scholar
  28. 28.
    Von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416CrossRefMathSciNetGoogle Scholar
  29. 29.
    Xu W, Gong Y (2004) Document clustering by concept factorization. In: Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval. ACM, pp 202–209Google Scholar
  30. 30.
    Xu W, Liu X, Gong Y (2003) Document clustering based on non-negative matrix factorization. In: Proceedings of the 26th annual international ACM SIGIR conference on research and development in informaion retrieval. ACM, pp 267–273Google Scholar
  31. 31.
    Yang Y, Xu D, Nie F, Yan S, Zhuang Y (2010) Image clustering using local discriminant models and global integration. IEEE Trans Image Process 19(10):2761–2773CrossRefMathSciNetGoogle Scholar
  32. 32.
    Yang Y, Shen H, Nie F, Ji R, Zhou X (2011) Nonnegative spectral clustering with discriminative regularization. In: 25th AAAI conference on artificial intelligence. AAAI Press, pp 555–560Google Scholar
  33. 33.
    Ye J, Zhao Z, Wu M (2007) Discriminative k-means for clustering. Adv Neural Inf Process Syst 20:1649–1656Google Scholar
  34. 34.
    Yu S, Shi J (2003) Multiclass spectral clustering. In: Proceedings of 9th IEEE international conference on computer vision, 2003. IEEE, pp 313–319Google Scholar
  35. 35.
    Zhang Y, Yeung D (2008) Semi-supervised discriminant analysis using robust path-based similarity. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008. IEEE, pp 1–8Google Scholar
  36. 36.
    Zhang Z, Wang J, Zha H (2005) Adaptive manifold learning. IEEE Trans Pattern Anal Mach Intell 34(2):253–265CrossRefGoogle Scholar
  37. 37.
    Zhang Z, Zha H, Zhang M (2008) Spectral methods for semi-supervised manifold learning. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008. IEEE, pp 1–6Google Scholar
  38. 38.
    Zhou D, Bousquet O, Lal T, Weston J, Schölkopf B (2004) Learning with local and global consistency. Adv Neural Inf Process Syst 16:321–328Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.MOE-Microsoft Laboratory for Intelligent Computing and Intelligent Systems, Department of Computer Science and EngineeringShanghai Jiao Tong UniversityShanghaiPeople’s Republic of China

Personalised recommendations