Semi-supervised Document Clustering with Simultaneous Text Representation and Categorization

  • Yanhua Chen
  • Lijun Wang
  • Ming Dong
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5781)


In order to derive high quality information from text, the field of text mining has advanced swiftly from simple document clustering to co-clustering with words and categories. However, document co-clustering without any prior knowledge or background information is a challenging problem. In this paper, we propose a Semi-Supervised Non-negative Matrix Factorization (SS-NMF) framework for document co-clustering. Our method computes new word-document and document-category matrices by incorporating user provided constraints through simultaneous distance metric learning and modality selection. Using an iterative algorithm, we perform tri-factorization of the new matrices to infer the document, category and word clusters. Theoretically, we show the convergence and correctness of SS-NMF co-clustering and the advantages of SS-NMF co-clustering over existing approaches. Through extensive experiments conducted on publicly available data sets, we demonstrate the superior performance of SS-NMF for document co-clustering.


Semi-supervised co-clustering Non-negative matrix factorization 


  1. 1.
    Liu, X., Gong, Y., Xu, W., Zhu, S.: Document clustering with Cluster Refinement and Model Selection Capabilites. In: Proc. of ACM SIGIR, pp. 191–198 (2002)Google Scholar
  2. 2.
    Willett, P.: Recent Trends in Hierarchic Document Clustering: a Critical Review. Information Process Management 24(5), 577–597 (1988)CrossRefGoogle Scholar
  3. 3.
    Ding, C., He, X., Zha, H., Simon, H.: A Min-max Cut Algorithm for Graph Partitioning and Data Clustering. In: Proc. of IEEE ICDM, pp. 107–114 (2001)Google Scholar
  4. 4.
    Xu, W., Liu, X., Gong, Y.: Document Clustering based on Non-negative Matrix Factorization. In: Proc. of ACM SIGIR, pp. 267–273 (2003)Google Scholar
  5. 5.
    Dhillon, I.S.: Co-Clustering Documents and Words Using Bipartite Spectral Graph Partitioning. In: Proc. of ACM SIGKDD, pp. 269–274 (2001)Google Scholar
  6. 6.
    Long, B., Zhang, Z., Yu, P.S.: Co-clustering by Block Value Decomposition. In: Proc. of ACM SIGKDD, pp. 635–640 (2005)Google Scholar
  7. 7.
    Gao, B., Liu, T.-Y., Cheng, Q., Feng, G., Qin, T., Ma, W.-Y.: Hierarchical Taxonomy Preparation for Text Categorization using Consistent Bipartite Spectral Graph Copartitioning. IEEE Transactions on Knowledge and Data Engineering 17(9), 1263–1273 (2005)CrossRefGoogle Scholar
  8. 8.
    Long, B., Zhang, Z., Wu, X., Yu, P.S.: Spectral Clustering for Multi-type Relational Data. In: Proc. of ICML, pp. 585–592 (2006)Google Scholar
  9. 9.
    Dhillon, I.S., Mallela, S., Modha, D.S.: Information-theoretic Co-clustering. In: Proc. of ACM SIGKDD, pp. 89–98 (2003)Google Scholar
  10. 10.
    Gao, B., Liu, T.-Y., Mao, W.-Y.: Star-structured High-order Heterogeneous Data Co-clustering based on Consistent Information Theory. In: Proc. of IEEE ICDM, pp. 880–884 (2006)Google Scholar
  11. 11.
    Rege, M., Dong, M., Fotouhoi, F.: Co-clustering Documents and Words using Bipartite Isoperimetric Graph Partitioning. In: Proc. of IEEE ICDM, pp. 532–541 (2006)Google Scholar
  12. 12.
    Rege, M., Dong, M., Hua, J.: Graph Theoretical Framework for Simultaneously Integrating Visual and Textual Features for Efficient Web Image Clustering. In: Proc. of WWW, pp. 317–326 (2008)Google Scholar
  13. 13.
    Hiu, M., Law, C., Topchy, A., Jain, A.K.: Model-based Clustering with Probabilistic Constraints. In: Proc. of SIAM ICDM, pp. 641–645 (2005)Google Scholar
  14. 14.
    Xing, E.P., Ng, A.Y., Jordan, M.I., Russell, S.: Distance Metirc Learning with Application to Clustering with Side-information. In: Proc. of NIPS, pp. 362–371 (2001)Google Scholar
  15. 15.
    Kulis, B., Basu, S., Dhillon, I.S., Mooney, R.: Semi-supervised Graph Clustering:a Kernel Approach. In: Proc. of ICML, pp. 457–464 (2005)Google Scholar
  16. 16.
    Ji, X., Xu, W.: Document Clustering with Prior Knowledge. In: Proc. of ACM SIGIR, pp. 405–412 (2006)Google Scholar
  17. 17.
    Chen, Y., Rege, M., Dong, M., Hua, J.: Incorporating User Provided Constraints into Document Clusteirng. In: Proc. of IEEE ICDM, pp. 577–582 (2007)Google Scholar
  18. 18.
    Chen, Y., Rege, M., Dong, M., Fotouhi, F.: Deriving Semantics for Image Clustering from Accumulated User Feedbacks. In: Proc. of ACM MM, pp. 313–316 (2007)Google Scholar
  19. 19.
    Chen, Y., Rege, M., Dong, M., Hua, J.: Non-negative Matrix Factorization for Semi-supervised Data Clustering. Journal of Knowledge and Information Systems 17(3), 355–379 (2008)CrossRefGoogle Scholar
  20. 20.
    Lee, D.D., Seung, H.S.: Learning the Parts of Objects by Non-negative Matrix Factorization. Nature 401, 788–791 (1999)CrossRefGoogle Scholar
  21. 21.
    Ding, C., Li, T., Peng, W., Park, H.: Orthogonal Nonnegative Matrix Tri-factorizations for Clustering. In: Proc. of ACM SIGKDD, pp. 126–135 (2006)Google Scholar
  22. 22.
    Lee, D.D., Seung, H.S.: Algorithms for Non-negative Matirx Factroization. In: Proc. of NIPS, pp. 362–371 (2001)Google Scholar
  23. 23.
    Han, E.-H., Karypis, G.: Centroid-based document classification: Analysis and experimental results. In: Zighed, D.A., Komorowski, J., Żytkow, J.M. (eds.) PKDD 2000. LNCS (LNAI), vol. 1910, pp. 424–431. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  24. 24.
    Lang, K.: News weeder: Learning to Filter Networks. In: Proc. of ICML, pp. 331–339 (1995)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Yanhua Chen
    • 1
  • Lijun Wang
    • 1
  • Ming Dong
    • 1
  1. 1.Machine Vision and Pattern Recognition Lab Department of Computer ScienceWayne State UniversityDetroitUSA

Personalised recommendations