Multilingual Document Clustering Using Wikipedia as External Knowledge

  • Kiran Kumar N.
  • Santosh G.S.K.
  • Vasudeva Varma
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6653)


This paper presents Multilingual Document Clustering (MDC) on comparable corpora. Wikipedia has evolved to be a major structured multilingual knowledge base. It has been highly exploited in many monolingual clustering approaches and also in comparing multilingual corpora. But there is no prior work which studied the impact of Wikipedia on MDC. Here, we have studied availing Wikipedia in enhancing MDC performance. We have leveraged Wikipedia knowledge structure (such as cross-lingual links, category, outlinks, Infobox information, etc.) to enrich the document representation for clustering multilingual documents. We have implemented Bisecting k-means clustering algorithm and experiments are conducted on a standard dataset provided by FIRE for their 2010 Ad-hoc Cross-Lingual document retrieval task on Indian languages. We have considered English and Hindi datasets for our experiments. By avoiding language-specific tools, our approach provides a general framework which can be easily extendable to other languages. The system was evaluated using F-score and Purity measures and the results obtained were encouraging.


Multilingual Document Clustering Wikipedia Document Representation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Pirkola, A., Hedlund, T., Keskustalo, H., Järvelin, K.: Dictionary-based cross-language information retrieval: Problems, methods, and research findings. Information Retrieval 4, 209–230 (2001)CrossRefzbMATHGoogle Scholar
  2. 2.
    Silva, J., Mexia, J., Coelho, C., Lopes, G.: A statistical approach for multilingual document clustering and topic extraction form clusters. In: Pliska Studia Mathematica Bulgarica, Seattle, WA, pp. 207–228 (2004)Google Scholar
  3. 3.
    Rauber, A., Dittenbach, M., Merkl, D.: Towards automatic content-based organization of multilingual digital libraries: An english, french, and german view of the russian information agency novosti news. In: Third All-Russian Conference Digital Libraries: Advanced Methods and Technologies, Digital Collections, Petrozavodsk, RCDI (2001)Google Scholar
  4. 4.
    Leftin, L.J.: News blaster russian-english clustering performance analysis. Technical report, Columbia computer science Technical Reports (2003)Google Scholar
  5. 5.
    Romaric, B.M., Mathieu, B., Besançon, R., Fluhr, C.: Multilingual document clusters discovery. In: RIAO, pp. 1–10 (2004)Google Scholar
  6. 6.
    Evans, D.K., Klavans, J.L., McKeown, K.R.: Columbia news blaster: multilingual news summarization on the web. In: HLT-NAACL–Demonstrations 2004: Demonstration Papers at HLT-NAACL 2004, pp. 1–4. Association for Computational Linguistics, Morristown (2004)Google Scholar
  7. 7.
    Steinberger, R., Pouliquen, B., Hagman, J.: Cross-lingual document similarity calculation using the multilingual thesaurus eurovoc. In: Gelbukh, A. (ed.) CICLing 2002. LNCS, vol. 2276, pp. 415–424. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Steinberger, R., Pouliquen, B., Ignat, C.: Exploiting multilingual nomenclatures and language-independent text features as an interlingua for cross-lingual text analysis applications. In: Proc. of the 4th Slovenian Language Technology Conf., Information Society (2004)Google Scholar
  9. 9.
    Hu, J., Fang, L., Cao, Y., Zeng, H.J., Li, H., Yang, Q., Chen, Z.: Enhancing text clustering by leveraging wikipedia semantics. In: SIGIR 2008: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 179–186. ACM, New York (2008)Google Scholar
  10. 10.
    Wang, P., Domeniconi, C.: Building semantic kernels for text classification using wikipedia. In: KDD 2008: Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 713–721. ACM, New York (2008)Google Scholar
  11. 11.
    Salton, G., Wong, A., Yang, C.S.: A vector space model for automatic indexing. Commun. ACM 18, 613–620 (1975)CrossRefzbMATHGoogle Scholar
  12. 12.
    Hu, X., Zhang, X., Lu, C., Park, E.K., Zhou, X.: Exploiting wikipedia as external knowledge for document clustering. In: KDD 2009: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 389–396. ACM, New York (2009)Google Scholar
  13. 13.
    Steinbach, M., Karypis, G., Kumar, V.: A comparison of document clustering techniques. In: TextMining Workshop, KDD (2000)Google Scholar
  14. 14.
    Ganesh, S., Harsha, S., Pingali, P., Varma, V.: Statistical transliteration for cross language information retrieval using hmm alignment and crf. In: Proceedings of 3rd International Joint Conference on Natural Language Processing, IJCNLP, Hyderabad, India (2008)Google Scholar
  15. 15.
    Zhao, Y., Karypis, G.: Criterion functions for document clustering: Experiments and analysis. Technical report, Department of Computer Science, University of Minnesota (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Kiran Kumar N.
    • 1
  • Santosh G.S.K.
    • 1
  • Vasudeva Varma
    • 1
  1. 1.International Institute of Information TechnologyHyderabadIndia

Personalised recommendations