Abstract
High dimensionality is a major challenge in document clustering. Some of the recent algorithms address this problem by using frequent itemsets for clustering. But, most of these algorithms neglect the semantic relationship between the words. On the other hand there are algorithms that take care of the semantic relations between the words by making use of external knowledge contained in WordNet, Mesh, Wikipedia, etc but do not handle the high dimensionality. In this paper we present an efficient solution that addresses both these problems. We propose a hierarchical clustering algorithm using closed frequent itemsets that use Wikipedia as an external knowledge to enhance the document representation. We evaluate our methods based on F-Score on standard datasets and show our results to be better than existing approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Pudi, V., Haritsa, J.R.: Generalized Closed Itemsets for Association Rule Mining. In: Proc. of IEEE Conf. on Data Engineering (2003)
Kaufman, L., Rousseeuw, P.J.: Finding Groups in Data. In: An introduction to Cluster Analysis. John Wiley & Sons, Inc., Chichester (1990)
Zhao, Y., Karypis, G.: Evaluation of Hierarchial Clustering Algorithms for Document Datasets. In: Proc. of Intl. Conf. on Information and Knowledge Management (2002)
Beil, F., Ester, M., Xu, X.: Frequent Term-based Text Clustering. In: Proc. of Intl. Conf. on Knowledge Discovery and Data Mining (2002)
Fung, B., Wang, K., Ester, M.: Hierarchical Document Clustering using Frequent Itemsets. In: Proc. of SIAM Intl. Conf. on Data Mining (2003)
Malik, H.H., Kender, J.R.: High Quality, Efficient Hierarchical Document Clustering Using Closed Interesting Itemsets. In: Proc. of IEEE Intl. Conf. on Data Mining (2006)
Yu, H., Searsmith, D., Li, X., Han, J.: Scalable Construction of Topic Directory with Nonparametric Closed Termset Mining. In: Proc. of Fourth IEEE Intl. Conf. on Data Mining (2004)
Hotho, A., Staab, S., et al.: Wordnet Improves Text Document Clustering. In: The 26th Annual Intl. ACM SIGIR Conf. on Proc. of Semantic Web Workshop (2003)
Hotho, A., Maedche, A., Staab, S.: Text Clustering Based on Good Aggregations. In: Proc. of IEEE Intl. Conf. on Data Mining (2001)
Zhang, X., Jing, L., Hu, X., et al.: A Comparative Study of Ontology Based Term Similarity Measures on Document Clustering. In: Proc. of 12th Intl. Conf. on Database Systems for Advanced Applications (2007)
Gabrilovich, E., Markovitch, S.: Overcoming the Brittleness Bottleneck Using Wikipedia: Enhancing Text Categorization with Encyclopedic Knowledge. In: Proc. of The 21st National Conf. on Artificial Intelligence (2006)
Gabrilovich, E., Markovitch, S.: Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis. In: Proc. of The 20th Intl. Joint Conf. on Artificial Intelligence (2007)
Hu, X., Zhang, X., Lu, C., et al.: Exploiting Wikipedia as External Knowledge for Document Clustering. In: Proc. of Knowledge Discovery and Data Mining (2009)
Hu, J., Fang, L., Cao, Y., et al.: Enhancing Text Clustering by Leveraging Wikipedia Semantics. In: Proc. of 31st Annual Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
G.V.R., K., Shankar, R., Pudi, V. (2010). Frequent Itemset Based Hierarchical Document Clustering Using Wikipedia as External Knowledge. In: Setchi, R., Jordanov, I., Howlett, R.J., Jain, L.C. (eds) Knowledge-Based and Intelligent Information and Engineering Systems. KES 2010. Lecture Notes in Computer Science(), vol 6277. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15390-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-15390-7_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15389-1
Online ISBN: 978-3-642-15390-7
eBook Packages: Computer ScienceComputer Science (R0)