Abstract
Topic modeling has been widely utilized in the fields of information retrieval, text mining, text classification etc. Most existing statistical topic modeling methods such as LDA and pLSA generate a term based representation to represent a topic by selecting single words from multinomial word distribution over this topic. There are two main shortcomings: firstly, popular or common words occur very often across different topics that bring ambiguity to understand topics; secondly, single words lack coherent semantic meaning to accurately represent topics. In order to overcome these problems, in this paper, we propose a two-stage model that combines text mining and pattern mining with statistical modeling to generate more discriminative and semantic rich topic representations. Experiments show that the optimized topic representations generated by the proposed methods outperform the typical statistical topic modeling method LDA in terms of accuracy and certainty.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Mei, Q., Zhai, C.X.: Discovering Evolutionary Theme Patterns from Text: An Exploration of Temporal Text Mining. In: KDD 2005, pp. 198–207 (2005)
Zhai, C.X., Velivelli, A., Yu, B.: A Cross-collection Mixture Model for Comparative Text Mining. In: KDD 2004, pp. 1285-129 (2004)
Wei, X., Croft, W.B.: LDA-based Document Models for Ad-hoc Retrieval. In: SIGIR 2006, pp. 178–185 (2006)
Wang, X., McCallum, A., Wei, X.: Topical N-grams: Phrase and Topic Discovery, with an Application to Information Retrieval. In: ICDM 2007, pp. 697–702 (2007)
Deerwester, S., et al.: Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science 41(6), 391–407 (1990)
Hofmann, T.: Unsupervised Learning by Probabilistic Latent Semantic Analysis. Machine Learning 42(1), 177–196 (2001)
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet Allocation. Journal of Machine Learning Research 3, 993–1022 (2003)
Blei, D.M., Lafferty, J.D.: Dynamic Topic Models. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 113–120 (2006)
Blei, D.M., Lafferty, J.D.: A Correlated Topic Model of Science. Annals of Applied Statistics 1(1), 17–35 (2007)
Blei, D.M., McAuliffe, J.D.: Supervised Topic Models. In: Adv. NIPS (2007)
Wang, C., Blei, D.M.: Collaborative Topic Modeling for Recommending Scientific articles. In: KDD 2011, pp. 448–456 (2011)
Mei, Q., Shen, X., Zhai, C.: Automatic Labeling of Multinomial Topic Models. In: KDD 2007, pp. 490–499 (2007)
Lau, J.H., et al.: Automatic Labelling of Topic Models. In: Proceeding of the 49th Annual Meeting of the Association for Computational Linguistics, pp. 1536–1545 (2011)
Lau, J.H., et al.: Best Topic Word Selection for Topic Labelling. In: Proceedings of the 23rd International Conference on Computional Linguistics, pp. 605–613 (2010)
Lewis, D.D.: An Evaluation of Phrasal and Clustered Representations on a Text Categorization task. In: SIGIR 1992, 15th ACM International Conference on Research and Development in Information Retrieval, pp. 37–50 (1992)
Scott, S., Matwin, S.: Feature Engineering for Text Classification. In: The 16th International Conference on Machine Learning, pp. 379–388 (1999)
Sebastiani, F.: Machine Learning in Automated Text Categorization. ACM Computing Surveys (CSUR) 34(1), 1–47 (2002)
Wu, S.-T., Li, Y., Xu, Y.: Deploying Approaches for Pattern Refinement in Text Mining. In: IEEE International Conference on Data Mining, pp. 1157–1161 (2006)
Steyvers, M., Griffiths, T.L.: Finding Scientific Topics. Proceedings of the National Academy of Sciences 101, 5228–5235 (2004)
Steyvers, M., Griffiths, T.: Probabilistic Topic Models. Handbook of Latent Semantic Analysis 427(7), 424–440 (2007)
Magatti, D., et al.: Automatic Labeling of Topics. In: Ninth International Conference on Intelligent Systems Design and Applications, pp. 1227–1232 (2009)
Wang, X., McCallum, A., Wei, X.: Topical N-grams: Phrase and Topic Discovery, with an Application to Information Retrieval. In: ICDM 2007, pp. 697–702 (2007)
Zhao, W.X., et al.: Topical keyphrase extraction from Twitter. In: Proceedings of 49th Annual Meeting of the Assocation for Computational Linguistics: Human Language Technologies (HLT 2011) (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gao, Y., Xu, Y., Li, Y., Liu, B. (2013). A Two-Stage Approach for Generating Topic Models. In: Pei, J., Tseng, V.S., Cao, L., Motoda, H., Xu, G. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2013. Lecture Notes in Computer Science(), vol 7819. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37456-2_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-37456-2_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-37455-5
Online ISBN: 978-3-642-37456-2
eBook Packages: Computer ScienceComputer Science (R0)