Abstract
Text clustering has many applications in various areas. Before being clustered, texts often have already been grouped or partially grouped in practise. Texts from the same group are related to each other and concentrate on a few topics. The group information turns out to be valuable for text clustering. In this paper, we propose a model called Non-parametric Gaussian Mixture Experts to get better clustering result through utilizing group information. After converting texts to vectors by semantic embedding, our model can automatically infer proper cluster number for every group and the whole corpus. We develop an online variational inference algorithm which is scalable and can handle incremental datasets. Our algorithm is tested on various text datasets. The results demonstrate our model has significantly better performance in cluster quality than some other classical and recent text clustering methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
http://news.google.com/, We obtain it from the author of [12].
- 3.
- 4.
References
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv: 1301.3781 (2013)
Le, Q.V., Mikolov, T.: Distributed representations of sentences, documents. arXiv preprint arXiv: 1405.4053 (2014)
Pennington, J., Socher, R., Manning, C.D., Glove: global vectors for word representation. In: Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014), vol. 12 (2014)
Rasmussen, C.E.: The infinite gaussian mixture model. In: NIPS, vol. 12, pp. 554–560 (1999)
Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Hierarchical dirichlet processes. J. Am. Stat. Assoc. 101(476), 1566–1581 (2006)
Sethuraman, J.: A constructive definition of dirichlet priors. Technical report, DTIC Document (1991)
Hoffman, M.D., Blei, D.M., Wang, C., Paisley, J.: Stochastic variational inference. J. Mach. Learn. Res. 14(1), 1303–1347 (2013)
Blei, D.M., Jordan, M.I., et al.: Variational inference for dirichlet process mixtures. Bayesian Anal. 1(1), 121–143 (2006)
Amari, S.-I.: Natural gradient works efficiently in learning. Neural Comput. 10(2), 251–276 (1998)
Yin, J., Wang, J.: A dirichlet multinomial mixture model-based approach for short text clustering. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 233–242. ACM (2014)
Kuang, D., Park, H.: Fast rank-2 nonnegative matrix factorization for hierarchical document clustering. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 739–747. ACM (2013)
Blei, D.M.: Mcauliffe, J.D.: Supervised topic models. In: Neural Information Processing Systems (2007)
Perotte, A.J., Wood, F., Elhadad, N., Bartlett, N.: Hierarchically supervised latent Dirichlet allocation. In: Advances in Neural Information Processing Systems, pp. 2609–2617 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Tian, Y., Rong, Y., Yao, Y., Liu, W., Song, J. (2016). Grouped Text Clustering Using Non-Parametric Gaussian Mixture Experts. In: Booth, R., Zhang, ML. (eds) PRICAI 2016: Trends in Artificial Intelligence. PRICAI 2016. Lecture Notes in Computer Science(), vol 9810. Springer, Cham. https://doi.org/10.1007/978-3-319-42911-3_42
Download citation
DOI: https://doi.org/10.1007/978-3-319-42911-3_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-42910-6
Online ISBN: 978-3-319-42911-3
eBook Packages: Computer ScienceComputer Science (R0)