Abstract
Text clustering is a big challenge in the text mining field; traditional algorithms are powerless when dealing with short texts. Short messages are a much more flexible form of data in social media, containing not only textual information, but also comment, time and regional information. We propose an algorithm to extract semantic and multidimensional feature representation from such texts. In particular, by using the fact that comments are semantically related to the short message, we can get the supervised information and train the text representation, with which we transform the problem into a semi-supervised problem. We use a convolutional-pooling structure that aims at mapping the text into a semantic representation. What’s more, we expand the semantic representation with time- and region-related features, leading to a much more flexible and strong representation for short messages. Our approach shows great advantages in labelled data over traditional feature representation methods and performs better than other clustering methods via deep neural network representation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuglu, K., Kuksa, P.: Natural language processing (Almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)
Xing, E.P., Jordan, M.I., Russell, S.J., Ng, A.Y.: Distance metric learning with application to clustering with side-information. In: Proceedings of Advances in Neural Information Processing Systems, pp. 521–528 (2003)
Ying, Y., Li, P.: Distance metric learning with eigenvalue optimization. J. Mach. Learn. Res. 1–26 (2012)
Gabrilovich, E., Markovitch, S.: Computing semantic relatedness of words and texts in Wikipedia-derived semantic space. In: Proceedings of IJCAI, vol. 7, pp. 1606–1611 (2006)
Xu, J., Wang, P., Tian, G., et al.: Short text clustering via convolutional neural networks. In: Proceedings of NAACL-HLT, pp. 62–69 (2015)
Bair, E.: Semi-supervised clustering methods. Wiley Interdiscip. Rev. Comput. Stat. 5(5), 349–361 (2013)
Wang, Z., Mi, H., Ittycheriah, A.: Semi-supervised clustering for short text via deep representation learning. CoNLL 2016, 31 (2016)
Wang, Z., Mi, H., Ittycheriah, A.: Semi-supervised clustering for short text via deep representation learning (2016). arXiv:1602.06797
Hartigan, J.A., Wong, M.A.: Algorithm AS 136: a k-means clustering algorithm. J. Roy. Stat. Soc. Ser. C (Appl. Stat.) 28(1), 100–108 (1979)
Pelleg, D., Baras, D.: K-means with large and noisy constraint sets. In: European Conference on Machine Learning, pp. 674–682. Springer, Berlin, Heidelberg (2007)
Johnson, S.C.: Hierarchical clustering schemes. Psychometrika 32(3), 241–254 (1967)
Ester, M., Kriegel, H.P., Sander, J., Xu, X.: Density-based spatial clustering of applications with noise. In: International Conference on Knowledge Discovery and Data Mining, vol. 240 (1996)
Bilenko, M., Basu, S., Mooney, R.J.: Integrating constraints and metric learning in semi-supervised clustering. In: Proceedings of the Twenty-First International Conference on Machine Learning, p. 11. ACM (2004)
Vinh, N.X., Epps, J., Bailey, J.: Information theoretic measures for clusterings comparison: is a correction for chance necessary? In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 1073–1080. ACM (2009)
Ji, X., Xu, W.: Document clustering with prior knowledge. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 405–412. ACM (2006)
Kim, Y.: Convolutional neural networks for sentence classification (2014). arXiv:1408.5882
Huang, P.S., He, X., Gao, J., Deng, L., Acero, A., Heck, L.: Learning deep structured semantic models for web search using clickthrough data. In: Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, pp. 2333–2338. ACM. (2013)
Shen, Y., He, X., Gao, J., Deng, L., Mesnil, G.: A latent semantic model with convolutional-pooling structure for information retrieval. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 101–110. ACM (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wu, S., Zhang, H., Xu, C., Guo, T. (2019). Text Clustering on Short Message by Using Deep Semantic Representation. In: Bhatia, S., Tiwari, S., Mishra, K., Trivedi, M. (eds) Advances in Computer Communication and Computational Sciences. Advances in Intelligent Systems and Computing, vol 760. Springer, Singapore. https://doi.org/10.1007/978-981-13-0344-9_11
Download citation
DOI: https://doi.org/10.1007/978-981-13-0344-9_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-0343-2
Online ISBN: 978-981-13-0344-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)