Abstract
The Latent Dirichlet Allocation (LDA) is a parametric approach and the number of topics must be predefined. So it is natural to try to capture uncertainty regarding the number of topics. This paper proposes a Tag Hierarchical Dirichlet Process (THDP) that automatically infers the number of topics while also leveraging the tag information associated with each document. In this model, we assume that an author is clear in his mind that the content will contains which aspects and for each aspect he will choose a tag to describe it, and then we consider problems involving groups of tag, where each tag within a group is a draw from a mixture model and it is desirable to share topic between groups. In this setting it is natural to consider Hierarchical Dirichlet Process, Where the well-known clustering property of the Dirichlet process provides a nonparametric prior for the number of topic within each tag. Experimental results on corpora demonstrate superior performance over the THDP model.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
About Tumblr.com (accessed October 13, 2012)
WordPress.com Stats (accessed October 13, 2012)
Tingting, H., Fang, L.: Semantic KnowledgeAcquisition from Blogs with Tag-Topic Model. China Communications 9(3), 38–48 (2012)
The, Y.W., Jordan, M., Beal, M., Blei, D.M.: Hierarchical Dirichlet processes. Journal of the American Statistical Association 101(476), 1566–1581 (2006)
Wang, X., Tieu, K., Gee-Wah, N., Grimson, W.E.L.: Trajectoryanalysis and semantic region modeling using a nonparametric Bayesian model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE, Anchorage (2008)
Sudderth, E.B., Torralba, A., Freeman, W.T., Willsky, A.S.: Describing visual scenes using transformed objects and parts. International Journal of Computer Vision 77(1-3), 291–330 (2008)
Sudderth, E.B.: Graphical Models for Visual Object Recognition and Tracking [Ph. D. dissertation]. Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, USA (2006)
Yakhnenko, O., Honavar, V.: Multimodal hierarchical Dirichlet process model for predicting image annotation andimageobject label correspondence. In: Proceedings of the SIAM International Conference on Data Mining, pp. 281–294. SIAM, Sparks (2009)
Cowans, P.J.: Information retrieval using hierarchical Dirichlet processes. In: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 564–565. ACM, New York (2004)
Xu, T.B., Zhang, Z.F., Yu, P.S., Long, B.: Evolutionary clusteringby hierarchical Dirichlet process with hidden Markov state. In: Proceedings of the 8th IEEE International Conferenceon Data Mining, pp. 658–667. IEEE, Pisa (2008)
Williamson, S., Wang, C., Heller, K.A., Blei, D.M.: The IBP compound Dirichlet process and itsapplication to focused topic modeling. In: ICML (2010)
Chan, X., Zhou., M., Carin, L.: The contextual Focused Topic Model. In: KDD, pp. 96–104 (2012)
Issei, S., Kenichi, K., Hiroshi, N.: Practical Collapsed Variational Bayes Inference for Hierarchical Dirichlet Process. In: KDD, pp. 105–113 (2012)
Turney, P.D.: Mining the Web for Synonyms: PMI- IR versus LSA on TOEFL. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, pp. 491–502. Springer, Heidelberg (2001)
Heinrich, G.: “Infinite LDA”– Implementing the HDP with minimum code complexity (2011)
Deng, H., Han, J., Zhao, B., Yu, Y., Lin, C.: Probabilistic topic models with biased propagation onheterogeneous information networks. In: KDD (2011)
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet Allocation. Journal of Machine Learning Research 3(4-5), 993–1022 (2003)
Heinrich, G.: Parameter estimation for text analysis (2005)
Steyvers, M., Griffiths, T.: Probabilistic topic models. Handbook of latent semantic analysis 427(7), 424–440 (2007)
Ranganathan, A.: The Dirichlet Process Mixture (DPM) Model (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, M., He, T., Li, F., Peng, L. (2013). Incorporating Hierarchical Dirichlet Process into Tag Topic Model. In: Liu, P., Su, Q. (eds) Chinese Lexical Semantics. CLSW 2013. Lecture Notes in Computer Science(), vol 8229. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-45185-0_39
Download citation
DOI: https://doi.org/10.1007/978-3-642-45185-0_39
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-45184-3
Online ISBN: 978-3-642-45185-0
eBook Packages: Computer ScienceComputer Science (R0)