Topic Representation using Semantic-Based Patterns

  • Dakshi Kapugama GeeganageEmail author
  • Yue Xu
  • Yuefeng Li
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1127)


Topic modelling is the state of the art technique for understanding, organizing, and extracting information from text collections. Traditional topic modeling approaches apply probabilistic techniques to generate the list of topics from collections. Nevertheless, human understands, summarizes and discovers the topics based on the meaning of the content. Hence, the quality of the topic models can be improved by grasping the meaning from the content. In this paper, we propose an approach to identify sets of meaningful terms based on ontology, called Semantic-based Patterns, which represent the content of a collection of documents. A set of related semantic-based patterns can be used to represent a latent topic in the collection. The proposed Topic Representation using Semantic-based Patterns aims to generate semantically meaningful patterns based on ontology rather than term co-occurrence as what existing topic modelling methods do. The semantically meaningful patterns were evaluated by applying the information filtering to semantic-based topic representation. The semantic based patterns were used as features for information filtering and were evaluated by comparing against popular information filtering baseline systems. Topic quality was evaluated in terms of topic coherence and perplexity. The experimental results verified that the quality of the proposed patterns was better than features used in baseline systems for information filtering. Further, the quality of topic representation outperforms the generated topics of other topic modeling approaches.


Patterns Semantics Topic representation Concepts 


  1. 1.
    Evans, J.A., Aceves, P.: Machine translation: mining text for social theory. Ann. Rev. Sociol. 42(1), 21–50 (2016)CrossRefGoogle Scholar
  2. 2.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)zbMATHGoogle Scholar
  3. 3.
    Blei, D.M.: Probabilistic topic models. Commun. ACM 427, 77–84 (2012)CrossRefGoogle Scholar
  4. 4.
    Hofmann, T.: Probabilistic latent semantic indexing. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR 1999. ACM Press (1999)Google Scholar
  5. 5.
    Le, Q.V., Mikolov, T.: Distributed Representations of Sentences and Documents ([n. d.])Google Scholar
  6. 6.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space ([n. d.])Google Scholar
  7. 7.
    Andrzejewski, D., Zhu, X., Craven, M.: Incorporating domain knowledge into topic modeling via Dirichlet forest priors. In: ICML, pp. 25–32. ACM (2009)Google Scholar
  8. 8.
    Gao, Y., Xu, Y., Li, Y.: Pattern-based topics for document modelling in information filtering. IEEE Trans. Knowl. Data Eng. 27(6), 1629–1642 (2015)CrossRefGoogle Scholar
  9. 9.
    Yao, L., Zhang, Y., Wei, B., Qian, H., Wang, Y.: Incorporating probabilistic knowledge into topic models. In: Cao, T., Lim, E.-P., Zhou, Z.-H., Ho, T.-B., Cheung, D., Motoda, H. (eds.) PAKDD 2015. LNCS (LNAI), vol. 9078, pp. 586–597. Springer, Cham (2015). Scholar
  10. 10.
    Tang, Y.K., Mao, X.L., Huang, H., Shi, X., Wen, G.: Conceptualization topic modeling. Multimed. Tools Appl. 77, 3455–3471 (2017)CrossRefGoogle Scholar
  11. 11.
    Gao, Y., Li, Y., Lau, R.Y.K., Xu, Y., Bashar, M.A.: Finding semantically valid and relevant topics by association-based topic selection model. ACM Trans. Intelligent Syst. Technol. 9(1), 1–22 (2017)CrossRefGoogle Scholar
  12. 12.
    Miller, G.A.: WordNet: a lexical database for English. Commun. ACM 38, 39–41 (1995)CrossRefGoogle Scholar
  13. 13.
    Wu, W., Li, H., Wang, H., Zhu, K.Q.: Probase. In: Proceedings of International Conference on Management of Data - SIGMOD 2012. ACM Press (2012)Google Scholar
  14. 14.
    Landauer, T.K., Foltz, P.W., Laham, D.: An introduction to latent semantic analysis. Discourse Process. 25, 259–284 (1998)CrossRefGoogle Scholar
  15. 15.
    Wang, X., McCallum, A., Wei, X.: Topical N-grams: phrase and topic discovery, with an application to information retrieval. In: Seventh IEEE International Conference on Data Mining (2007)Google Scholar
  16. 16.
    Chen, Z., Mukherjee, A., Liu, B., Hsu, M., Castellanos, M., Ghosh, R.: Discovering coherent topics using general knowledge. In: CIKM, pp. 209–218. ACM (2013)Google Scholar
  17. 17.
    Lewis, D.D., Yang, Y., Rose, T.G., Li, F.: RCV1: a new benchmark collection for text categorization research. J. Mach. Learn. Res. 5, 361–397 (2004)Google Scholar
  18. 18.
    Mimno, D., Wallach, H.M., Talley, E., Leenders, M., McCallum, A.: Optimizing semantic coherence in topic models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 262–272 (2011)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Dakshi Kapugama Geeganage
    • 1
    Email author
  • Yue Xu
    • 1
  • Yuefeng Li
    • 1
  1. 1.Queensland University of TechnologyBrisbaneAustralia

Personalised recommendations