Skip to main content
Book cover

European Semantic Web Conference

ESWC 2019: The Semantic Web pp 346–362Cite as

Knowledge-Based Short Text Categorization Using Entity and Category Embedding

Part of the Lecture Notes in Computer Science book series (LNISA,volume 11503)

Abstract

Short text categorization is an important task due to the rapid growth of online available short texts in various domains such as web search snippets, etc. Most of the traditional methods suffer from sparsity and shortness of the text. Moreover, supervised learning methods require a significant amount of training data and manually labeling such data can be very time-consuming and costly. In this study, we propose a novel probabilistic model for Knowledge-Based Short Text Categorization (KBSTC), which does not require any labeled training data to classify a short text. This is achieved by leveraging entities and categories from large knowledge bases, which are further embedded into a common vector space, for which we propose a new entity and category embedding model. Given a short text, its category (e.g. Business, Sports, etc.) can then be derived based on the entities mentioned in the text by exploiting semantic similarity between entities and categories. To validate the effectiveness of the proposed method, we conducted experiments on two real-world datasets, i.e., AG News and Google Snippets. The experimental results show that our approach significantly outperforms the classification approaches which do not require any labeled data, while it comes close to the results of the supervised approaches.

Keywords

  • Short text classification
  • Dataless text classification
  • Network embeddings

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-21348-0_23
  • Chapter length: 17 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-21348-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.

Notes

  1. 1.

    http://goo.gl/JyCnZq.

  2. 2.

    http://jwebpro.sourceforge.net/data-web-snippets.tar.gz.

  3. 3.

    https://en.wikipedia.org/wiki/Category:Sports.

References

  1. Burel, G., Saif, H., Alani, H.: Semantic wide and deep learning for detecting crisis-information categories on social media. In: d’Amato, C., et al. (eds.) ISWC 2017. LNCS, vol. 10587, pp. 138–155. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68288-4_9

    CrossRef  Google Scholar 

  2. Chang, M.W., Ratinov, L.A., Roth, D., Srikumar, V.: Importance of semantic representation: dataless classification. In: AAAI (2008)

    Google Scholar 

  3. Gabrilovich, E., Markovitch, S.: Computing semantic relatedness using wikipedia-based explicit semantic analysis. In: IJCAI (2007)

    Google Scholar 

  4. Li, C., Xing, J., Sun, A., Ma, Z.: Effective document labeling with very few seed words: a topic model approach. In: CIKM (2016)

    Google Scholar 

  5. Li, Y., Zheng, R., Tian, T., Hu, Z., Iyer, R., Sycara, K.P.: Joint embedding of hierarchical categories and entities for concept categorization and dataless classification. In: COLING (2016)

    Google Scholar 

  6. Mendes, P.N., Jakob, M., García-Silva, A., Bizer, C.: Dbpedia spotlight: shedding light on the web of documents. In: I-SEMANTICS (2011)

    Google Scholar 

  7. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: NIPS (2013)

    Google Scholar 

  8. Nigam, K., McCallum, A., Thrun, S., Mitchell, T.M.: Text classification from labeled and unlabeled documents using EM. Mach. Learn. 39, 103–134 (2000)

    CrossRef  Google Scholar 

  9. Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: KDD (2014)

    Google Scholar 

  10. Phan, X.H., Nguyen, L.M., Horiguchi, S.: Learning to classify short and sparse text & web with hidden topics from large-scale data collections. In: WWW (2008)

    Google Scholar 

  11. Ristoski, P., Paulheim, H.: RDF2Vec: RDF graph embeddings for data mining. In: Groth, P., et al. (eds.) ISWC 2016. LNCS, vol. 9981, pp. 498–514. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46523-4_30

    CrossRef  Google Scholar 

  12. Röder, M., Usbeck, R., Hellmann, S., Gerber, D., Both, A.: N\({^3}\) - a collection of datasets for named entity recognition and disambiguation in the NLP interchange format. In: LREC (2014)

    Google Scholar 

  13. Song, G., Ye, Y., Du, X., Huang, X., Bie, S.: Short text classification: a survey. J. Multimedia 9(5), 635–644 (2014)

    CrossRef  Google Scholar 

  14. Song, Y., Roth, D.: On dataless hierarchical text classification. In: AAAI (2014)

    Google Scholar 

  15. Tang, J., Qu, M., Mei, Q.: PTE: predictive text embedding through large-scale heterogeneous text networks. In: KDD (2015)

    Google Scholar 

  16. Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: LINE: large-scale information network embedding. In: WWW (2015)

    Google Scholar 

  17. Türker, R., Zhang, L., Koutraki, M., Sack, H.: TECNE: knowledge based text classification using network embeddings. In: EKAW (2018)

    Google Scholar 

  18. Türker, R., Zhang, L., Koutraki, M., Sack, H.: “The less is more” for text classification. In: SEMANTiCS (2018)

    Google Scholar 

  19. Usbeck, R., et al.: GERBIL: general entity annotator benchmarking framework. In: WWW (2015)

    Google Scholar 

  20. Wang, J., Wang, Z., Zhang, D., Yan, J.: Combining knowledge with deep convolutional neural networks for short text classification. In: IJCAI (2017)

    Google Scholar 

  21. Wang, P., Xu, B., Xu, J., Tian, G., Liu, C.L., Hao, H.: Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification. Neurocomputing 174, 806–814 (2016)

    CrossRef  Google Scholar 

  22. Xuan, J., Jiang, H., Ren, Z., Yan, J., Luo, Z.: Automatic bug triage using semi-supervised text classification. In: SEKE (2010)

    Google Scholar 

  23. Zhang, X., LeCun, Y.: Text understanding from scratch. CoRR (2015)

    Google Scholar 

  24. Zhang, X., Wu, B.: Short text classification based on feature extension using the n-gram model. In: FSKD. IEEE (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. FIZ Karlsruhe – Leibniz Institute for Information Infrastructure, Karlsruhe, Germany

    Rima Türker, Lei Zhang, Maria Koutraki & Harald Sack

  2. Karlsruhe Institute of Technology, Institute AIFB, Karlsruhe, Germany

    Rima Türker, Maria Koutraki & Harald Sack

  3. L3S Research Center, Leibniz University of Hannover, Hannover, Germany

    Maria Koutraki

Authors
  1. Rima Türker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maria Koutraki
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Harald Sack
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rima Türker .

Editor information

Editors and Affiliations

  1. Wright State University, Dayton, OH, USA

    Prof. Dr. Pascal Hitzler

  2. KMi, The Open University, Milton Keynes, UK

    Miriam Fernández

  3. University of California, Santa Barbara, CA, USA

    Krzysztof Janowicz

  4. Maastricht University, Maastricht, The Netherlands

    Amrapali Zaveri

  5. Heriot-Watt University, Edinburgh, UK

    Alasdair J.G. Gray

  6. IBM Research, Dublin, Ireland

    Vanessa Lopez

  7. The Australian National University, Canberra, ACT, Australia

    Armin Haller

  8. Jönköping University, Jönköping, Sweden

    Karl Hammar

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Türker, R., Zhang, L., Koutraki, M., Sack, H. (2019). Knowledge-Based Short Text Categorization Using Entity and Category Embedding. In: , et al. The Semantic Web. ESWC 2019. Lecture Notes in Computer Science(), vol 11503. Springer, Cham. https://doi.org/10.1007/978-3-030-21348-0_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-21348-0_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-21347-3

  • Online ISBN: 978-3-030-21348-0

  • eBook Packages: Computer ScienceComputer Science (R0)