Abstract
Background. Topic modelling is a method of automated probabilistic detection of topics in a text collection. Use of topic modelling for short texts, e.g. tweets or search engine queries, is complicated due to their short length and grammatical flaws, including broken word order, abbreviations, and contamination of different languages. At the same time, as our research shows, human coding cannot be perceived as a baseline for topic quality assessment. Objectives. We use biterm topic model (BTM) to test the relations between two topic quality metrics independent from topic coherence with the human topic interpretability. Topic modelling is applied to three cases of conflictual Twitter discussions in three different languages, namely the Charlie Hebdo shooting (France), the Ferguson unrest (the USA), and the anti-immigrant bashings in Biryulevo (Russia), which represent, respectively, a global multilingual, a large monolingual, and a mid-range monolingual type of discussions. Method. First, we evaluate the human baseline coding by providing evidence for the Russian case on the coding by two pairs of coders who have varying levels of knowledge of the case. We then measure the quality of modelling on the level of topics by looking at topic interpretability (by experienced coders), topic robustness, and topic saliency. Results. The results of the experiment show that: 1) the idea of human coding as baseline needs to be rejected; 2) topic interpretability, robustness, and saliency can be inter-related; 3) the multilingual discussion performs better than the monolingual ones in terms of interdependence of the metrics. Conclusion. We formulate the idea of an ‘ideal topic’ that rethinks the goal of topic modelling towards finding a smaller number of good topics rather instead of maximization of the number of interpretable topics.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ramage, D., Dumais, S., Liebling, D.: Characterizing microblogs with topic models. In: Proceedings of the Fourth International AAAI Conference on Weblogs and Social Media, pp. 130–137. AAAI (2010)
Blekanov, I., Tarasov, N., Maksimov, A.: Topic modeling of conflict Ad Hoc discussions in social networks. In: Proceedings of the 3rd International Conference on Applications in Information Technology, pp. 122–126. ACM (2018)
Koltsova, O., Koltcov, S.: Mapping the public agenda with topic modeling: the case of the Russian live journal. Policy Internet 5(2), 207–227 (2013)
Jonnagaddala, J., Jue, T.R., Dai, H.J.: Binary classification of Twitter posts for adverse drug reactions. In: Proceedings of the Social Media Mining Shared Task Workshop at the Pacific Symposium on Biocomputing, pp. 4–8 (2016)
Ligutom, C., Orio, J.V., Ramacho, D.A.M., Montenegro, C., Roxas, R.E., Oco, N.: Using Topic Modelling to make sense of typhoon-related tweets. In: Proceedings of the 2016 International Conference on Asian Language Processing (IALP), pp. 362–365. IEEE (2016)
Maceda, L.L., Llovido, J.L., Palaoag, T.D.: Corpus analysis of earthquake related Tweets through topic modelling. Int. J. Mach. Learn. Comput. 7(6), 194–197 (2017)
Mazarura, J.R., De Waal, A., Kanfer, F., Millard, S.M.: Topic modelling for short text (2015). researchgate.net/publication/279195527_Topic_Modelling_for_Short_Text
Lutovinova, O.V.: Internet as a new ‘oral-written’ system of communication. Bull. Russ. State Pedagogical Univ. 71, 58–65 (2008). https://cyberleninka.ru/article/n/internet-kak-novaya-ustno-pismennaya-sistema-kommunikatsii
Smoliarova, A.S., Bodrunova, S.S., Yakunin, A.V., Blekanov, I., Maksimov, A.: Detecting pivotal points in social conflicts via topic modeling of Twitter content. In: Bodrunova, S.S., et al. (eds.) INSCI 2018. LNCS, vol. 11551, pp. 61–71. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-17705-8_6
Sridhar, V.K.R.: Unsupervised topic modeling for short texts using distributed representations of words. In: Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pp. 192–200 (2015)
Acknowledgements
This research has been supported in full by Russian Science Foundation, grant 16-18-10125-P (2016–2018, prolonged to 2019–2020).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Blekanov, I.S., Bodrunova, S.S., Zhuravleva, N., Smoliarova, A., Tarasov, N. (2020). The Ideal Topic: Interdependence of Topic Interpretability and Other Quality Features in Topic Modelling for Short Texts. In: Meiselwitz, G. (eds) Social Computing and Social Media. Design, Ethics, User Behavior, and Social Network Analysis. HCII 2020. Lecture Notes in Computer Science(), vol 12194. Springer, Cham. https://doi.org/10.1007/978-3-030-49570-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-49570-1_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-49569-5
Online ISBN: 978-3-030-49570-1
eBook Packages: Computer ScienceComputer Science (R0)