Abstract
The paper deals with development and application of automatic word clustering (AWC) tool aimed at processing Russian texts of various types, which should satisfy the requirements of flexibility and compatibility with other linguistic resources. The construction of AWC tool requires computer implementation of latent semantic analysis (LSA) combined with clustering algorithms. To meet the need, Python-based software has been developed. Major procedures performed by AWC tool are segmentation of input texts and context analysis, co-occurrence matrix construction, agglomerative and K-means clustering. Special attention is drawn to experimental results on clustering words in raw texts with changing parameters.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Azarova, I.V., Marina, A.S.: Avtomatizirovannaja klassifikacija kontekstov pri podgotovke dannyh dl’a kompjuternogo tezaurusa RussNet. In: Kompjuternaja lingvistika i intellektual’nyje tehnologii: Trudy meždunarodnoj konferencii Dialog–2006, Moscow, pp. 13–17 (2006)
Baglej, S.G., Antonov, A.V., Meškov, V.S., Suhanov, A.V.: Klasterizacija dokumentov s ispol’zovanijem metainformacii. In: Kompjuternaja lingvistika i intellektual’nyje tehnologii: Trudy meždunarodnoj konferencii Dialog–2006, Moscow, pp. 38–45 (2006)
Križanovskij, A.A.: Avtomatizirovannoje postrojenije spiskov semantičeski blizkih slov na osnove rejtinga tekstov v korpuse s giperssylkami i kategorijami. In: Kompjuternaja lingvistika i intellektual’nyje tehnologii: Trudy meždunarodnoj konferencii Dialog–2006, Moscow, pp. 297–302 (2006)
Landauer, Th., Foltz, P.W., Laham, D.: Introduction to Latent Semantic Analysis. Discourse Processes 25, 259–284 (1998)
Smrž, P., Rychlý, P.: Finding Semantically Related Words in Large Corpora. In: Matoušek, V., Mautner, P., Mouček, R., Tauser, K. (eds.) TSD 2001. LNCS (LNAI), vol. 2166, pp. 108–115. Springer, Heidelberg (2001)
Pekar, V.: Linguistic Preprocessing for Distributional Classification of Words. In: Proceedings of the COLING–04 Workshop on Enhancing and Using Electronic Dictionaries. Geneva, pp. 15–21 (2004)
Stein, B., Niggemann, O.: On the Nature of Structure and its Identification. In: Widmayer, P., Neyer, G., Eidenbenz, S. (eds.) WG 1999. LNCS, vol. 1665, pp. 122–134. Springer, Heidelberg (1999)
Lin, D., Pantel, P.: Induction of Semantic Classes from Natural Language Text. In: Proceedings of ACM Conference on Knowledge Discovery and Data Mining KDD–01. San Francisco, CA, pp. 317–322 (2001)
Shin, S.-I., Choi, K.-S.: Automatic Word Sense Clustering Using Collocation for Sense Adaptation. In: Proceedings of the Second International WordNet Conference GWC–2004, Brno, Czech Republic, pp. 320–325 (2004)
Pantel, P.: Clustering by Committee. Ph.D. Dissertation, Department of Computing Science, University of Alberta (2003), http://www.isi.edu/~pantel/Content/publications.htm
Stein, B., Meyer zu Eissen, S.: Document Categorization with MajorClust. In: Proceedings of the 12th Workshop on Information Technology and Systems WITS–02. Barcelona, Spain, pp. 91–96 (2002)
Azarova, I.V., Sinopal’nikova, A.A.: Ispol’zovanije statistiko-kombinatornyh svojstv korpusa sovremennyh tekstov dl’a formirovanija struktury kompjuternogo tezaurusa RussNet. In: Trudy meždunarodnoj konferencii Korpusnaja lingvistika – 2004, St.-Petersburg, pp. 5–15 (2004)
Doklady naučnoj konferencii Korpusnaja lingvistika i lingvističeskije bazy dannyh – 2002. St.-Petersburg (2002)
Trudy meždunarodnoj konferencii Korpusnaja lingvistika – 2004. St.-Petersburg (2004)
Trudy meždunarodnoj konferencii MegaLing–2005: Prikladnaja lingvistika v poiske novyh putej. St.-Petersburg (2005)
Trudy meždunarodnoj konferencii Korpusnaja lingvistika – 2006. St.-Petersburg (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mitrofanova, O., Mukhin, A., Panicheva, P., Savitsky, V. (2007). Automatic Word Clustering in Russian Texts. In: Matoušek, V., Mautner, P. (eds) Text, Speech and Dialogue. TSD 2007. Lecture Notes in Computer Science(), vol 4629. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74628-7_13
Download citation
DOI: https://doi.org/10.1007/978-3-540-74628-7_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74627-0
Online ISBN: 978-3-540-74628-7
eBook Packages: Computer ScienceComputer Science (R0)