Comparing Neural Lexical Models of a Classic National Corpus and a Web Corpus: The Case for Russian
- 2.4k Downloads
In this paper we compare the Russian National Corpus to a larger Russian web corpus composed in 2014; the assumption behind our work is that the National corpus, being limited by the texts it contains and their proportions, presents lexical contexts (and thus meanings) which are different from those found ‘in the wild’ or in a language in use.
To do such a comparison, we used both corpora as training sets to learn vector word representations and found the nearest neighbors or associates for all top-frequency nominal lexical units. Then the difference between these two neighbor sets for each word was calculated using the Jaccard similarity coefficient. The resulting value is the measure of how much the meaning of a given word is different in the language of web pages from the Russian language in the National corpus. About 15% of words were found to acquire completely new neighbors in the web corpus.
In this paper, the methodology of research is described and implications for Russian National Corpus are proposed. All experimental data are available online.
Keywordscorpora comparison deep learning semantic similarity vector representations of lexical units lexical co-occurrence networks Russian National Corpus Web as corpus word2vec
Unable to display preview. Download preview PDF.
- 2.Baroni, M., Ueyama, M.: Building general-and special-purpose corpora by web crawling. In: Proceedings of the 13th NIJL International Symposium, Language Corpora: Their Compilation and Application, pp. 31–40 (2006)Google Scholar
- 3.Belikov, V.: What are sociolinguists and lexicographers lacking in a digitized world? (in Russian). In: Proceedings of the Dialog Conference (2011)Google Scholar
- 4.Sharoff, S.: In the garden and in the jungle: Comparing genres in the bnc and the internet. In: Genres on the Web, pp. 149–166. Springer (2011)Google Scholar
- 5.Belikov, V., Kopylov, N., Piperski, A., Selegey, V., Sharoff, S.: Corpus as language: from scalability to register variation (in Russian). In: Proceeding of the Dialog Conference (2013)Google Scholar
- 6.Baroni, M., Dinu, G., Kruszewski, G.: Don’t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, vol. 1 (2014)Google Scholar
- 7.Kohlschütter, C., Fankhauser, P., Nejdl, W.: Boilerplate detection using shallow text features. In: Proceedings of the Third ACM International Conference on Web Search and Data Mining, pp. 441–450. ACM (2010)Google Scholar
- 8.Segalovich, I.: A fast morphological algorithm with unknown word guessing induced by a dictionary for a web search engine. In: MLMTA, Citeseer, pp. 273–280 (2003)Google Scholar
- 9.Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)Google Scholar
- 10.Curran, J.R.: From distributional to semantic similarity. PhD thesis, University of Edinburgh (2004)Google Scholar
- 11.Lenci, A.: Distributional semantics in linguistic and cognitive research. Italian Journal of Linguistics 20(1), 1–31 (2008)Google Scholar
- 12.Bruni, E., Tran, G.B., Baroni, M.: Distributional semantics from text and images. In: Proceedings of the GEMS 2011 Workshop on GEometrical Models of Natural Language Semantics, pp. 22–32 (2011)Google Scholar
- 14.Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)Google Scholar
- 15.Levy, O., Goldberg, Y.: Dependency-based word embeddings. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, vol. 2 (2014)Google Scholar
- 16.Panchenko, A., Loukachevitch, N.V., Ustalov, D., Paperno, D., Meyer, C.M., Konstantinova, N.: Russe: The first workshop on russian semantic similarity. In: Proceeding of the Dialogue 2015 Conference (2015)Google Scholar
- 17.Jaccard, P.: Distribution de la Flore Alpine: dans le Bassin des dranses et dans quelques régions voisines. Rouge (1901)Google Scholar