Part of Speech and Gramset Tagging Algorithms for Unknown Words Based on Morphological Dictionaries of the Veps and Karelian Languages

Abstract

This research devoted to the low-resource Veps and Karelian languages. Algorithms for assigning part of speech tags to words and grammatical properties to words are presented in the article. These algorithms use our morphological dictionaries, where the lemma, part of speech and a set of grammatical features (gramset) are known for each word form. The algorithms are based on the analogy hypothesis that words with the same suffixes are likely to have the same inflectional models, the same part of speech and gramset. The accuracy of these algorithms were evaluated and compared. 66 thousand Karelian and 313 thousand Vepsian words were used to verify the accuracy of these algorithms. The special functions were designed to assess the quality of results of the developed algorithms. 86.8% of Karelian words and 92.4% of Vepsian words were assigned a correct part of speech by the developed algorithm. 90.7% of Karelian words and 95.3% of Vepsian words were assigned a correct gramset by our algorithm. Morphological and semantic tagging of texts, which are closely related and inseparable in our corpus processes, are described in the paper.