Abstract
Arabic language analyzers have been studied constructed and studied by many researchers. Most of research projects and commercial applications in the area of Arabic Natural Language Processing (ANLP) build their own analyzer. Typically they are intended for just one project or application and can not be generalized to work in other areas of ANLP. All of these analyzers didn’t cover our requirements from the analyzer which make us to build a new one. Our analyzer is also a part of a complete Arabic tagging system. It receives the output of a tokenizer in the form of inflected words, and produces for each of them a set of several possible analyzes, consisting of: a POS tag, features and a lemma. It differs from most of the existing analyzers because it produces a lemma rather than stem or root, which is a significantly harder task in Arabic, and because POS and features are described by a new very rich tagset, described in a previous publication. The test dataset was a small corpus of 16 k words, manually annotated by a single analysis for each word, correct for this particular use of that word. In the test, for 99.67% of words, the correct analysis was among those produced by the analyzer. On the other hand, in a manual verification of the output of the analyzer, only 0.1% of all analyses were grammatically incorrect.
It is part of PhD dissertation of Whole Arabic Tagging System with rich tagset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Tarek, E., Fatma, E.: An Accurate Arabic Root-Based Lemmatizer for Information Retrieval Purposes. IJCSI International Journal of Computer Science Issues 9(1,3) (January 2012)
Khoja, S.: Stemming Arabic Text. Computing Department, Lancaster University, Lancaster (1999)
Nizar, H., Owen, R., George, K.: Morphological Analysis and Generation for Arabic Dialects. In: Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, Michigan, USA (2005)
Kareem, D.: Building a Shallow Arabic Morphological Analyzer in One Day. In: Proceedings of the ACL 2002 Workshop on Computational Approaches to Semitic Languages, PA, USA (2002)
Mourad, G., Noureddine, C.: Developing a New System for Arabic Morphological Analysis and Generation. In: Proceedings of the 2nd Workshop on South and Southeast Asian Natural Language Processing (WSSANLP), IJCNLP 2011, Chiang Mai, Thailand, pp. 52–57 (2011)
Otakar, S.: Functional Arabic Morphology, Formal System and Implementation. Doctoral Thesis, Institute of Formal and Applied Linguistics, Faculty of Mathematics and Physics, Charles University, Prague (2007)
Buckwalter, T.: Isues in Arabic Orthography and morphology Analysis. In: Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages, COLING, Geneva (2004)
El-Dahdah, A.: An Intermediate Dictionary of Verb Conjugation, 1st edn. Libaririe Du Liban, Beirut (1994) (in Aabic)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Aliwy, A.H. (2012). Arabic Language Analyzer with Lemma Extraction and Rich Tagset. In: Isahara, H., Kanzaki, K. (eds) Advances in Natural Language Processing. JapTAL 2012. Lecture Notes in Computer Science(), vol 7614. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33983-7_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-33983-7_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33982-0
Online ISBN: 978-3-642-33983-7
eBook Packages: Computer ScienceComputer Science (R0)