Abstract
In this paper, we first present a dependency-based method for parser evaluation. We then use the method to evaluate a broad-coverage parser, called MINIPAR, with the SUSANNE corpus. The method allows us to evaluate not only the overall performance of the parser, but also its performance with respect to different grammatical relationships and phenomena. The evaluation results show that MINIPAR is able to cover about 79% of the dependency relationships in the SUSANNE corpus with about 89% precision.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bömová, A., Hajič, J., Hajicová, E., Hladká, B. (2003). The Prague Dependency Treebank: A Three level Annotation Scenario. In Abeillé, Anne (ed.) Treebanks: Building and Using Syntactically Annotated Corpora, Kluwer Academic Publishers, this volume.
Berwick, R. C., S. P. Abney, C. Tenny (eds.) (1991). Principle-Based Parsing: Computation and Psycholinguistics. Kluwer Academic Publishers.
Black, E., S. Abney, D. Flickenger, C. Gdaniec, R. Grishman, P. Harrison, D. Hindle, R. Ingria, F. Jelinek, J. Klavans, M. Liberman, M. Marcus, S. Roukos, B. Santorini, T. Strzalkowski (1991). A Procedure for Quantitatively Comparing the Syntactic Coverage of English Grammars. Proceedings of Speech and Natural Language Workshop, p. 306–311.
Carroll, J., G. Minnen, T. Briscoe (1999). Corpus Annotation for Parser Evaluation. Proceedings of the EACL Workshop on Linguistically Interpreted Corpora. Bergen, Norway.
Chomsky, N. (1995). The Minimalist Program. MIT Press.
Collins, M. J. (1996). ‘A New Statistical Parser Based on Bigram Lexical Dependencies’. Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics. Santa Cruz, CA, p. 184–191.
Collins, M. J. (1997). Three Generative, Lexicalized Models for Statistical Parsing. Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics. Madrid, Spain, p. 16–23.
Hays, D. (1964). Dependency theory: a formalism and some observations. Language 40, p. 511–525.
Hudson, R. (1984). Word Grammar. Oxford, England: Basil Blackwell Publishers Limited.
Lin, D. (1993). Principle-based Parsing without Overgeneration. Proceedings of ACL-93. Columbus, Ohio, pp. 112–120.
Lin, D. (1994) PRINCIPAR—An Efficient, Broad-coverage, Principle-based Parser. Proceedings of COLING-94. Kyoto, Japan: p. 482–488.
Lin, D. (1995) A Dependency-based Method for Evaluating Broad-coverage Parsers. Proceedings of IJCAI-95. Montreal, Canada, p. 1420–1425.
Lin, D. (1998). A Dependency-based Method for Evaluating Broad-Coverage Parsers. Journal of Natural Language Engineering, p. 97–114.
Magerman, D. (1995). Statistical Decision-Tree Model for Parsing. Proceedings of ACL-96. Cambridge, MA, p. 276–283.
Marcus, M. P., B. Santorini, M. A. Marcinkiewicz (1993). Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics 19(2), p. 313–330.
Mel’čuk, I. A. (1987). Dependency syntax: theory and practice. Albany: State University of New York Press.
Miller, G. A., R. Beckwith, C. Fellbaum, D. Gross, K. J. Miller (1990). ‘Introduction to WordNet: An on-line lexical database’. International Journal of Lexicography 3(4), p. 235–244.
Sampson, G. (2003). Thoughts on two decades of drawing trees. This volume.
Sampson, G. R. (1995). English for the Computer. Oxford University Press.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Lin, D. (2003). Dependency-Based Evaluation of Minipar. In: Abeillé, A. (eds) Treebanks. Text, Speech and Language Technology, vol 20. Springer, Dordrecht. https://doi.org/10.1007/978-94-010-0201-1_18
Download citation
DOI: https://doi.org/10.1007/978-94-010-0201-1_18
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-1335-5
Online ISBN: 978-94-010-0201-1
eBook Packages: Springer Book Archive