Abstract
We present a neural network based natural language parser. Training the neural network induces hidden representations of unbounded partial parse histories, which are used to estimate probabilities for parser decisions. This induction process is given domain-specific biases by matching the flow of information in the network to structural locality in the parse tree, without imposing any independence assumptions. The parser achieves performance on the benchmark datasets which is roughly equivalent to the best current parsers.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
E.K.S. Ho and L.W. Chan. How to design a connectionist holistic parser. Neural Computation, 11(8):1995–2016, 1999.
F. Costa, V. Lombardo, P. Frasconi, and G. Soda. Wide coverage incremental parsing by learning attachment preferences. In Proc. of the Conf. of the Italian Association for Artificial Intelligence, 2001.
Peter Lane and James Henderson. Incremental syntactic parsing of natural language corpora with simple synchrony networks. IEEE Transactions on Knowledge and Data Engineering, 13(2):219–231, 2001.
James Henderson. Aneural network parser that handles sparse data. In Proc. 6th Int. Workshop. on Parsing Technologies, pages 123–134, Trento, Italy, 2000.
Adwait Ratnaparkhi. Learning to parse natural language with maximum entropy models. Machine Learning, 34:151–175, 1999.
Michael Collins. Head-Driven Statistical Models for Natural Language Parsing. PhD thesis, University of Pennsylvania, Philadelphia, PA, 1999.
Jeffrey L. Elman. Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7:195–225, 1991.
James Henderson. Inducing history representations for broad coverage statistical parsing. In Proc. joint meeting of North American Chapter of the Association for Computational Linguistics and the Human Language Technology Conf., Edmonton, Canada, 2003.
D.J. Rosenkrantz and P.M. Lewis. Deterministic left corner parsing. In Proc. 11th Symposium on Switching and Automata Theory, pages 139–152, 1970.
Christopher M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, Oxford, UK, 1995.
Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330, 1993.
Adwait Ratnaparkhi. Amaximum entropy model for part-of-speech tagging. In Proc. Conf. on Empirical Methods in Natural Language Processing, pages 133–142, Univ. of Pennsylvania, PA, 1996.
Eugene Charniak. Statistical parsing with a context-free grammar and word statistics. In Proc. 14th National Conference on Artificial Intelligence, Providence, RI, 1997. AAAI Press/MIT Press.
Michael Collins. Discriminative reranking for natural language parsing. In Proc. 17th Int. Conf. on Machine Learning, pages 175–182, Stanford, CA, 2000.
P. Frasconi, M. Gori, and A. Sperduti. A general framework for adaptive processing of data structures. IEEE Transactions on Neural Networks, 9:768–786, 1998.
Lokendra Shastri and Venkat Ajjanagadde. From simple associations to systematic reasoning: A connectionist representation of rules, variables, and dynamic bindings using temporal synchrony. Behavioral and Brain Sciences, 16:417–451, 1993.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Henderson, J. (2003). Structural Bias in Inducing Representations for Probabilistic Natural Language Parsing. In: Kaynak, O., Alpaydin, E., Oja, E., Xu, L. (eds) Artificial Neural Networks and Neural Information Processing — ICANN/ICONIP 2003. ICANN ICONIP 2003 2003. Lecture Notes in Computer Science, vol 2714. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44989-2_3
Download citation
DOI: https://doi.org/10.1007/3-540-44989-2_3
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40408-8
Online ISBN: 978-3-540-44989-8
eBook Packages: Springer Book Archive