Abstract
We present an experiment evaluating the contribution of a system called GReG for reranking the snippets returned by Google’s search engine in the 10 hits presented to the user and captured by the use of Google’s API. The evaluation aims at establishing whether or not the introduction of deep linguistic information may improve the accuracy of Google or rather it is the opposite case as maintained by the majority of people working in Information Retrieval and using a Bag Of Words approach. We used 900 questions and answers taken from TREC 8 and 9 competitions and execute three different types of evaluation: one without any linguistic aid; a second one with tagging and syntactic constituency contribution; another run with what we call Partial Logical Form. Even though GReG is still work in progress, it is possible to draw clear cut conclusions: adding linguistic information to the evaluation process of the best snippet that can answer a question improves enormously the performance. In another experiment we used the actual texts associated to the Q/A pairs distributed by one of TREC’s participant and got even higher accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bresnan, J.: Lexical-Functional Syntax. Blackwell, Malden (2000)
ComLex, http://nlp.cs.nyu.edu/comlex
Baker, C.F., Fillmore, C.J., Lowe, J.B.: The Berkeley FrameNet project. In: Proceedings of COLING-ACL 1998, Montreal, Canada (1998)
Ellsworth, M., Erk, K., Kingsbury, P., Pado, S.: PropBank, SALSA, and FrameNet: How design determines product. In: Proceedings of the LREC 2004 Workshop on Building Lexical Resources from Semantically Annotated Corpora, Lisbon (2004)
Fellbaum, C. (ed.): WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)
Delmonte, R. (ed.): Computational Linguistic Text Processing – Logical Form, Semantic Interpretation, Discourse Relations and Question Answering. Nova Science Publishers, New York (2007)
Delmonte, R.: Computational Linguistic Text Processing – Lexicon, Grammar, Parsing and Anaphora Resolution. Nova Science Publishers, New York (2009)
Delmonte, R.: Deep & Shallow Linguistically Based Parsing. In: Delmonte, R., Di Sciullo, A.M. (eds.) UG and External Systems, pp. 335–374. John Benjamins, Amsterdam (2005)
Delmonte, R., Bristot, A., Piccolino Boniforti, M.A., Tonelli, S.: Entailment and Anaphora Resolution in RTE3. In: ACL Workshop on Text Entailment and Paraphrasing, Prague, ACL Madison, USA, pp. 48–53 (2007)
Cui, H., Sun, R., Li, K., Kan, M.-Y., Chua, T.-S.: Question Answering Pas-sage Retrieval Using Dependency Relations. In: SIGIR 2005, pp. 400–406. ACM, Salvador (2005)
Litkowski, K.C.: Syntactic Clues and Lexical Resources in Question-Answering. In: Voorhees, E.M., Harman, D.K. (eds.) The Ninth Text Retrieval Conference (TREC-9), pp. 157–166. NIST Special Publication, Gaithersburg (2001)
Wang, M., Smith, N.A., Mitamura, T.: What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Prague, pp. 22–32 (2007)
Traum, D., Habash, N.: Generation from Lexical Conceptual Structure. In: Workshop on Applied Interlinguas, ANLP 2000, Seattle, WA, pp. 123–134 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Delmonte, R., Tripodi, R. (2011). Linguistically-Based Reranking of Google’s Snippets with GreG. In: Pallotta, V., Soro, A., Vargiu, E. (eds) Advances in Distributed Agent-Based Retrieval Tools. Studies in Computational Intelligence, vol 361. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21384-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-21384-7_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21383-0
Online ISBN: 978-3-642-21384-7
eBook Packages: EngineeringEngineering (R0)