Question Answering By Predictive Annotation

  • John Prager
  • Jennifer Chu-Carroll
  • Eric W. Brown
  • Krzysztof Czuba
Part of the Text, Speech and Language Technology book series (TLTB, volume 32)

We present in this chapter a description of the major components of a Question-Answering system which has fared well in the three TREC QA evaluations so far, and is currently participating in the ARDA AQUAINT program. Our approach centres around the technique of Predictive Annotation, in which an extended set of named entities is recognized prior to indexing, so that the semantic class labels can be indexed along with text and included in the query string. In addition we present other techniques that are employed for specific question types, such as Virtual Annotation for definition questions. We describe the Answer Selection component, which extracts and ranks answer candidates from the passages returned by the search engine based on linguistic as well as statistical features. We present numerous examples as well as quantitative evaluations.


Search Engine Semantic Type Query Term Question Answering Semantic Class 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

10. References

  1. Brill, E., Lin J., Banko M., Dumais S., Ng A. “Data-intensive Question Answering.” In Proceedings of TREC-2001, Gaithersburg, MD, 2001.Google Scholar
  2. Byrd, R. and Ravin, Y. “Identifying and Extracting Relations in Text.” In Proceedings of NLDB 99, Klagenfurt, Austria, 1999.Google Scholar
  3. Clarke, L.A., Cormack, G.V. and Lynam, T.R. “Exploiting Redundancy in Question Answering”, In Proceedings of SIGIR 2001, New Orleans, LA, 2001.Google Scholar
  4. Czuba, K., Prager J., Chu-Carroll, J., “A Machine-Learning Approach to Introspection in a Question Answering System”, In Proceedings of EMNLP-02, Philadelphia, PA, 2002.Google Scholar
  5. Harabagiu, S., Moldovan, D., Pasca, M., Mihalcea, R., Surdeanu, M., Bunescu, R., Girju, R., Rus, V., and Morarescu, P. “The Role of Lexico-Semantic Feedback in Open-Domain Textual Question-Answering”, In Proceedings of ACL 2001, Toulouse, France, 2001.Google Scholar
  6. Hovy, E., Gerber, L., Hermjakob, U. Junk, M. and Lin, C-Y. “Question Answering in Webclopedia. In Proceedings of the 9 th Text Retrieval Conference (TREC9), Gaithersburg, MD, 2001.Google Scholar
  7. Hovy E., Hermjakob U. and Lin C-Y. “The Use of External Knowledge in Factoid QA.” In Proceedings of TREC-2001, Gaithersburg, MD, 2001.Google Scholar
  8. Ittycheriah, A., Franz, M., Zhu, W-J., Ratnaparkhi, A. and Mammone, R.J. “Question Answering Using Maximum Entropy Components”, Proceedings NAACL , Pittsburgh, PA, 2001.Google Scholar
  9. Kupiec, J. “A robust linguistic approach for question answering using an on-line encyclopedia.” In Proceedings of the 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 181-190, Pittsburgh, 1993.Google Scholar
  10. Lehnert, W.G. The Process of Question Answering. Lawrence Erlbaum Associates, Hillsdale, NJ, 1978.Google Scholar
  11. Lenat, D.B. “Cyc: A Large-Scale Investment in Knowledge Infrastructure.” Communications of the ACM 38, no. 11, Nov. 1995.Google Scholar
  12. Miller, G. “WordNet: A Lexical Database for English”, Communications of the ACM 38(11) pp. 39-41, 1995.CrossRefGoogle Scholar
  13. Pasca, M.A. and Harabagiu, S.M. “High Performance Question/Answering”, Proceedings of SIGIR 2001, New Orleans, LA, 2001.Google Scholar
  14. Prager, J.M. “In Question-Answering, Hit-List Size Matters”, in preparation.Google Scholar
  15. Prager, J.M., Chu-Carroll, J. and Czuba, K. “Statistical Answer-Type Identification in Open-Domain Question-Answering”, In Proceedings of HLT 2002, San Diego CA, March 2002.Google Scholar
  16. Prager, J.M., Radev, D.R. and Czuba, K. “Answering What-Is Questions by Virtual Annotation”. In Proceedings of Human Language Technologies Conference, San Diego CA, March 2001.Google Scholar
  17. Prager, J.M., Brown, E.W., Coden, A. and Radev, D. “Question-Answering by Predictive Annotation”. In Proceedings of SIGIR 2000, pp. 184-191, Athens, Greece.Google Scholar
  18. Prager, J.M., Radev, R., Brown, E.W, Coden, A. and Samn, V. “The Use of Predictive Annotation for Question-Answering in TREC8”. In Proceedings of TREC-8, Gaithersburg, MD, 2000.Google Scholar
  19. Radev, D.R., Prager, J.M. and Samn, V. “Ranking Suspected Answers to Natural Language Questions using Predictive Annotation”. In Proceedings of ANLP 2000, pp. 150-157,Google Scholar
  20. Seattle, WA. Salton, Gerald. Automatic Text Processing: the Transformation, Analysis and Retrieval of Information by Computer, Addison-Wesley, Reading, MA, 1989.Google Scholar
  21. Subbotin, M. “Patterns of Potential Answer Expressions as Clues to the Right Answers”, in Proceedings of TREC-2001, Gaithersburg, MD, 2001.Google Scholar
  22. Voorhees, Ellen.“The TREC-8 Question Answering Track Report”, in Proceedings of TREC-8, Gaithersburg, MD, 2000.Google Scholar

Copyright information

© Springer 2008

Authors and Affiliations

  • John Prager
    • 1
  • Jennifer Chu-Carroll
    • 1
  • Eric W. Brown
    • 1
  • Krzysztof Czuba
    • 2
  1. 1.IBM T.J. Watson Research CenterYorktown HeightsUSA
  2. 2.Google Inc.New YorkUSA

Personalised recommendations