Advertisement

Creating the DISEQuA Corpus: A Test Set for Multilingual Question Answering

  • Bernardo Magnini
  • Simone Romagnoli
  • Alessandro Vallin
  • Jesús Herrera
  • Anselmo Peñas
  • Víctor Peinado
  • Felisa Verdejo
  • Maarten de Rijke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3237)

Abstract

This paper describes the procedure adopted by the three coordinators of the CLEF 2003 question answering track (ITC-irst, UNED and ILLC) to create the question set for the monolingual tasks. Despite the few resources available, the three groups managed to formulate and verify a large pool of original questions in three different languages: Dutch, Italian and Spanish. Part of these queries was translated into English and shared between the three coordinating groups. A second cross-verification was then conducted in order to identify the queries that had an answer in all three monolingual document collections. The result of the joint efforts was the creation of the DISEQuA (Dutch Italian Spanish English Questions and Answers) corpus, a useful and reusable resource that is freely available for the research community. We report on the different stages of the corpus creation, from the monolingual kernels to the multilingual extension.

Keywords

Machine Translation Document Collection Question Answering Source Language Verification Phase 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Kluck, M., Womser-Hacker, C.: Inside the Evaluation Process of the Cross Language Evaluation Forum (CLEF): Issues of Multilingual Topic Creation and Multilingual Relevance Assessment. In: Proceedings of the Third International Conference on Language Resources and Evaluation (LREC 2002). Las Palmas de Gran Canaria, May 29–31 (2002)Google Scholar
  2. 2.
    Magnini, B.: Evaluation of Cross-Language Question Answering Systems. In: Proposal presentation made at the CLEF Workshop (2002), URL: http://clef.iei.pi.cnr.it:2002/workshop2002/presentations/q-a.pdf
  3. 3.
    Tice, D.M., Voorhees, E.M.: The TREC-8 Question Answering Track Evaluation. In: Proceedings of the Eighth Text REtrieval Conference (TREC-8), Gaithersburg, MD (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Bernardo Magnini
    • 1
  • Simone Romagnoli
    • 1
  • Alessandro Vallin
    • 1
  • Jesús Herrera
    • 2
  • Anselmo Peñas
    • 2
  • Víctor Peinado
    • 2
  • Felisa Verdejo
    • 2
  • Maarten de Rijke
    • 3
  1. 1.ITC-irstCentro per la Ricerca Scientifica e TecnologicaPovo (TN)Italy
  2. 2.Dpto. Lenguajes y Sistemas InformaticosUNED, Spanish Distance Learning UniversityMadridSpain
  3. 3.Language and Inference Technology Group, ILLCUniversity of AmsterdamAmsterdamThe Netherlands

Personalised recommendations