Creating the DISEQuA Corpus: A Test Set for Multilingual Question Answering
This paper describes the procedure adopted by the three coordinators of the CLEF 2003 question answering track (ITC-irst, UNED and ILLC) to create the question set for the monolingual tasks. Despite the few resources available, the three groups managed to formulate and verify a large pool of original questions in three different languages: Dutch, Italian and Spanish. Part of these queries was translated into English and shared between the three coordinating groups. A second cross-verification was then conducted in order to identify the queries that had an answer in all three monolingual document collections. The result of the joint efforts was the creation of the DISEQuA (Dutch Italian Spanish English Questions and Answers) corpus, a useful and reusable resource that is freely available for the research community. We report on the different stages of the corpus creation, from the monolingual kernels to the multilingual extension.
KeywordsMachine Translation Document Collection Question Answering Source Language Verification Phase
Unable to display preview. Download preview PDF.
- 1.Kluck, M., Womser-Hacker, C.: Inside the Evaluation Process of the Cross Language Evaluation Forum (CLEF): Issues of Multilingual Topic Creation and Multilingual Relevance Assessment. In: Proceedings of the Third International Conference on Language Resources and Evaluation (LREC 2002). Las Palmas de Gran Canaria, May 29–31 (2002)Google Scholar
- 2.Magnini, B.: Evaluation of Cross-Language Question Answering Systems. In: Proposal presentation made at the CLEF Workshop (2002), URL: http://clef.iei.pi.cnr.it:2002/workshop2002/presentations/q-a.pdf
- 3.Tice, D.M., Voorhees, E.M.: The TREC-8 Question Answering Track Evaluation. In: Proceedings of the Eighth Text REtrieval Conference (TREC-8), Gaithersburg, MD (2000)Google Scholar