Overview of QAST 2008
This paper describes the experience of QAST 2008, the second time a pilot track of CLEF has been held aiming to evaluate the task of Question Answering in Speech Transcripts. Five sites submitted results for at least one of the five scenarios (lectures in English, meetings in English, broadcast news in French and European Parliament debates in English and Spanish). In order to assess the impact of potential errors of automatic speech recognition, for each task contrastive conditions are with manual and automatically produced transcripts. The QAST 2008 evaluation framework is described, along with descriptions of the five scenarios and their associated data, the system submissions for this pilot track and the official evaluation results.
KeywordsQuestion answering Spontaneous speech transcripts
Unable to display preview. Download preview PDF.
- 1.Galliano, S., Geoffrois, E., Gravier, G., Bonastre, J.F., Mostefa, D., Choukri, K.: Corpus description of the ESTER Evaluation Campaign for the Rich Transcription of French Broadcast News. In: Proceedings of LREC 2006, Genoa, pp. 315–320 (2006)Google Scholar
- 2.Gravier, G., Bonastre, J.F., Galliano, S., Geoffrois, E., McTait, K., Choukri, K.: The ESTER evaluation campaign of Rich Transcription of French Broadcast News. In: Proceedings of LREC 2004, Lisbon, pp. 885–888 (2004)Google Scholar
- 3.Peters, C., Clough, P., Gey, F.C., Karlgren, J., Magnini, B., Oard, D.W., de Rijke, M., Stempfhuber, M. (eds.): CLEF 2006. LNCS, vol. 4730. Springer, Heidelberg (2007)Google Scholar
- 4.TC-Star (2004-2008), http://www.tc-star.org
- 6.Voorhees, E.M., Buckland, L.L. (eds.): The Fifteenth Text Retrieval Conference Proceedings, TREC 2006 (2006)Google Scholar