Abstract
The Answer Validation Exercise at the Cross Language Evaluation Forum (CLEF) is aimed at developing systems able to decide whether the answer of a Question Answering (QA) system is correct or not. We present here the exercise description, the changes in the evaluation with respect to the last edition and the results of this third edition (AVE 2008). Last year’s changes allowed us to measure the possible gain in performance obtained by using AV systems as the selection method of QA systems. Then, in this edition we wanted to reward AV systems able to detect also if all the candidate answers to a question are incorrect. 9 groups have participated with 24 runs in 5 different languages, and compared with the QA systems, the results show an evidence of the potential gain that more sophisticated AV modules might introduce in the task of QA.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Dietterich, T.G.: Machine learning research: Four current directions. AI Magazine 18(4), 97–136 (1997)
Forner, P., Peñas, A., Alegria, I., Forascu, C., Moreau, N., Osenova, P., Prokopidis, P., Rocha, P., Sacaleanu, B., Sutcliffe, R., Sang, E.T.K.: Overview of the CLEF 2008 Multilingual Question Answering Track. In: Working Notes for the CLEF 2008 Workshop, Aarhus, Denmark, September 17-19 (2008)
Giampiccolo, D., Forner, P., Herrera, J., Peñas, A., Ayache, C., Forascu, C., Jijkoun, V., Osenova, P., Rocha, P., Sacaleanu, B., Sutcliffe, R.F.E.: Overview of the CLEF 2007 Multilingual Question Answering Track. In: Peters, et al [16], pp. 200–236
Glöckner, I.: RAVE: A Fast Logic-Based Answer Validator. In: Peters, C., et al. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 468–471. Springer, Heidelberg (2009)
Harabagiu, S., Hickl, A.: Methods for Using Textual Entailment in Open-Domain Question Answering. In: Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, Sydney, pp. 905–912 (2006)
Hartrumpf, S., Glöckner, I., Leveling, J.: University of Hagen at QA@CLEF 2008: Efficient Question Answering with Question Decomposition and Multiple Answer Streams. In: Working Notes for the CLEF 2008 Workshop, Aarhus, Denmark, September 17-19 (2008)
Iftene, A., Balahur-Dobrescu, A.: Answer Validation on English and Romanian Languages. In: Peters, C., et al. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 448–451. Springer, Heidelberg (2009)
Jacquin, C., Monceaux, L., Desmontils, E.: The Answer Validation System ProdicosAV dedicated to French. In: Peters, C., et al. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 452–459. Springer, Heidelberg (2009)
Magnini, B., Giampiccolo, D., Forner, P., Ayache, C., Jijkoun, V., Osenova, P., Peñas, A., Rocha, P., Sacaleanu, B., Sutcliffe, R.F.E.: Overview of the CLEF 2006 Multilingual Question Answering Track. In: Peters, et al [15], pp. 223–256
Moriceau, V., Tannier, X., Grappy, A., Grau, B.: Justification of Answers by Verification of Dependency Relations - The French AVE Task. In: Working Notes for the CLEF 2008 Workshop, Aarhus, Denmark, September 17-19 (2008)
Ferrández, Ó., Muñoz, R., Palomar, M.: Studying the Influence of Semantic Constraints in AVE. In: Peters, C., et al. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 460–467. Springer, Heidelberg (2009)
Peñas, A., Rodrigo, Á., Sama, V., Verdejo, F.: Overview of the Answer Validation Exercise 2006. In: Peters, et al [15], pp. 257–264
Peñas, A., Rodrigo, Á., Sama, V., Verdejo, F.: Testing the Reasoning for Question Answering Validation. Journal of Logic and Computation 18(3), 459–474 (2008)
Peñas, A., Rodrigo, Á., Verdejo, F.: Overview of the Answer Validation Exercise 2007. In: Peters, et al [16], pp. 237–248
Peters, C., Clough, P., Gey, F.C., Karlgren, J., Magnini, B., Oard, D.W., de Rijke, M., Stempfhuber, M. (eds.): CLEF 2006. LNCS, vol. 4730. Springer, Heidelberg (2007)
Peters, C., Jijkoun, V., Mandl, T., Müller, H., Oard, D.W., Peñas, A., Petras, V., Santos, D. (eds.): CLEF 2007. LNCS, vol. 5152. Springer, Heidelberg (2008)
Rodrigo, Á., Peñas, A., Herrera, J., Verdejo, F.: The Effect of Entity Recognition on Answer Validation. In: Peters, et al [15], pp. 483–489
Téllez-Valero, A., Juárez-González, A., Gómez, M.M., Villaseñor-Pineda, L.: Using Non-Overlap Features for Supervised Answer Validation. In: Peters, C., et al. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 476–479. Springer, Heidelberg (2009)
Wang, R., Neumann, G.: Information Synthesis for Answer Validation. In: Peters, C., et al. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 472–475. Springer, Heidelberg (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rodrigo, Á., Peñas, A., Verdejo, F. (2009). Overview of the Answer Validation Exercise 2008. In: Peters, C., et al. Evaluating Systems for Multilingual and Multimodal Information Access. CLEF 2008. Lecture Notes in Computer Science, vol 5706. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04447-2_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-04447-2_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04446-5
Online ISBN: 978-3-642-04447-2
eBook Packages: Computer ScienceComputer Science (R0)