Abstract
This paper provides an overview of the current version of SIGA, a system that supports the organization of information retrieval (IR) evaluations. SIGA was recently used in Págico, an evaluation contest where both automatic and human participants competed to find answers to 150 topics in the Portuguese Wikipedia, and we describe its new capabilities in this context as well as provide preliminary results from Págico.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Santos, D., Cabral, L.M.: Summing GikiCLEF up: expectations and lessons learned. In: Peters, C., Nunzio, G.D., Kurimo, M., Mandl, T., Mostefa, D., Peñas, A., Roda, G., Borri, F., Nardi, A., Peters, C. (eds.) Multilingual Information Access Evaluation. Text Retrieval Experiments, vol. I, pp. 212–222. Springer (2009)
Santos, D., Cabral, L.M., Forascu, C., Forner, P., Gey, F., Lamm, K., Mandl, T., Osenova, P., Peñas, A., Rodrigo, Á., Schulz, J., Skalban, Y., Sang, E.T.K.: GikiCLEF: Crosscultural Issues in Multilingual Information Access. In: Calzolari, N., Choukri, K., Maegaard, B., Mariani, J., Odijk, J., Piperidis, S., Rosner, M., Tapias, D. (eds.) Proceedings of the 7th conference on International Language Resources and Evaluation (LREC 2010), Valletta, Malta, European Language Resources Association, ELRA (May 2010)
Dussin, M., Ferro, N.: Direct: applying the DIKW hierarchy to large-scale evaluation campaigns. In: Larsen, R.L., Paepcke, A., Borbinha, J.L., Naaman, M. (eds.) Proceedings of the 8th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 424–424. ACM, New York (2008)
Lalmas, M., Piwowarski, B.: INEX 2006 relevance assessment guide. In: INEX 2006 Workshop Pre-Proceedings, pp. 389–395 (2006)
Santos, D.: Porquê o Págico? Linguamática 4 (2012)
Santos, D.: Caminhos percorridos no mapa da portuguesificação: A Linguateca em perspectiva. Linguamática 1(1), 25–59 (2009)
Gonçalo Oliveira, H., Cardoso, N.: SAHARA: an online service for HAREM Named Entity Recognition Evaluation. In: The 7th Brazilian Symposium in Information and Human Language Technology, STIL 2009 (2009)
Mota, C., Santos, D. (eds.): Desafios na avaliação conjunta do reconhecimento de entidades mencionadas: O Segundo HAREM. Linguateca (2008)
Mota, C., Freitas, C., Costa, L., Rocha, P.: O que é uma resposta? notas de uns avaliadores estafados. Linguamática 4 (2012)
Karlgren, J.: Stylistic Experiments for Information Retrieval. PhD thesis, Stockholm University (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Costa, L., Mota, C., Santos, D. (2012). SIGA, a System to Manage Information Retrieval Evaluations. In: Caseli, H., Villavicencio, A., Teixeira, A., Perdigão, F. (eds) Computational Processing of the Portuguese Language. PROPOR 2012. Lecture Notes in Computer Science(), vol 7243. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28885-2_32
Download citation
DOI: https://doi.org/10.1007/978-3-642-28885-2_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28884-5
Online ISBN: 978-3-642-28885-2
eBook Packages: Computer ScienceComputer Science (R0)