A Test Collection for Passage Retrieval Evaluation of Spanish Health-Related Resources

  • Eleni KamateriEmail author
  • Theodora Tsikrika
  • Spyridon Symeonidis
  • Stefanos Vrochidis
  • Wolfgang Minker
  • Yiannis Kompatsiaris
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11438)


This paper describes a new test collection for passage retrieval from health-related Web resources in Spanish. The test collection contains 10,037 health-related documents in Spanish, 37 topics representing complex information needs formulated in a total of 167 natural language questions, and manual relevance assessments of text passages, pooled from multiple systems. This test collection is the first to combine search in a language beyond English, passage retrieval, and health-related resources and topics targeting the general public.


Test collection Passage retrieval Inter-rater agreement 



This is supported by EU H2020 KRISTINA project (645012) and GR Research-Create-Innovate REA project (T1EDK-00686).


  1. 1.
    Allan, J.: HARD track overview in TREC 2004 - high accuracy retrieval from documents. In: Proceedings of TREC 2004 (2004)Google Scholar
  2. 2.
    Dietz, L., Verma, M., Radlinski, F., Craswell, N.: TREC complex answer retrieval overview. In: Proceedings of TREC 2017 (2017)Google Scholar
  3. 3.
    Habernal, I., et al.: New collection announcement: focused retrieval over the web. In: Proceedings of ACM SIGIR 2016, pp. 701–704 (2016)Google Scholar
  4. 4.
    Hersh, W., Cohen, A., Roberts, P., Rekapalli, H.: TREC 2006 genomics track overview. In: Proceedings of TREC 2006 (2006)Google Scholar
  5. 5.
    Hirschberg, D.S.: The longest common subsequence problem. Ph.D. thesis, Princeton, NJ, USA (1975). aAI7623803Google Scholar
  6. 6.
    Kamps, J., Geva, S., Trotman, A., Woodley, A., Koolen, M.: Overview of the INEX 2008 Ad Hoc track. In: Geva, S., Kamps, J., Trotman, A. (eds.) INEX 2008. LNCS, vol. 5631, pp. 1–28. Springer, Heidelberg (2009). Scholar
  7. 7.
    Keikha, M., Park, J., Croft, W., Sanderson, M.: Retrieving passages and finding answers. In: Proceedings of ADCS 2014, pp. 81–84. ACM (2014)Google Scholar
  8. 8.
    Kohlschütter, C., Fankhauser, P., Nejdl, W.: Boilerplate detection using shallow text features. In: Proceedings of ACM WSDM 2010, pp. 441–450 (2010)Google Scholar
  9. 9.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst. 26, 3111–3119 (2013)Google Scholar
  10. 10.
    Soboroff, I., Griffitt, K., Strassel, S.: The BOLT IR test collections of multilingual passage retrieval from discussion forums. In: Proceedings of ACM SIGIR 2016, pp. 713–716. ACM (2016)Google Scholar
  11. 11.
    Suominen, H., et al.: Overview of the CLEF eHealth evaluation lab 2018. In: Bellot, P., et al. (eds.) International Conference of the Cross-Language Evaluation Forum for European Languages, vol. 11018, pp. 286–301. Springer, Cham (2018). Scholar
  12. 12.
    Tsatsaronis, G., Balikas, G., Malakasiotis, P., Partalas, I., et al.: An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. BMC Bioinform. 16(1), 138 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Eleni Kamateri
    • 1
    • 2
    Email author
  • Theodora Tsikrika
    • 2
  • Spyridon Symeonidis
    • 2
  • Stefanos Vrochidis
    • 2
  • Wolfgang Minker
    • 1
  • Yiannis Kompatsiaris
    • 2
  1. 1.Centre for Research and Technology-HellasThessalonikiGreece
  2. 2.Ulm UniversityUlmGermany

Personalised recommendations