Advertisement

CLEF eHealth 2019 Evaluation Lab

  • Liadh KellyEmail author
  • Lorraine Goeuriot
  • Hanna Suominen
  • Mariana Neves
  • Evangelos Kanoulas
  • Rene Spijker
  • Leif Azzopardi
  • Dan Li
  • Jimmy
  • João Palotti
  • Guido Zuccon
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11438)

Abstract

Since 2012 CLEF eHealth has focused on evaluation resource building efforts around the easing and support of patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring eHealth information in a multilingual setting. This year’s lab offers three tasks: Task 1 on multilingual information extraction; Task 2 on technology assisted reviews in empirical medicine; and Task 3 on consumer health search in mono- and multilingual settings. Herein, we describe the CLEF eHealth evaluation series to-date and then present the 2019 tasks, evaluation methodology, and resources.

Keywords

eHealth Medical informatics Information extraction Information storage and retrieval Information management Systematic reviews 

Notes

Acknowledgements

We gratefully acknowledge the people involved in the CLEF eHealth labs as participants or organizers. We also acknowledge the many organizations that have supported CLEF eHealth labs since 2012. The CLEF eHealth 2019 evaluation lab is supported in part by the CLEF Initiative, Data61/CSIRO, a Google Faculty Research Award, and ARC DECRA grant DE180101579.

References

  1. 1.
    Bert, B., et al.: Rethinking 3R strategies: digging deeper into AnimalTestInfo promotes transparency in in vivo biomedical research. PLoS Biol. 15(12), 1–20 (2017).  https://doi.org/10.1371/journal.pbio.2003217CrossRefGoogle Scholar
  2. 2.
    Goeuriot, L., et al.: ShARe/CLEF eHealth evaluation lab 2013, task 3: information retrieval to address patients’ questions when reading clinical reports. CLEF 2013 Online Working Notes 8138 (2013)Google Scholar
  3. 3.
    Goeuriot, L., et al.: An analysis of evaluation campaigns in ad-hoc medical information retrieval: CLEF eHealth 2013 and 2014. Springer Inf. Retr. J. 21(6), 507–540 (2018)CrossRefGoogle Scholar
  4. 4.
    Goeuriot, L., et al.: ShARe/CLEF eHealth evaluation lab 2014, task 3: user-centred health information retrieval. In: CLEF 2014 Evaluation Labs and Workshop: Online Working Notes, Sheffield, UK (2014)Google Scholar
  5. 5.
    Goeuriot, L., et al.: Overview of the CLEF eHealth evaluation lab 2015. In: Mothe, J., et al. (eds.) CLEF 2015. LNCS, vol. 9283, pp. 429–443. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24027-5_44CrossRefGoogle Scholar
  6. 6.
    Goeuriot, L., et al.: CLEF 2017 eHealth evaluation lab overview. In: Jones, G.J.F., et al. (eds.) CLEF 2017. LNCS, vol. 10456, pp. 291–303. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-65813-1_26CrossRefGoogle Scholar
  7. 7.
    Jimmy, Zuccon, G., Palotti, J., Goeuriot, L., Kelly, L.: Overview of the CLEF 2018 consumer health search task. In: Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings (2018)Google Scholar
  8. 8.
    Kanoulas, E., Li, D., Azzopardi, L., Spijker, R.: CLEF 2017 technologically assisted reviews in empirical medicine overview. In: Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings (2017)Google Scholar
  9. 9.
    Kanoulas, E., Li, D., Azzopardi, L., Spijker, R.: CLEF 2018 technologically assisted reviews in empirical medicine overview. In: Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings (2018)Google Scholar
  10. 10.
    Kelly, L., Goeuriot, L., Suominen, H., Névéol, A., Palotti, J., Zuccon, G.: Overview of the CLEF eHealth evaluation lab 2016. In: Fuhr, N., et al. (eds.) CLEF 2016. LNCS, vol. 9822, pp. 255–266. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-44564-9_24CrossRefGoogle Scholar
  11. 11.
    Kelly, L., et al.: Overview of the ShARe/CLEF eHealth evaluation lab 2014. In: Kanoulas, E., et al. (eds.) CLEF 2014. LNCS, vol. 8685, pp. 172–191. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-11382-1_17CrossRefGoogle Scholar
  12. 12.
    Névéol, A., et al.: Clinical information extraction at the CLEF eHealth evaluation lab 2016. In: Balog, K., Cappellato, L., Ferro, N., Macdonald, C. (eds.) CLEF 2016 Working Notes. CEUR Workshop Proceedings (CEUR-WS.org) (2016). http://ceur-ws.org/Vol-1609/. ISSN 1613-0073
  13. 13.
    Névéol, A., et al.: CLEF eHealth 2017 multilingual information extraction task overview: ICD10 coding of death certificates in English and French. In: CLEF 2017 Online Working Notes. CEUR-WS (2017)Google Scholar
  14. 14.
    Névéol, A., et al.: CLEF eHealth 2018 multilingual information extraction task overview: ICD10 coding of death certificates in French, Hungarian and Italian. In: CLEF 2018 Online Working Notes. CEUR-WS (2018)Google Scholar
  15. 15.
    Palotti, J., et al.: CLEF eHealth evaluation lab 2015, task 2: retrieving information about medical symptoms. In: CLEF 2015 Online Working Notes. CEUR-WS (2015)Google Scholar
  16. 16.
    Palotti, J., et al.: CLEF 2017 task overview: the IR task at the eHealth evaluation lab. In: Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings (2017)Google Scholar
  17. 17.
    Suominen, H.: CLEF 2012 Working Notes. CEUR Workshop Proceedings (CEUR-WS.org). Ed. by Forner, P., Karlgren, J., Womser-Hacker, C., Ferro, N. (2012). http://ceur-ws.org/Vol-1178/. ISSN 1613-0073
  18. 18.
    Suominen, H., Kelly, L., Goeuriot, L.: Scholarly influence of the conference and labs of the evaluation forum ehealth initiative: review and bibliometric study of the 2012 to 2017 outcomes. JMIR Res. Protoc. 7(7), e10961 (2018).  https://doi.org/10.2196/10961CrossRefGoogle Scholar
  19. 19.
    Suominen, H., et al.: Overview of the CLEF ehealth evaluation lab 2018. In: Bellot, P., et al. (eds.) CLEF 2018. LNCS, vol. 11018, pp. 286–301. Springer, Heidelberg (2018).  https://doi.org/10.1007/978-3-319-98932-7_26CrossRefGoogle Scholar
  20. 20.
    Suominen, H., et al.: Overview of the ShARe/CLEF eHealth Evaluation Lab 2013. In: Forner, P., Müller, H., Paredes, R., Rosso, P., Stein, B. (eds.) CLEF 2013. LNCS, vol. 8138, pp. 212–231. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-40802-1_24CrossRefGoogle Scholar
  21. 21.
    Zuccon, G., et al.: The IR task at the CLEF eHealth evaluation lab 2016: user-centred health information retrieval. In: CLEF 2016 Evaluation Labs and Workshop: Online Working Notes, CEUR-WS, September 2016Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Liadh Kelly
    • 1
    Email author
  • Lorraine Goeuriot
    • 2
  • Hanna Suominen
    • 3
    • 4
    • 5
    • 6
  • Mariana Neves
    • 7
  • Evangelos Kanoulas
    • 8
  • Rene Spijker
    • 9
    • 10
  • Leif Azzopardi
    • 11
  • Dan Li
    • 8
  • Jimmy
    • 12
  • João Palotti
    • 13
    • 14
  • Guido Zuccon
    • 12
  1. 1.Maynooth UniversityMaynoothIreland
  2. 2.Univ. Grenoble AlpesGrenobleFrance
  3. 3.Research School of Computer ScienceThe Australian National University (ANU)CanberraAustralia
  4. 4.Data61Commonwealth Scientific and Industrial Research Organisation (CSIRO)CanberraAustralia
  5. 5.Faculty of Science and TechnologyUniversity of CanberraCanberraAustralia
  6. 6.Department of Future TechnologiesUniversity of TurkuTurkuFinland
  7. 7.German Federal Institute for Risk Assessment (BfR)BerlinGermany
  8. 8.Informatics InstituteUniversity of AmsterdamAmsterdamNetherlands
  9. 9.Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, UMC UtrechtUtrecht UniversityUtrechtNetherlands
  10. 10.Medical Library, Amsterdam Public Health, Amsterdam UMCUniversity of AmsterdamAmsterdamNetherlands
  11. 11.Computer and Information SciencesUniversity of StrathclydeGlasgowUK
  12. 12.The University of QueenslandSt LuciaAustralia
  13. 13.Vienna University of TechnologyViennaAustria
  14. 14.Qatar Computing Research InstituteDohaQatar

Personalised recommendations