Abstract
This chapter assesses the scholarly impact of the CLEF evaluation campaign by performing a bibliometric analysis of the citations of the CLEF 2010–2017 papers collected through Google Scholar. The analysis extends an earlier 2013 study by Tsikrika et al. of the CLEF Proceedings for the period 2000–2009 and compares the impact of the first half of CLEF to the second. It also extends the analysis by including the CLEF Working notes, a less formal but important part of the CLEF oeuvre. Results show that, despite the different nature of the peer-reviewed CLEF Proceedings papers and the less formal and much more numerous Working note papers, both types of publications have high citation impact. In particular, overview papers from the various labs and tasks in CLEF attract large amounts of citations in both Proceedings and Working Notes. A significant proportion of the total number of citations appear to be from outside CLEF—there are simply not enough CLEF papers every year to explain that many citations. In conclusion, the analysis of the productivity and citation impact of CLEF in the period 2010–2017 shows that CLEF is a very strong and vibrant initiative that has managed a major change of format between 2009/2010 and continues to produce relevant research, datasets and tools.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Tsikrika T, Larsen B, Müller H, Endrullis S, Rahm E (2013) The scholarly impact of CLEF (2000–2009). In: Forner P, Müller H, Paredes R, Rosso P, Stein B (eds) Information access evaluation meets multilinguality, multimodality, and visualization. Proceedings of the fourth international conference of the CLEF initiative (CLEF 2013). Lecture notes in computer science (LNCS), vol 8138. Springer, Heidelberg, pp 1–12
Acknowledgements
We wish to thank Lucas Chaves, University of Copenhagen, Denmark for assistance in matching DBLP and Google Scholar data, Anne-Wil Harzing for help with the ‘Publish or Perish’ software platform, as well as two anonymous reviewers for constructive feedback.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Larsen, B. (2019). The Scholarly Impact of CLEF 2010–2017. In: Ferro, N., Peters, C. (eds) Information Retrieval Evaluation in a Changing World. The Information Retrieval Series, vol 41. Springer, Cham. https://doi.org/10.1007/978-3-030-22948-1_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-22948-1_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22947-4
Online ISBN: 978-3-030-22948-1
eBook Packages: Computer ScienceComputer Science (R0)