Advertisement

From XML Retrieval to Semantic Search and Beyond

The INEX, SBS, and MC2 Labs of CLEF 2012–2018
  • Jaap KampsEmail author
  • Marijn Koolen
  • Shlomo Geva
  • Ralf Schenkel
  • Eric SanJuan
  • Toine Bogers
Chapter
Part of the The Information Retrieval Series book series (INRE, volume 41)

Abstract

INEX ran as an independent evaluation forum for 10 years before it teamed up with CLEF in 2012. Even before 2012 there was considerable collaboration between INEX and CLEF, and these collaborations increased in intensity when CLEF moved beyond its traditional cross-lingual focus in 2009/2010 shifting to include all experimental IR. This led to the merger of CLEF and INEX, and effectively to the inclusion of INEX as a large track or lab into CLEF in 2012. This chapter details the efforts of the INEX lab in CLEF (2012–2014), as well as the ongoing activities as separate labs, under the labels Social Book Search (2015–2016), and Microblog Contextualization (2016–2018).

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

Thanks to the CLEF Association, CLEF Conference, and CLEF Labs organizers for the wonderful support over the years, that greatly facilitated all the work reported in this chapter. Special thanks to the editors of this volume for support, encouragement and great flexibility.

References

  1. Alonso O, Hearst MA, Kamps J (2015a) Report on the first SIGIR workshop on graph search and beyond (GSB’15). SIGIR Forum 49(2):89–97. http://doi.acm.org/10.1145/2888422.2888436 CrossRefGoogle Scholar
  2. Alonso O, Kamps J, Karlgren J (2015b) Report on the seventh workshop on exploiting semantic annotations in information retrieval (ESAIR’14). SIGIR Forum 49(1):27–34. http://doi.acm.org/10.1145/2795403.2795412 CrossRefGoogle Scholar
  3. Balog K, Dalton J, Doucet A, Ibrahim Y (2016) Report on the eighth workshop on exploiting semantic annotations in information retrieval (ESAIR ’15). SIGIR Forum 50(1):49–57. http://doi.acm.org/10.1145/2964797.2964806 CrossRefGoogle Scholar
  4. Beckers T, Bellot P, Demartini G, Denoyer L, Vries CMD, Doucet A, Fachry KN, Fuhr N, Gallinari P, Geva S, Huang WC, Iofciu T, Kamps J, Kazai G, Koolen M, Kutty S, Landoni M, Lehtonen M, Moriceau V, Nayak R, Nordlie R, Pharo N, SanJuan E, Schenkel R, Tannier X, Theobald M, Thom JA, Trotman A, de Vries AP (2010) Report on INEX 2009. SIGIR Forum 44(1):38–57. http://doi.acm.org/10.1145/1842890.1842897 CrossRefGoogle Scholar
  5. Belkin NJ, Clarke CLA, Gao N, Kamps J, Karlgren J (2011) Report on the SIGIR workshop on “entertain me”: supporting complex search tasks. SIGIR Forum 45(2):51–59. http://doi.acm.org/10.1145/2093346.2093354 CrossRefGoogle Scholar
  6. Bellot P, Chappell T, Doucet A, Geva S, Gurajada S, Kamps J, Kazai G, Koolen M, Landoni M, Marx M, Mishra A, Moriceau V, Mothe J, Preminger M, Ramírez G, Sanderson M, Sanjuan E, Scholer F, Schuh A, Tannier X, Theobald M, Trappett M, Trotman A, Wang Q (2012a) Report on INEX 2012. SIGIR Forum 46(2):50–59. http://doi.acm.org/10.1145/2422256.2422264 CrossRefGoogle Scholar
  7. Bellot P, Chappell T, Doucet A, Geva S, Kamps J, Kazai G, Koolen M, Landoni M, Marx M, Moriceau V, Mothe J, Ramírez G, Sanderson M, SanJuan E, Scholer F, Tannier X, Theobald M, Trappett M, Trotman A, Wang Q (2012b) Report on INEX 2011. SIGIR Forum 46(1):33–42. http://doi.acm.org/10.1145/2215676.2215679 CrossRefGoogle Scholar
  8. Bellot P, Doucet A, Geva S, Gurajada S, Kamps J, Kazai G, Koolen M, Mishra A, Moriceau V, Mothe J, Preminger M, SanJuan E, Schenkel R, Tannier X, Theobald M, Trappett M, Wang Q (2013a) Overview of INEX 2013. In: Forner P, Müller H, Paredes R, Rosso P, Stein B (eds) Information access evaluation meets multilinguality, multimodality, and visualization. Proceedings of the fourth international conference of the CLEF initiative (CLEF 2013). Lecture Notes in Computer Science (LNCS) 8138. Springer, Heidelberg, pp 269–281Google Scholar
  9. Bellot P, Moriceau V, Mothe J, SanJuan E, Tannier X (2013b) Overview of INEX tweet contextualization 2013 track. In: Forner P, Navigli R, Tufis D, Ferro N (eds) Working notes for CLEF 2013 conference, Valencia, Spain, September 23–26, 2013. CEUR-WS.org. CEUR workshop proceedings, vol 1179. http://ceur-ws.org/Vol-1179/CLEF2013wn-INEX-BellotEt2013.pdf
  10. Bellot P, Bogers T, Geva S, Hall MA, Huurdeman HC, Kamps J, Kazai G, Koolen M, Moriceau V, Mothe J, Preminger M, SanJuan E, Schenkel R, Skov M, Tannier X, Walsh D (2014a) Overview of INEX 2014. In: Kanoulas E, Lupu M, Clough P, Sanderson M, Hall M, Hanbury A, Toms E (eds) Information access evaluation – multilinguality, multimodality, and interaction. Proceedings of the fifth international conference of the CLEF initiative (CLEF 2014). Lecture Notes in Computer Science (LNCS) 8685. Springer, Heidelberg, pp 212–228Google Scholar
  11. Bellot P, Moriceau V, Mothe J, SanJuan E, Tannier X (2014b) Overview of INEX tweet contextualization 2014 track. In: Cappellato L, Ferro N, Halvey M, Kraaij W (eds) Working notes for CLEF 2014 conference, Sheffield, UK, September 15–18, 2014. CEUR-WS.org. CEUR Workshop proceedings, vol 1180, pp 494–500. http://ceur-ws.org/Vol-1180/CLEF2014wn-Inex-BellotEt2014.pdf
  12. Bennett PN, Gabrilovich E, Kamps J, Karlgren J (2014) Report on the sixth workshop on exploiting semantic annotations in information retrieval (ESAIR’13). SIGIR Forum 48(1):13–20. http://doi.acm.org/10.1145/2641383.2641387 CrossRefGoogle Scholar
  13. Bogers T, Koolen M (2015) Second workshop on new trends in content-based recommender systems (cbrecsys 2015). In: Werthner H, Zanker M, Golbeck J, Semeraro G (eds) Proceedings of the 9th ACM conference on recommender systems, RecSys 2015, Vienna, Austria, September 16–20, 2015. ACM, New York, pp 339–340. http://doi.acm.org/10.1145/2792838.2798718 Google Scholar
  14. Bogers T, Koolen M, Cantador I (2014) Workshop on new trends in content-based recommender systems: (cbrecsys 2014). In: Kobsa A, Zhou MX, Ester M, Koren Y (eds) Eighth ACM conference on recommender systems, RecSys ’14, Foster City, Silicon Valley, CA, USA - October 06–10, 2014. ACM, New York, pp 379–380. http://doi.acm.org/10.1145/2645710.2645784 Google Scholar
  15. Bogers T, Hendrickx I, Koolen M, Verberne S (2016a) Overview of the SBS 2016 mining track. In: Balog K, Cappellato L, Ferro N, Macdonald C (eds) Working notes of CLEF 2016 - conference and labs of the evaluation forum, Évora, Portugal, 5–8 September, 2016. CEUR-WS.org, CEUR workshop proceedings, vol 1609, pp 1053–1063. http://ceur-ws.org/Vol-1609/16091053.pdf
  16. Bogers T, Koolen M, Musto C, Lops P, Semeraro G (2016b) Third workshop on new trends in content-based recommender systems (cbrecsys 2016). In: Sen S, Geyer W, Freyne J, Castells P (eds) Proceedings of the 10th ACM conference on recommender systems, Boston, MA, USA, September 15–19, 2016. ACM, New York, pp 419–420. http://doi.acm.org/10.1145/2959100.2959200 Google Scholar
  17. Bogers T, Gäde M, Hall MM, Freund L, Koolen M, Petras V, Skov M (2018) Report on the workshop on barriers to interactive IR resources re-use (BIIRRR 2018). SIGIR Forum 52(1):119–128. https://doi.org/10.1145/3274784.3274795 CrossRefGoogle Scholar
  18. Bogers T, Dodson S, Freund L, Gäde M, Hall MM, Koolen M, Petras V, Pharo N, Skov M (2019) Workshop on barriers to interactive IR resources re-use (BIIRRR 2019). In: Azzopardi L, Halvey M, Ruthven I, Joho H, Murdock V, Qvarfordt P (eds) Proceedings of the 2019 conference on human information interaction and retrieval, CHIIR 2019, Glasgow, Scotland, UK, March 10–14, 2019. ACM, New York, pp 389–392. https://doi.org/10.1145/3295750.3298965 Google Scholar
  19. Chappell T, Geva S (2012) Overview of the INEX 2012 relevance feedback track. In: Forner P, Karlgren J, Womser-Hacker C, Ferro N (eds) CLEF 2012 working notes. CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1178/
  20. Cossu J, Gonzalo J, Hajjem M, Hamon O, Latiri C, SanJuan E (2018) CLEF MC2 2018 lab technical overview of cross language microblog search and argumentative mining. In: Cappellato L, Ferro N, Nie J, Soulier L (eds) Working notes of CLEF 2018 - conference and labs of the evaluation forum, Avignon, France, September 10–14, 2018. CEUR-WS.org. CEUR workshop proceedings, vol 2125, http://ceur-ws.org/Vol-2125/invited_paper_7.pdf
  21. Dean-Hall A, Clarke CLA, Kamps J, Thomas P, Voorhees EM (2012) Overview of the TREC 2012 contextual suggestion track. In: Voorhees EM, Buckland LP (eds) Proceedings of the twenty-first text retrieval conference, TREC 2012, Gaithersburg, Maryland, USA, November 6–9, 2012. National Institute of Standards and Technology (NIST), Gaithersburg, vol Special Publication 500-298, http://trec.nist.gov/pubs/trec21/papers/CONTEXTUAL12.overview.pdf
  22. Dean-Hall A, Clarke CLA, Simone N, Kamps J, Thomas P, Voorhees EM (2013) Overview of the TREC 2013 contextual suggestion track. In: Voorhees EM (ed) Proceedings of the twenty-second text retrieval conference, TREC 2013, Gaithersburg, Maryland, USA, November 19–22, 2013. National Institute of Standards and Technology (NIST), Gaithersburg, vol Special Publication 500-302 http://trec.nist.gov/pubs/trec22/papers/CONTEXT.OVERVIEW.pdf
  23. Dean-Hall A, Clarke CLA, Kamps J, Thomas P, Voorhees EM (2014) Overview of the TREC 2014 contextual suggestion track. In: Voorhees EM, Ellis A (eds) Proceedings of the twenty-third text retrieval conference, TREC 2014, Gaithersburg, Maryland, USA, November 19–21, 2014. National Institute of Standards and Technology (NIST), Gaithersburg, vol Special Publication 500-308, http://trec.nist.gov/pubs/trec23/papers/overview-context.pdf
  24. Dean-Hall A, Clarke CLA, Kamps J, Kiseleva J, Voorhees EM (2015) Overview of the TREC 2015 contextual suggestion track. In: Voorhees EM, Ellis A (eds) Proceedings of the twenty-fourth text retrieval conference, TREC 2015, Gaithersburg, Maryland, USA, November 17–20, 2015. National Institute of Standards and Technology (NIST), Gaithersburg, vol Special Publication 500-319 http://trec.nist.gov/pubs/trec24/papers/Overview-CX.pdf
  25. Doucet A, Kazai G, Colutto S, Mühlberger G (2013) ICDAR 2013 competition on book structure extraction. In: 12th international conference on document analysis and recognition, ICDAR 2013, Washington, DC, USA, August 25–28, 2013, IEEE Computer Society, Washington, pp 1438–1443.  https://doi.org/10.1109/ICDAR.2013.290
  26. Ermakova L, Goeuriot L, Mothe J, Mulhem P, Nie J, SanJuan E (2016) Cultural micro-blog contextualization 2016 workshop overview: data and pilot tasks. In: Balog K, Cappellato L, Ferro N, Macdonald C (eds) Working notes of CLEF 2016 - conference and labs of the evaluation forum, Évora, Portugal, 5–8 September, 2016. CEUR-WS.org. CEUR workshop proceedings, vol 1609, pp 1197–1200. http://ceur-ws.org/Vol-1609/16091197.pdf
  27. Ermakova L, Goeuriot L, Mothe J, Mulhem P, Nie J, SanJuan E (2017a) CLEF 2017 microblog cultural contextualization lab overview. In: Jones GJF, Lawless S, Gonzalo J, Kelly L, Goeuriot L, Mandl T, Cappellato L, Ferro N (eds) Experimental IR meets multilinguality, multimodality, and interaction - 8th international conference of the CLEF association, CLEF 2017, Dublin, Ireland, September 11–14, 2017, Proceedings. Lecture Notes in Computer Science, vol 10456, pp 304–314. Springer, Basel. https://doi.org/10.1007/978-3-319-65813-1_27 Google Scholar
  28. Ermakova L, Mothe J, SanJuan E (2017b) CLEF 2017 microblog cultural contextualization content analysis task overview. In: Cappellato L, Ferro N, Goeuriot L, Mandl T (eds) Working notes of CLEF 2017 - conference and labs of the evaluation forum, Dublin, Ireland, September 11–14, 2017. CEUR-WS.org, CEUR workshop proceedings, vol 1866. http://ceur-ws.org/Vol-1866/invited_paper_14.pdf
  29. Gäde M, Hall MM, Huurdeman HC, Kamps J, Koolen M, Skov M, Toms E, Walsh D (2015a) Overview of the SBS 2015 interactive track. In: Cappellato L, Ferro N, Jones GJF, SanJuan E (eds) Working notes of CLEF 2015 - conference and labs of the evaluation forum, Toulouse, France, September 8–11, 2015. CEUR-WS.org, CEUR workshop proceedings, vol 1391. http://ceur-ws.org/Vol-1391/78-CR.pdf
  30. Gäde M, Hall MM, Huurdeman HC, Kamps J, Koolen M, Skov M, Toms E, Walsh D (2015b) Report on the first workshop on supporting complex search tasks. SIGIR Forum 49(1):50–56. http://doi.acm.org/10.1145/2795403.2795415 CrossRefGoogle Scholar
  31. Gäde M, Hall MM, Huurdeman HC, Kamps J, Koolen M, Skov M, Bogers T, Walsh D (2016) Overview of the SBS 2016 interactive track. In: Balog K, Cappellato L, Ferro N, Macdonald C (eds) Working notes of CLEF 2016 - conference and labs of the evaluation forum, Évora, Portugal, 5–8 September, 2016. CEUR-WS.org. CEUR workshop proceedings, vol 1609, pp 1024–1038. http://ceur-ws.org/Vol-1609/16091024.pdf
  32. Goeuriot L, Mothe J, Mulhem P, Murtagh F, SanJuan E (2016) Overview of the CLEF 2016 cultural micro-blog contextualization workshop. In: Fuhr N, Quaresma P, Gonçalves T, Larsen B, Balog K, Macdonald C, Cappellato L, Ferro N (eds) Experimental IR meets multilinguality, multimodality, and interaction. Proceedings of the seventh international conference of the CLEF association (CLEF 2016), Lecture notes in computer science (LNCS) 9822. Springer, Heidelberg, pp 371–378CrossRefGoogle Scholar
  33. Goeuriot L, Mulhem P, SanJuan E (2017) CLEF 2017 MC2 search and time line tasks overview. In: Cappellato L, Ferro N, Goeuriot L, Mandl T (eds) Working notes of CLEF 2017 - conference and labs of the evaluation forum, Dublin, Ireland, September 11–14, 2017. CEUR-WS.org. CEUR workshop proceedings, vol 1866. http://ceur-ws.org/Vol-1866/invited_paper_15.pdf
  34. Gurajada S, Kamps J, Mishra A, Schenkel R, Theobald M, Wang Q (2013) Overview of the INEX 2013 linked data track. In: Forner P, Navigli R, Tufis D, Ferro N (eds) Working notes for CLEF 2013 conference, Valencia, Spain, September 23–26, 2013. CEUR-WS.org, CEUR workshop proceedings, vol 1179. http://ceur-ws.org/Vol-1179/CLEF2013wn-INEX-GurajadaEt2013.pdf
  35. Hajjem M, Cossu J, Latiri C, SanJuan E (2018) CLEF MC2 2018 lab overview. In: Bellot P, Trabelsi C, Mothe J, Murtagh F, Nie J, Soulier L, SanJuan E, Cappellato L, Ferro N (eds) Experimental IR meets multilinguality, multimodality, and interaction - 9th international conference of the CLEF association, CLEF 2018, Avignon, France, September 10–14, 2018, Proceedings. Lecture Notes in Computer Science, vol 11018, pp 302–308. Springer. https://doi.org/10.1007/978-3-319-98932-7_27 CrossRefGoogle Scholar
  36. Hall MM, Huurdeman HC, Koolen M, Skov M, Walsh D (2014) Overview of the INEX 2014 interactive social book search track. In: Cappellato L, Ferro N, Halvey M, Kraaij W (eds) Working notes for CLEF 2014 conference, Sheffield, UK, September 15–18, 2014. CEUR-WS.org, CEUR workshop proceedings, vol 1180, pp 480–493. http://ceur-ws.org/Vol-1180/CLEF2014wn-Inex-HallEt2014.pdf
  37. Hashemi SH, Clarke CLA, Kamps J, Kiseleva J, Voorhees EM (2016) Overview of the TREC 2016 contextual suggestion track. In: Voorhees EM, Ellis A (eds) Proceedings of the twenty-fifth text retrieval conference, TREC 2016, Gaithersburg, Maryland, USA, November 15–18, 2016, National Institute of Standards and Technology (NIST), Gaithersburg, vol Special Publication 500-321. http://trec.nist.gov/pubs/trec25/papers/Uamsterdam-CX.pdf
  38. Kamps J, Geva S, Peters C, Sakai T, Trotman A, Voorhees EM (2009) Report on the SIGIR 2009 workshop on the future of IR evaluation. SIGIR Forum 43(2):13–23. http://doi.acm.org/10.1145/1670564.1670567 CrossRefGoogle Scholar
  39. Kamps J, Karlgren J, Mika P, Murdock V (2013) Report on the fifth workshop on exploiting semantic annotations in information retrieval (ESAIR’12). SIGIR Forum 47(1):38–45. http://doi.acm.org/10.1145/2492189.2492196 CrossRefGoogle Scholar
  40. Kazai G (2009, 2018) INEX. In: Liu L, Özsu MT (eds) Encyclopedia of database systems. Springer, New York, p 1472. https://doi.org/10.1007/978-0-387-39940-9_2846 https://doi.org/10.1007/978-0-387-39940-9_2846
  41. Koolen M, Kazai G, Kamps J, Preminger M, Doucet A, Landoni M (2012) Overview of the INEX 2012 social book search track. In: Forner P, Karlgren J, Womser-Hacker C, Ferro N (eds) CLEF 2012 working notes. CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1178/
  42. Koolen M, Kazai G, Preminger M, Doucet A (2013) Overview of the INEX 2013 social book search track. In: Forner P, Navigli R, Tufis D, Ferro N (eds) Working notes for CLEF 2013 conference, Valencia, Spain, September 23–26, 2013. CEUR-WS.org, CEUR workshop proceedings, vol 1179. http://ceur-ws.org/Vol-1179/CLEF2013wn-INEX-KoolenEt2013b.pdf
  43. Koolen M, Bogers T, Kamps J, Kazai G, Preminger M (2014) Overview of the INEX 2014 social book search track. In: Cappellato L, Ferro N, Halvey M, Kraaij W (eds) Working notes for CLEF 2014 conference, Sheffield, UK, September 15–18, 2014. CEUR-WS.org, CEUR workshop proceedings, vol 1180, pp 462–479. http://ceur-ws.org/Vol-1180/CLEF2014wn-Inex-KoolenEt2014.pdf
  44. Koolen M, Bogers T, Gäde M, Hall MA, Huurdeman HC, Kamps J, Skov M, Toms E, Walsh D (2015a) Overview of the CLEF 2015 social book search lab. In: Mothe J, Savoy J, Kamps J, Pinel-Sauvagnat K, Jones GJF, SanJuan E, Cappellato L, Ferro N (eds) Experimental IR meets multilinguality, multimodality, and interaction. Proceedings of the sixth international conference of the CLEF association (CLEF 2015). Lecture notes in computer science (LNCS) 9283. Springer, Heidelberg, pp 545–564CrossRefGoogle Scholar
  45. Koolen M, Bogers T, Kamps J (2015b) Overview of the SBS 2015 suggestion track. In: Cappellato L, Ferro N, Jones GJF, SanJuan E (eds) Working notes of CLEF 2015 - conference and labs of the evaluation forum, Toulouse, France, September 8–11, 2015. CEUR-WS.org. CEUR workshop proceedings, vol 1391. http://ceur-ws.org/Vol-1391/76-CR.pdf
  46. Koolen M, Bogers T, Gäde M, Hall M, Hendrickx I, Huurdeman HC, Kamps J, Skov M, Verberne S, Walsh D (2016a) Overview of the CLEF 2016 social book search lab. In: Fuhr N, Quaresma P, Gonçalves T, Larsen B, Balog K, Macdonald C, Cappellato L, Ferro N (eds) Experimental IR meets multilinguality, multimodality, and interaction. Proceedings of the seventh international conference of the CLEF association (CLEF 2016). Lecture notes in computer science (LNCS) 9822. Springer, Heidelberg, pp 351–370CrossRefGoogle Scholar
  47. Koolen M, Bogers T, Kamps J (2016b) Overview of the SBS 2016 suggestion track. In: Balog K, Cappellato L, Ferro N, Macdonald C (eds) Working notes of CLEF 2016 - conference and labs of the evaluation forum, Évora, Portugal, 5–8 September, 2016. CEUR-WS.org, CEUR workshop proceedings, vol 1609, pp 1039–1052. http://ceur-ws.org/Vol-1609/16091039.pdf
  48. Koolen M, Kamps J, Bogers T, Belkin NJ, Kelly D, Yilmaz E (2017) Report on the second workshop on supporting complex search tasks. SIGIR Forum 51(1):58–66. http://doi.acm.org/10.1145/3130332.3130343 CrossRefGoogle Scholar
  49. Nordlie R, Pharo N (2012) Seven years of INEX interactive retrieval experiments - lessons and challenges. In: Catarci T, Forner P, Hiemstra D, Peñas A, Santucci G (eds) Information access evaluation. multilinguality, multimodality, and visual analytics. Proceedings of the third international conference of the CLEF initiative (CLEF 2012). Lecture notes in computer science (LNCS) 7488. Springer, Heidelberg, pp 13–23CrossRefGoogle Scholar
  50. Petras V, Ferro N, Gäde M, Isaac A, Kleineberg M, Masiero I, Nicchio M, Stiller J (2012) Cultural heritage in CLEF (CHiC) overview 2012. In: Forner P, Karlgren J, Womser-Hacker C, Ferro N (eds) CLEF 2012 working notes, CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1178/
  51. Petras V, Bogers T, Hall M, Savoy J, Malak P, Pawlowski A, Ferro N, Masiero I (2013) Cultural heritage in CLEF (CHiC) 2013. In: Forner P, Müller H, Paredes R, Rosso P, Stein B (eds) Information access evaluation meets multilinguality, multimodality, and visualization. Proceedings of the fourth international conference of the CLEF initiative (CLEF 2013). Lecture notes in computer science (LNCS) 8138. Springer, Heidelberg, pp 192–211CrossRefGoogle Scholar
  52. SanJuan E, Moriceau V, Tannier X, Bellot P, Mothe J (2012) Overview of the INEX 2012 tweet contextualization track. In: Forner P, Karlgren J, Womser-Hacker C, Ferro N (eds) CLEF 2012 working notes. CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1178/
  53. Thanos C, Casarosa V (2017) The key role of the DELOS network of excellence in establishing digital libraries as a research field in Europe. LIBER Quart 26(4):296–307. http://doi.org/10.18352/lq.10165 CrossRefGoogle Scholar
  54. Trappett M, Geva S, Trotman A, Scholer F, Sanderson M (2012) Overview of the INEX 2012 snippet retrieval track. In: Forner P, Karlgren J, Womser-Hacker C, Ferro N (eds) CLEF 2012 working notes. CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1178/
  55. Trappett M, Geva S, Trotman A, Scholer F, Sanderson M (2013) Overview of the INEX 2013 snippet retrieval track. In: Forner P, Navigli R, Tufis D, Ferro N (eds) Working notes for CLEF 2013 conference, Valencia, Spain, September 23–26, 2013. CEUR-WS.org, CEUR workshop proceedings, vol 1179. http://ceur-ws.org/Vol-1179/CLEF2013wn-INEX-TrappettEt2013.pdf
  56. Wang Q, Kamps J, Ramírez Camps G, Marx M, Schuth A, Theobald M, Gurajada S, Mishra A (2012) Overview of the INEX 2012 linked data track. In: Forner P, Karlgren J, Womser-Hacker C, Ferro N (eds) CLEF 2012 working notes. CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1178/

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Jaap Kamps
    • 1
    Email author
  • Marijn Koolen
    • 2
  • Shlomo Geva
    • 3
  • Ralf Schenkel
    • 4
  • Eric SanJuan
    • 5
  • Toine Bogers
    • 6
  1. 1.University of AmsterdamAmsterdamNetherlands
  2. 2.Royal Netherlands Academy of Arts and SciencesAmsterdamNetherlands
  3. 3.QUTBrisbaneAustralia
  4. 4.University of TrierTrierGermany
  5. 5.University of AvignonAvignonFrance
  6. 6.Aalborg University CopenhagenCopenhagenDenmark

Personalised recommendations