Advertisement

Multi-Lingual Retrieval of Pictures in ImageCLEF

  • Paul CloughEmail author
  • Theodora Tsikrika
Chapter
Part of the The Information Retrieval Series book series (INRE, volume 41)

Abstract

CLEF first launched a multi-lingual visual information retrieval task in 2003 as part of the ImageCLEF track. Several such tasks subsequently followed, encompassing both the medical and non-medical domains. The main aim of such ad hoc image retrieval tasks was to investigate the effectiveness of retrieval approaches that exploit textual and visual evidence in the context of large and heterogeneous collections of images that are searched for by users with diverse information needs. This chapter presents an overview of the image retrieval activities within ImageCLEF from 2003 to 2011, focusing on the non-medical domain and, in particular, on the photographic retrieval and Wikipedia image retrieval tasks. We review the available test collections built in the context of these activities, present the main evaluation results, and summarise the contributions and lessons learned.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

We thank all those involved in helping to coordinate and organise ImageCLEF over the years and all those who have engaged with the track, providing submissions and presenting at CLEF.

References

  1. Agrawal R, Gollapudi S, Halverson A, Ieong S (2009) Diversifying search results. In: Proceedings of the second ACM international conference on web search and data mining, WSDM‘09. ACM, New York, pp 5–14. http://doi.acm.org/10.1145/1498759.1498766 CrossRefGoogle Scholar
  2. Arni T, Clough P, Sanderson M, Grubinger M (2009) Overview of the ImageCLEFphoto 2008 photographic retrieval task. In: Peters C, Deselaers T, Ferro N, Gonzalo J, Jones GJF, Kurimo M, Mandl T, Peñas A (eds) Evaluating systems for multilingual and multimodal information access: ninth workshop of the cross–language evaluation forum (CLEF 2008). Revised Selected Papers, Lecture notes in computer science (LNCS), vol 5706. Springer, Heidelberg, pp 500–511CrossRefGoogle Scholar
  3. Boididou C, Papadopoulos S, Kompatsiaris Y, Schifferes S, Newman N (2014) Challenges of computational verification in social multimedia. In: Proceedings of the 23rd international conference on world wide web, WWW‘14 companion. ACM, New York, pp 743–748. http://doi.acm.org/10.1145/2567948.2579323 CrossRefGoogle Scholar
  4. Buckley C, Voorhees EM (2000) Evaluating evaluation measure stability. In: Proceedings of the 23rd annual international ACM SIGIR conference on research and development in information retrieval, SIGIR‘00. ACM, New York, pp 33–40. http://doi.acm.org/10.1145/345508.345543 CrossRefGoogle Scholar
  5. Buckley C, Voorhees EM (2004) Retrieval evaluation with incomplete information. In: Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval, SIGIR‘04. ACM, New York, pp 25–32. http://doi.acm.org/10.1145/1008992.1009000 Google Scholar
  6. Clarke CL, Kolla M, Cormack GV, Vechtomova O, Ashkan A, Büttcher S, MacKinnon I (2008) Novelty and diversity in information retrieval evaluation. In: Proceedings of the 31st annual international ACM SIGIR conference on research and development in information retrieval, SIGIR‘08. ACM, New York, pp 659–666. http://doi.acm.org/10.1145/1390334.1390446 CrossRefGoogle Scholar
  7. Clough P, Sanderson M (2004) The CLEF 2003 cross language image retrieval track. In: Peters C, Braschler M, Gonzalo J, Kluck M (eds) Comparative evaluation of multilingual information access systems: fourth workshop of the cross–language evaluation forum (CLEF 2003), revised selected papers. Lecture notes in computer science (LNCS), vol 3237. Springer, Heidelberg, pp 581–593CrossRefGoogle Scholar
  8. Clough P, Müller H, Sanderson M (2005) The CLEF 2004 cross-language image retrieval track. In: Peters C, Clough P, Gonzalo J, Jones GJF, Kluck M, Magnini B (eds) Multilingual information access for text, speech and images: fifth workshop of the cross–language evaluation forum (CLEF 2004). Revised selected papers. Lecture notes in computer science (LNCS), vol 3491. Springer, Heidelberg, pp 597–613CrossRefGoogle Scholar
  9. Clough P, Grubinger M, Deselaers T, Hanbury A, Müller H (2006a) Overview of the ImageCLEF 2006 photographic retrieval and object annotation tasks. In: Nardi A, Peters C, Vicedo JL, Ferro N (eds) CLEF 2006 working notes. CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1172/
  10. Clough PD, Sanderson M, Reid N (2006b) The eurovision St Andrews collection of photographs. SIGIR Forum 40(1):21–30. http://doi.acm.org/10.1145/1147197.1147199 CrossRefGoogle Scholar
  11. Clough P, Grubinger M, Deselaers T, Hanbury A, Müller H (2007) Overview of the ImageCLEF 2006 photographic retrieval and object annotation tasks. In: Peters C, Clough P, Gey FC, Karlgren J, Magnini B, Oard DW, de Rijke M, Stempfhuber M (eds) Evaluation of multilingual and multi-modal information retrieval: seventh workshop of the cross–language evaluation forum (CLEF 2006). Revised selected papers, Lecture notes in computer science (LNCS), vol 4730. Springer, Heidelberg, pp 223–256CrossRefGoogle Scholar
  12. Del Bimbo A (1999) Visual information retrieval. Morgan Kaufmann, San FranciscoGoogle Scholar
  13. Grubinger M, Clough P, Leung C (2006) The IAPR TC-12 benchmark for visual information search. IAPR Newsl 28(2):10–12Google Scholar
  14. Grubinger M, Clough P, Hanbury A, Müller H (2008) Overview of the ImageCLEFphoto 2007 photographic retrieval task. In: Peters C, Jijkoun V, Mandl T, Müller H, Oard DW, Peñas A, Petras V, Santos D (eds) Advances in multilingual and multimodal information retrieval: eighth workshop of the cross–language evaluation forum (CLEF 2007). Revised selected papers, Lecture notes in computer science (LNCS), vol 5152. Springer, Heidelberg, pp 433–444CrossRefGoogle Scholar
  15. Husain SS, Bober M (2017) Improving large-scale image retrieval through robust aggregation of local descriptors. IEEE Trans Pattern Anal Mach Intell 39(9):1783–1796.  https://doi.org/10.1109/TPAMI.2016.2613873 CrossRefGoogle Scholar
  16. Müller H, Deselaers T, Grubinger M, Clough P, Hanbury A, Hersh W (2007) Problems with running a successful multimedia retrieval benchmark. In: Proceedings of the third MUSCLE/ImageCLEF workshop on image and video retrieval evaluationGoogle Scholar
  17. Müller H, Kalpathy-Cramer J, Eggel I, Bedrick S, Radhouani S, Bakke B, Khan CE, Jr, Hersh WR (2010) Overview of the CLEF 2009 medical image retrieval track. In: Peters C, Tsikrika T, Müller H, Kalpathy-Cramer J, Jones GJF, Gonzalo J, Caputo B (eds) Multilingual information access evaluation vol. ii multimedia experiments – tenth workshop of the cross–language evaluation forum (CLEF 2009). Revised selected papers, Lecture notes in computer science (LNCS). Springer, Heidelberg, pp 72–84Google Scholar
  18. Paramita ML, Grubinger M (2010) Photographic image retrieval. In: Müller H, Clough PD, Deselaers T, Caputo B (eds) ImageCLEF, experimental evaluation in visual information retrieval. Springer, Berlin, pp 141–162. https://doi.org/10.1007/978-3-642-15181-1_8 CrossRefGoogle Scholar
  19. Paramita ML, Sanderson M, Clough P (2009a) Developing a test collection to support diversity analysis. In: Proceedings of redundancy, diversity, and IDR workshop-SIGIR, vol 9, pp 39–45Google Scholar
  20. Paramita ML, Sanderson M, Clough P (2009b) Diversity in photo retrieval: overview of the imageclefphoto task 2009. In: Workshop of the cross-language evaluation forum for European languages. Springer, Berlin pp 45–59Google Scholar
  21. Popescu A, Tsikrika T, Kludas J (2010) Overview of the wikipedia retrieval task at ImageCLEF 2010. In: Braschler M, Harman DK, Pianta E, Ferro N (eds) CLEF 2010 working notes, CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1176/
  22. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252. https://doi.org/10.1007/s11263-015-0816-y MathSciNetCrossRefGoogle Scholar
  23. Santos RLT, Macdonald C, Ounis I (2015) Search result diversification. Found Trends Inf Retr 9(1):1–90. http://dx.doi.org/10.1561/1500000040 CrossRefGoogle Scholar
  24. Tsikrika T, Kludas J (2009) Overview of the WikipediaMM task at ImageCLEF 2008. In: Peters C, Deselaers T, Ferro N, Gonzalo J, Jones GJF, Kurimo M, Mandl T, Peñas A (eds) Evaluating systems for multilingual and multimodal information access: ninth workshop of the cross–language evaluation forum (CLEF 2008). Revised selected papers, Lecture notes in computer science (LNCS), vol 5706. Springer, Heidelberg, pp 539–550CrossRefGoogle Scholar
  25. Tsikrika T, Kludas J (2010) Overview of the WikipediaMM task at ImageCLEF 2009. In: Peters C, Tsikrika T, Müller H, Kalpathy-Cramer J, Jones GJF, Gonzalo J, Caputo B (eds) Multilingual information access evaluation vol. ii multimedia experiments – tenth workshop of the cross–language evaluation forum (CLEF 2009). Revised selected papers, Lecture notes in computer science (LNCS). Springer, Heidelberg, pp 60–71Google Scholar
  26. Tsikrika T, Garcia Seco de Herrera A, Müller H (2011a) Assessing the scholarly impact of ImageCLEF. In: Forner P, Gonzalo J, Kekäläinen J, Lalmas M, de Rijke M (eds) Multilingual and multimodal information access evaluation. Proceedings of the second international conference of the cross-language evaluation forum (CLEF 2011). Lecture notes in computer science (LNCS), vol 6941. Springer, Heidelberg, pp 95–106CrossRefGoogle Scholar
  27. Tsikrika T, Popescu A, Kludas J (2011b) Overview of the Wikipedia image retrieval task at ImageCLEF 2011. In: Petras V, Forner P, Clough P, Ferro N (eds) CLEF 2011 working notes. CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1177/
  28. Tsikrika T, Kludas J, Popescu A (2012) Building reliable and reusable test collections for image retrieval: the wikipedia task at imageclef. IEEE MultiMedia 19(3):24–33CrossRefGoogle Scholar
  29. Tsikrika T, Larsen B, Müller H, Endrullis S, Rahm E (2013) The scholarly impact of CLEF (2000–2009). In: Forner P, Müller H, Paredes R, Rosso P, Stein B (eds) Information access evaluation meets multilinguality, multimodality, and visualization. Proceedings of the fourth international conference of the CLEF initiative (CLEF 2013). Lecture notes in computer science (LNCS), vol 8138. Springer, Heidelberg, pp 1–12CrossRefGoogle Scholar
  30. Wan J, Wang D, Hoi SCH, Wu P, Zhu J, Zhang Y, Li J (2014) Deep learning for content-based image retrieval: a comprehensive study. In: Proceedings of the 22Nd ACM international conference on multimedia, MM‘14. ACM, New York, pp 157–166. http://doi.acm.org/10.1145/2647868.2654948 Google Scholar
  31. Westerveld T, van Zwol R (2007) Multimedia retrieval at inex 2006. ACM SIGIR Forum 41(1):58–63CrossRefGoogle Scholar
  32. Zhai CX, Cohen WW, Lafferty J (2003) Beyond independent relevance: methods and evaluation metrics for subtopic retrieval. In: Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval, SIGIR‘03. ACM, New York, pp 10–17. http://doi.acm.org/10.1145/860435.860440 Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Information SchoolUniversity of SheffieldSheffieldUK
  2. 2.Information Technologies Institute, Centre for Research and Technology HellasThessalonikiGreece

Personalised recommendations