External Query Reformulation for Text-Based Image Retrieval

  • Jinming Min
  • Gareth J. F. Jones
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7024)


In text-based image retrieval, the Incomplete Annotation Problem (IAP) can greatly degrade retrieval effectiveness. A standard method used to address this problem is pseudo relevance feedback (PRF) which updates user queries by adding feedback terms selected automatically from top ranked documents in a prior retrieval run. PRF assumes that the target collection provides enough feedback information to select effective expansion terms. This is often not the case in image retrieval since images often only have short metadata annotations leading to the IAP. Our work proposes the use of an external knowledge resource (Wikipedia) in the process of refining user queries. In our method, Wikipedia documents strongly related to the terms in user query (“definition documents”) are first identified by title matching between the query and titles of Wikipedia articles. These definition documents are used as indicators to re-weight the feedback documents from an initial search run on a Wikipedia abstract collection using the Jaccard coefficient. The new weights of the feedback documents are combined with the scores rated by different indicators. Query-expansion terms are then selected based on these new weights for the feedback documents. Our method is evaluated on the ImageCLEF WikipediaMM image retrieval task using text-based retrieval on the document metadata fields. The results show significant improvement compared to standard PRF methods.


Image Retrieval Query Expansion User Query Mean Average Precision Feedback Term 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tsikrika, T., Kludas, J.: Overview of the wikipediamm task at imageCLEF 2008. In: Peters, C., Deselaers, T., Ferro, N., Gonzalo, J., Jones, G.J.F., Kurimo, M., Mandl, T., Peñas, A., Petras, V. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 539–550. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  2. 2.
    Elsas, J.L., Arguello, J., Callan, J., Carbonell, J.G.: Retrieval and feedback models for blog feed search. In: SIGIR 2008: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development In Information Retrieval, pp. 347–354. ACM, New York (2008)Google Scholar
  3. 3.
    Yang, X., Jones, G.J.F., Wang, B.: Query dependent pseudo-relevance feedback based on Wikipedia. In: SIGIR 2009: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 59–66. ACM, New York (2009)Google Scholar
  4. 4.
    Yin, Z., Shokouhi, M., Craswell, N.: Query expansion using external evidence. In: Boughanem, M., Berrut, C., Mothe, J., Soule-Dupuy, C. (eds.) ECIR 2009. LNCS, vol. 5478, pp. 362–374. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  5. 5.
    Kwok, K.L.: Improving English and Chinese ad-hoc retrieval: A Tipster text phase 3 project report. Inf. Retr. 3(4), 313–338 (2000)CrossRefzbMATHGoogle Scholar
  6. 6.
    Xu, Y., Ding, F., Wang, B.: Entity-based query reformulation using Wikipedia. In: CIKM 2008: Proceeding of the 17th ACM Conference on Information and Knowledge Management, pp. 1441–1442. ACM, New York (2008)Google Scholar
  7. 7.
    Weerkamp, W., de Rijke, M.: External query expansion in the blogosphere. In: Seventeenth Text Retrieval Conference (TREC 2008), NIST (February 2009)Google Scholar
  8. 8.
    Custis, T., Al-Kofahi, K.: Investigating external corpus and clickthrough statistics for query expansion in the legal domain. In: CIKM 2008: Proceeding of the 17th ACM Conference on Information and Knowledge Management, pp. 1363–1364. ACM, New York (2008)Google Scholar
  9. 9.
    Weerkamp, W., Balog, K., de Rijke, M.: A generative blog post retrieval model that uses query expansion based on external collections. In: ACL-IJCNLP 2009: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, vol. 2, pp. 1057–1065. Association for Computational Linguistics, Morristown (2009)Google Scholar
  10. 10.
    Hersh, W.R., Bhupatiraju, R.T., Price, S.: Phrases, boosting, and query expansion using external knowledge resources for genomic information retrieval. In: TREC, pp. 503–509 (2003)Google Scholar
  11. 11.
    Min, J., Wilkins, P., Leveling, J., Jones, G.J.F.: DCU at WikipediaMM 2009: Document expansion from Wikipedia abstracts. In: Working Notes for the CLEF 2009 Workshop, Corfu, Greece (2009)Google Scholar
  12. 12.
    Robertson, S., Spärck Jones, K.: Simple, proven approaches to text retrieval. Technical Report UCAM-CL-TR-356, University of Cambridge, Computer Laboratory (December 1994)Google Scholar
  13. 13.
    Jaccard, P.: Etude comparative de la distribution florale dans une portion des alpes et des jura. Bulletin de la Socit Vaudoise des Sciences Naturelles 37, 547C–579C (1901)Google Scholar
  14. 14.
    Westerveld, T., van Zwol, R.: The INEX 2006 multimedia track. In: Fuhr, N., Lalmas, M., Trotman, A. (eds.) INEX 2006. LNCS (LNAI), vol. 4518, pp. 331–344. Springer, Heidelberg (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Jinming Min
    • 1
  • Gareth J. F. Jones
    • 1
  1. 1.Centre for Next Generation Localisation School of ComputingDublin City UniversityDublin 9Ireland

Personalised recommendations