Skip to main content
SpringerLink
Log in

Utilizing similarity relationships among existing data for high accuracy processing of content-based image retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Retrieving similar images based on its visual content is an important yet difficult problem. We propose in this paper a new method to improve the accuracy of content-based image retrieval systems. Typically, given a query image, existing retrieval methods return a ranked list based on the similarity scores between the query and individual images in the database. Our method goes further by relying on an analysis of the underlying connections among individual images in the database to improve this list. Initially, we consider each image in the database as a query and use an existing baseline method to search for its likely similar images. Then, the database is modeled as a graph where images are nodes and connections among possibly similar images are edges. Next, we introduce an algorithm to split this graph into stronger subgraphs, based on our notion of graph’s strength, so that images in each subgraph are expected to be truly similar to each other. We create for each subgraph a structure called integrated image which contains the visual features of all images in the subgraph. At query time, we compute the similarity scores not only between the query and individual database images but also between the query and the integrated images. The final similarity score of a database image is computed based on both its individual score and the score of the integrated image that it belongs to. This leads effectively to a re-ranking of the retrieved images. We evaluate our method on a common image retrieval benchmark and demonstrate a significant improvement over the traditional bag-of-words retrieval model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. http://opencv.willowgarage.com/wiki/

References

  1. Bay H, Ess A, Tuytelaars T, Gool LV (2008) Speeded-up robust features (surf). Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  2. Cao Y, Wang C, Li Z, Zhang L, Zhang L (2010) Spatial-bag-of-features. In: CVPR, pp 3352–3359

  3. Chi M, Zhang P, Zhao Y, Feng R, Xue X (2009) Web image retrieval reranking with multi-view clustering. In: WWW, pp 1189–1190

  4. Chum O, Mikulík A, Perdoch M, Matas J (2011) Total recall ii: query expansion revisited. In: CVPR, pp 889–896

  5. Chum O, Philbin J, Sivic J, Isard M, Zisserman A (2007) Total recall: automatic query expansion with a generative feature model for object retrieval. In: ICCV, pp 1–8

  6. Datar M, Immorlica N, Indyk P, Mirrokni VS (2004) Locality-sensitive hashing scheme based on p-stable distributions. In: Symposium on computational geometry, pp 253–262

  7. Elsayad I, Martinet J, Urruty T, Djeraba C (2012) Toward a higher-level visual representation for content-based image retrieval. Multimed Tools Appl 60(2):455–482

    Article  Google Scholar 

  8. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    Article  MathSciNet  Google Scholar 

  9. Gionis A, Indyk P, Motwani R (1999) Similarity search in high dimensions via hashing. In: VLDB, pp 518–529

  10. Hayter AJ (2007) Probability and Statistics for Engineering and Scientists, 3rd edn. Thomson Brooks/Cole, Belmont, CA

    Google Scholar 

  11. Hsiao JH, Chen MS (2009) Intention-focused active reranking for image object retrieval. In: CIKM, pp 157–166

  12. Huang Y, Zhang J, Zhao Y, Ma D (2012) A new re-ranking method using enhanced pseudo-relevance feedback for content-based medical image retrieval. IEICE Trans 95-D(2):694–698

    Google Scholar 

  13. Jain P, Kulis B, Grauman K (2008) Fast image search for learned metrics. In: CVPR

  14. Jegou H, Douze M, Schmid C (2008) Hamming embedding and weak geometric consistency for large scale image search. In: ECCV (1), pp 304–317

  15. Jégou H, Douze M, Schmid C (2009) Packing bag-of-features. In: ICCV, pp 2357–2364

  16. Jegou H, Douze M, Schmid C (2010) Improving bag-of-features for large scale image search. Int J Comput Vis 87(3):316–336

    Article  Google Scholar 

  17. Jegou H, Douze M, Schmid C, Pérez P (2010) Aggregating local descriptors into a compact image representation. In: CVPR, pp 3304–3311

  18. Jegou H, Harzallah H, Schmid C (2007) A contextual dissimilarity measure for accurate and efficient image search. In: CVPR

  19. Kilinç D, Alpkocak A (2011) An expansion and reranking approach for annotation-based image retrieval from web. Expert Syst. Appl. 38(10):13121–13127

    Article  Google Scholar 

  20. Kulis B, Grauman K (2009) Kernelized locality-sensitive hashing for scalable image search. In: ICCV, pp 2130–2137

  21. Li J, Ma Q, Asano Y, Yoshikawa M (2012) Re-ranking by multi-modal relevance feedback for content-based social image retrieval. In: APWeb, pp 399–410

  22. Lin WH, Jin R, Hauptmann AG (2003) Web image retrieval re-ranking with relevance model. In: Web Intelligence, pp 242–248

  23. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  24. Lu H, Ooi BC, Shen HT, Xue X (2006) Hierarchical indexing structure for efficient similarity search in video retrieval. IEEE Trans Knowl Data Eng 18(11):1544–1559

    Article  Google Scholar 

  25. Mai HT, Kim MH (2012) Integrating similar images to effectively improve image retrieval accuracy. In: ICCM, pp 410–415

  26. Nistér D, Stewénius H (2006) Scalable recognition with a vocabulary tree. In: CVPR (2), pp 2161–2168

  27. Park G, Baek Y, Lee HK (2005) Re-ranking algorithm using post-retrieval clustering for content-based image retrieval. Inf Process Manag 41(2):177–194

    Article  MATH  Google Scholar 

  28. Perdoch M, Chum O, Matas J (2009) Efficient representation of local geometry for large scale object retrieval. In: CVPR, pp 9–16

  29. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2007) Object retrieval with large vocabularies and fast spatial matching. In: CVPR

  30. Qin D, Gammeter S, Bossard L, Quack T, Gool LJV (2011) Hello neighbor: accurate object retrieval with k-reciprocal nearest neighbors. In: CVPR, pp 777–784

  31. Rahman MM, Bhattacharya P (2009) Image retrieval with automatic query expansion based on local analysis in a semantical concept feature space. In: CIVR

  32. Ramaswamy S, Rose K (2011) Adaptive cluster distance bounding for high-dimensional indexing. IEEE Trans Knowl Data Eng 23(6):815–830

    Article  Google Scholar 

  33. Sivic J, Zisserman A (2003) Video google: a text retrieval approach to object matching in videos. In: ICCV, pp 1470–1477

  34. Wang J, Kumar O, Chang SF (2010) Semi-supervised hashing for scalable image retrieval. In: CVPR, pp 3424–3431

  35. Weber R, Schek HJ, Blott S (1998) A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces. In: VLDB, pp 194–205

  36. Weiss Y, Torralba A, Fergus R (2008) Spectral hashing. In: NIPS, pp 1753–1760

  37. Zhang Y, Jia Z, Chen T (2011) Image retrieval with geometry-preserving visual phrases. In: CVPR, pp 809–816

Download references

Acknowledgements

This work was supported by the Brain Korea 21 Project, the Department of Computer Science, KAIST in 2012 and the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) (No. 2012R1A2A2A01046694). The authors also thank anonymous reviewers for valuable comments to improve this work.

Author information

Authors and Affiliations

  1. Department of Computer Science, KAIST, 291 Daehak-ro, Yuseong-gu, Daejeon, 305-701, Korea

    Hai Thanh Mai & Myoung Ho Kim

Authors
  1. Hai Thanh Mai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Myoung Ho Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hai Thanh Mai.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mai, H.T., Kim, M.H. Utilizing similarity relationships among existing data for high accuracy processing of content-based image retrieval. Multimed Tools Appl 72, 331–360 (2014). https://doi.org/10.1007/s11042-013-1360-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1360-9

Keywords

Search

Navigation