Consensus Guided Multiple Match Removal for Geometry Verification in Image Retrieval

  • Hong WuEmail author
  • Xing Heng
  • Zenglin Xu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9517)


State of the art image retrieval methods are mostly based on the bag-of-features (BOF) representation or its variations. Despite its success, BOF ignores the geometric relationships among local features, and thus leading to limited retrieval accuracy. This limitation has resulted in the necessity of geometric verification of the initial matches detected by visual word based matching. However, this approximate matching usually leads to many-to-many mapping, and thus introduces many false matches impairing geometric verification. To address this problem, we propose a Consensus-Guided Multiple Match Removal algorithm, which selects the confident matches relying not only on the quality of the matches, but also on their geometric consistency with others. For efficiency, the geometric consistency is verified by exploring match distribution over the Hough space instead of pairwise comparison. The matches in conflict with the confident ones are then removed based on a feature mapping constrain. Our multiple match removal method can be combined with existing geometric verification methods to improve image retrieval. Finally, we evaluate the proposed method on three benchmark datasets in comparison with previous multiple match removal method. The experimental results indicate that multiple match removal is a useful step for geometric verification, and our method can outperform the previous multiple match removal method and further improve the image retrieval performance.


Image retrieval Multiple match removal Geometric verification 



This work was supported by projects of NSF China (No. 61572111, 61433014, 61440036), a 973 project of China (No.2014CB340401), a 985 Project of UESTC (No.A1098531023601041) and a Basic Research Project of China Central University (No. ZYGX2014J058).


  1. 1.
    Sivic, J., Zisserman, A.: Video google: a text retrieval approach to object matching in videos. In: Ninth IEEE International Conference on computer vision, pp. 1470–1477. Nice (2003)Google Scholar
  2. 2.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  3. 3.
    Jegou, H., Douze, M., Schmid, C.: Hamming embedding and weak geometric consistency for large scale image search. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 304–317. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  4. 4.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. Minneapolis (2007)Google Scholar
  5. 5.
    Fischler, M., Bolles, R.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Chum, O., Matas, J., Obdrzalek, S.: Enhancing RANSAC by generalized model optimization. In: Asia Conference on Computer Vision, pp. 812–817 (2004)Google Scholar
  7. 7.
    Philbin, J., Chum, O., Sivic, J., Isard, M., Zisserman, A.: Lost in quantization: improving particular object retrieval in large scale image databases. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. Anchorage (2008)Google Scholar
  8. 8.
    Jégou, H., Douze, M., Schmid C.: On the burstiness of visual elements. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1169–1176. Miami (2009)Google Scholar
  9. 9.
    Li, X., Larson, M., Hanjalic, A.: Pairwise geometric matching for large-scale object retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1169–1176. Boston (2015)Google Scholar
  10. 10.
    Leordeanu, M., Hebert M.: A spectral technique for correspondence problems using pairwise constraints. In: Tenth IEEE International Conference on Computer Vision, pp. 1482–1489. Beijing (2005)Google Scholar
  11. 11.
    Nistér, D., Stewénius, H.: Scalable recognition with a vocabulary tree. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2161–2168 (2006)Google Scholar
  12. 12.
    Mikolajczyk, K.: Binaries for affine covariant region descriptors. (2007)

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringUniversity of Electronic Science and Technology of ChinaChengduChina
  2. 2.Big Data Research CenterUniversity of Electronic Science and Technology of ChinaChengduChina

Personalised recommendations