Adaptive and Optimal Combination of Local Features for Image Retrieval

  • Neelanjan BhowmikEmail author
  • Valérie Gouet-Brunet
  • Lijun Wei
  • Gabriel Bloch
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10133)


With the large number of local feature detectors and descriptors in the literature of Content-Based Image Retrieval (CBIR), in this work we propose a solution to predict the optimal combination of features, for improving image retrieval performances, based on the spatial complementarity of interest point detectors. We review several complementarity criteria of detectors and employ them in a regression based prediction model, designed to select the suitable detectors combination for a dataset. The proposal can improve retrieval performance even more by selecting optimal combination for each image (and not only globally for the dataset), as well as being profitable in the optimal fitting of some parameters. The proposal is appraised on three state-of-the-art datasets to validate its effectiveness and stability. The experimental results highlight the importance of spatial complementarity of the features to improve retrieval, and prove the advantage of using this model to optimally adapt detectors combination and some parameters.


CBIR Interest points Feature combination Spatial complementarity Regression model 



The authors are grateful to Nicéphore Cité, Institut national de l’information géographique et forestière (IGN) and French project POEME ANR-12-CORD-0031 for the financial support.


  1. 1.
    Agrawal, M., Konolige, K., Blas, M.: Censure: center surround extremas for realtime feature detection and matching. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5305, pp. 102–115. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  2. 2.
    Atrey, P.K., Hossain, M.A., Saddik, A.E., Kankanhalli, M.S.: Multimodal fusion for multimedia analysis: a survey. Multimed. Syst. 16, 345–379 (2010)CrossRefGoogle Scholar
  3. 3.
    Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 11(3), 346–359 (2008)CrossRefGoogle Scholar
  4. 4.
    Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. Pattern Anal. Mach. Intell. 24(4), 509–522 (2002)CrossRefGoogle Scholar
  5. 5.
    Bhowmik, N., Gonzalez, V.R., Gouet-Brunet, V., Pedrini, H., Bloch, G.: Efficient fusion of multidimensional descriptors for image retrieval. In: International Conference on Image Processing, pp. 5766–5770, October 2014Google Scholar
  6. 6.
    Deselaers, T., Keysers, D., Ney, H.: Features for image retrieval: an experimental comparison. Inf. Retr. 11(2), 77–107 (2008)CrossRefGoogle Scholar
  7. 7.
    Ehsan, S., Clark, A.F., McDonald-Maier, K.D.: Rapid online analysis of local feature detectors and their complementarity. Sensors 13(8), 10876 (2013)CrossRefGoogle Scholar
  8. 8.
    Ferreira, C.D., Santos, J.A., da Silva Torres, R., Goncalves, M.A., Rezende, R.C., Fan, W.: Relevance feedback based on genetic programming for image retrieval. Pattern Recogn. Lett. 32(1), 27–37 (2011). Image Processing, Computer Vision and Pattern Recognition in Latin AmericaCrossRefGoogle Scholar
  9. 9.
    Gales, G., Crouzil, A., Chambon, S.: Complementarity of feature point detectors. In: Richard, P., Braz, J. (eds.) VISAPP(1), pp. 334–339. INSTICC Press, Setubal (2010)Google Scholar
  10. 10.
    Heidemann, G.: Focus-of-attention from local color symmetries. Pattern Anal. Mach. Intell. 26(7), 817–830 (2004)CrossRefGoogle Scholar
  11. 11.
    Leutenegger, S., Chli, M., Siegwart, R.: BRISK: binary robust invariant scalable keypoints. In: International Conference on Computer Vision, pp. 2548–2555, November 2011Google Scholar
  12. 12.
    Li, X., Larson, M., Hanjalic, A.: Pairwise geometric matching for large-scale object retrieval. In: Computer Vision and Pattern Recognition, pp. 5153–5161, June 2015Google Scholar
  13. 13.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  14. 14.
    Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proceedings of the British Machine Vision Conference, pp. 36.1–36.10 (2002)Google Scholar
  15. 15.
    Mikolajczyk, K., Leibe, B., Schiele, B.: Local features for object class recognition. In: International Conference on Computer Vision, vol. 2, pp. 1792–1799, October 2005Google Scholar
  16. 16.
    Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. Int. J. Comput. Vis. 60(1), 63–86 (2004)CrossRefGoogle Scholar
  17. 17.
    Neshov, N.: Comparison on late fusion methods of low level features for content based image retrieval. In: Mladenov, V., Koprinkova-Hristova, P., Palm, G., Villa, A.E., Appollini, B., Kasabov, N. (eds.) ICANN 2013. LNCS, vol. 8131, pp. 619–627. Springer, Heidelberg (2013)Google Scholar
  18. 18.
    Rashedi, E., Nezamabadi-pour, H., Saryazdi, S.: A simultaneous feature adaptation and feature selection method for content-based image retrieval systems. Knowl.-Based Syst. 39, 85–94 (2013)CrossRefGoogle Scholar
  19. 19.
    Risojevic, V., Babic, Z.: Fusion of global and local descriptors for remote sensing image classification. IEEE Geosci. Remote Sens. Lett. 10(4), 836–840 (2013)CrossRefGoogle Scholar
  20. 20.
    Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to sift or surf. In: International Conference on Computer Vision, pp. 2564–2571, November 2011Google Scholar
  21. 21.
    da Silva Torres, R., Falcao, A.X., Goncalves, M.A., Papa, J.P., Zhang, B., Fan, W., Fox, E.A.: A genetic programming framework for content-based image retrieval. Pattern Recogn. 42(2), 283–292 (2009). Learning Semantics from Multimedia ContentCrossRefzbMATHGoogle Scholar
  22. 22.
    Schmid, C., Mohr, R.: Local grayvalue invariants for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 19(5), 530–534 (1997)CrossRefGoogle Scholar
  23. 23.
    Snoek, C.G.M., Worring, M., Smeulders, A.W.M.: Early versus late fusion in semantic video analysis. In: Proceedings of the 13th Annual ACM International Conference on Multimedia. pp. 399–402. ACM, New York (2005)Google Scholar
  24. 24.
    Sun, J.: Local selection of features for image search and annotation. In: Proceedings of the 22nd ACM International Conference on Multimedia. pp. 655–658. ACM, New York (2014)Google Scholar
  25. 25.
    Wacht, M., Shan, J., Qi, X.: A short-term and long-term learning approach for content-based image retrieval. In: 2006 Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2006, vol. 2, p. II, May 2006Google Scholar
  26. 26.
    Yue, J., Li, Z., Liu, L., Fu, Z.: Content-based image retrieval using color and texture fused features. Math. Comput. Model. 54(3–4), 1121–1127 (2011). Mathematical and Computer Modeling in AgricultureCrossRefGoogle Scholar
  27. 27.
    Zhang, W., Qin, Z., Wan, T.: Image scene categorization using multi-bag-of-features. In: Proceedings of International Conference on Machine Learning and Cybernetics, vol. 4, pp. 1804–1808 (2011)Google Scholar
  28. 28.
    Zhou, Y., Zeng, D., Zhang, S., Tian, Q.: Augmented feature fusion for image retrieval system. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 447–450. ACM, New York (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Neelanjan Bhowmik
    • 1
    • 2
    Email author
  • Valérie Gouet-Brunet
    • 1
  • Lijun Wei
    • 1
  • Gabriel Bloch
    • 2
  1. 1.University Paris-Est, LASTIG MATIS, IGN, ENSGSaint-MandeFrance
  2. 2.Nicéphore CitéChalon-sur-SaôneFrance

Personalised recommendations