Weighted multi-deep ranking supervised hashing for efficient image retrieval

  • Jiayong Li
  • Wing W. Y. NgEmail author
  • Xing TianEmail author
  • Sam Kwong
  • Hui Wang
Original Article


Deep hashing has proven to be efficient and effective for large-scale image retrieval due to the strong representation capability of deep networks. Existing deep hashing methods only utilize a single deep hash table. In order to achieve both higher retrieval recall and precision, longer hash codes can be used but at the expense of higher space usage. To address this issue, a novel deep hashing method is proposed in this paper, weighted multi-deep ranking supervised hashing (WMDRH), which employs multiple weighted deep hash tables to improve precision/recall without increasing space usage. The hash table is constructed as an additional layer in a deep network. Hash codes are generated by minimizing the loss function that contains two terms: (1) the ranking pairwise loss and (2) the classification loss. The ranking pairwise loss ensures to generate discriminative hash codes by penalizing more for the (dis)similar image pairs with (small)large Hamming distances. The classification loss guarantees the hash codes to be effective for category prediction. Different hash bits in each individual hash table are treated differently by assigning corresponding weights based on information preservation and bit diversity. Moreover, multiple hash tables are integrated by assigning the appropriate weight to each table according to its mean average precision (MAP) score for image retrieval. Experiments on three widely-used image databases show the proposed method outperforms state-of-the-art hashing methods.


Deep hashing Image retrieval Multi-table Weighting Ranking loss 



This work was supported by National Natural Science Foundation of China under Grants 61876066, 61572201 and 61672443, Guangzhou Science and Technology Plan Project 201804010245, Hong Kong RGC General Research Funds under Grant 9042038 (CityU 11205314) and Grant 9042322 (CityU 11200116), and EU Horizon 2020 Programme (700381, ASGARD).


  1. 1.
    Datar M, Immorlica N, Indyk P, Mirrokni VS (2004) Locality-sensitive hashing scheme based on p-stable distributions. In: Proc. of the twentieth annual symposium on computational geometry, pp 253–262Google Scholar
  2. 2.
    Gong Y, Lazebnik S, Gordo A, Perronnin F (2013) Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans Pattern Anal Mach Intell 35(12):2916–2929CrossRefGoogle Scholar
  3. 3.
    Weiss Y, Torralba A, Fergus R (2009) Spectral hashing. In: Proc. of advances in neural information processing systems, pp 1753–1760Google Scholar
  4. 4.
    Norouzi M,Fleet DJ (2011) Minimal loss hashing for compact binary codes. In: Proc. international conference on machine learning, pp 353–360Google Scholar
  5. 5.
    Dniz G, Bueno J, la Salido TFD (2011) Face recognition using histograms of oriented gradients. Proc Pattern Recogn Lett 32(12):1598–1603CrossRefGoogle Scholar
  6. 6.
    Huang D, Shan C, Ardabilian M, Wang Y, Chen L (2011) Local binary patterns and its application to facial image analysis: a survey. IEEE Trans Syst Man Cybern 41(6):765–781CrossRefGoogle Scholar
  7. 7.
    Ahonen T, Rahtu E, Ojansivu V, Heikkila J (2008) Recognition of blurred faces using local phase quantization. In: Proc. international conference on pattern recognitionGoogle Scholar
  8. 8.
    Paul E, Ajeena BAS (2015) Mining images for image annotation using surf detection technique. In: Proc. international conference on control, communication and computing IndiaGoogle Scholar
  9. 9.
    Purandare V, Talele KT (2014) Efficient heterogeneous face recognition using scale invariant feature transform. In: Proc. international conference on circuits, systems, communication and information technology applicationsGoogle Scholar
  10. 10.
    Liu H, Wang R, Shan S, Chen X (2016) Deep supervised hashing for fast image retrieval. In: Proc. IEEE conference on computer vision and pattern recognition, pp 2064–2072Google Scholar
  11. 11.
    Xia R, Pan Y, Lai H, Liu C, Yan S (2014) Supervised hashing for image retrieval via image representation learning. In: Proc. AAAI conference on artificial intelligence, pp 2156–2162Google Scholar
  12. 12.
    Lai H, Pan Y, Liu Y, Yan S (2015) Simultaneous feature learning and hash coding with deep neural networks, pp 3270–3278Google Scholar
  13. 13.
    Yao T, Long F, Mei T, Rui Y (2016) Deep semantic-preserving and ranking-based hashing for image retrieval. In: Proc. international joint conference on artificial intelligence, pp 3931–3937Google Scholar
  14. 14.
    Zhang R, Lin L, Zhang R, Zuo W, Zhang L (2015) Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Trans Image Process 24(12):4766–4779MathSciNetCrossRefGoogle Scholar
  15. 15.
    Lin J, Li Z, Tang J (2017) Discriminative deep hashing for scalable face image retrieval. In: Proc. international joint conference on artificial intelligence, pp 2266–2272Google Scholar
  16. 16.
    Xu H, Wang J, Li Z, Zeng G, Li S, Yu N (2011) Complementaryhashing for approximate nearest neighbor search. In: Proc international conference on computer visionGoogle Scholar
  17. 17.
    Li P, Cheng J, Lu H (2013) Hashing with dual complementary projection learning for fast image retrieval. Proc Neurocomput 120(10):83–89CrossRefGoogle Scholar
  18. 18.
    Raginsky M, Lazebnik S (2009) Locality-sensitive binary codes from shift-invariant kernels. In: Proc of Conf and workshop on neural information processing systems, pp 1509–1517Google Scholar
  19. 19.
    Ng WWY, Lv Y, Zeng Z, Yeung DS, Chan PPK (2017) Sequential conditional entropy maximization semi-supervised hashing for semantic image retrieval. Int J Mach Learn Cybern 8(2):571–586CrossRefGoogle Scholar
  20. 20.
    Liu Y, Feng L, Liu S, Sun M (2019) An ELM based local topology preserving hashing. Int J Mach Learn Cybern 2019:1–18Google Scholar
  21. 21.
    Kulis B, Grauman K (2009) Kernelized locality-sensitive hashing for scalable image search. In: Proc of IEEE Int Conf on computer vision, pp 2130–2137Google Scholar
  22. 22.
    Lv Y, Ng WW, Zeng Z, Yeung DS, Chan PP (2015) Asymmetric cyclical hashing for large scale image retrieval. IEEE Trans Multimedia 17(8):1225–1235CrossRefGoogle Scholar
  23. 23.
    Guo Y, Ding G, Liu L, Han J, Shao L (2017) Learning to hash with optimized anchor embedding for scalable retrieval. IEEE Trans Image Process 26(3):1344–1354MathSciNetCrossRefGoogle Scholar
  24. 24.
    Liu X, Du B, Deng C, Liu M, Lang B (2016) Structure sensitive hashing with adaptive product quantization. IEEE Trans Cybern 46(10):2252–2264CrossRefGoogle Scholar
  25. 25.
    Wang J, Kumar S, Chang S-F (2012) Semi-supervised hashing for large-scale search. IEEE Trans Pattern Anal Mach Intell 34(12):2393–2406CrossRefGoogle Scholar
  26. 26.
    Wu C, Zhu J, Cai D, Chen C, Bu J (2013) Semi-supervised nonlinear hashing using bootstrap sequential projection learning. IEEE Trans Knowl Data Eng 25(6):1380–1393CrossRefGoogle Scholar
  27. 27.
    Liu W, Wang J, Ji R, Jiang Y-G, Chang S-F (2012) Supervised hashing with kernels. In: Proc. computer vision and pattern recognition, pp 2074–2081Google Scholar
  28. 28.
    Kulis B, Darrell T (2009) Learning to hash with binary reconstructive embeddings. In: Proc. neural information processing systems, pp 1042–1050Google Scholar
  29. 29.
    Song D, Liu RJW, Meyer DA, Smith JR (2015) Top rank supervised binary coding for visual search. In: Proc. IEEE international conference on computer vision, pp 1922–1930Google Scholar
  30. 30.
    Ng WWY, Li J, Feng S, Yeung DS, Chan PPK (2015) Sensitivity based image filtering for multi-hashing in large scale image retrieval problems. Int J Mach Learn Cybern 6(5):777–794CrossRefGoogle Scholar
  31. 31.
    Salakhutdinov R, Hinton G (2009) Semantic hashing. Proc Int J Approx Reasoning 50(7):969–978CrossRefGoogle Scholar
  32. 32.
    Zhu H, Long M, Wang J, Cao Y (2016) Deep hashing network for efficient similarity retrieval. In: Proc AAAI conference on artificial intelligence, pp 2415–2421Google Scholar
  33. 33.
    Li W, Wang S, Kang W (2016) Feature learning based deep supervised hashing with pairwise labels. In: Proc. international joint conference on artificial intelligence, pp 1711–1717Google Scholar
  34. 34.
    Wang X, Shi Y, Kitani K (2016) Deep supervised hashing with triplet labels. In: Proc. Asian conference on computer vision, pp 70–84Google Scholar
  35. 35.
    Liu Y, Song J, Zhou K, Yan L, Liu L, Zou F, Shao L (2018) Deep self-taught hashing for image retrieval. IEEE Trans Cybern 2018:1–13Google Scholar
  36. 36.
    Galton F (1892) Finger prints. Macmillan, LondonGoogle Scholar
  37. 37.
    Chua T-S, Tang J, Hong R, Li H, Luo Z, Zheng Y (2009) Nus-wide: a real-world web image database from national university of singapore. In: Proceeding crence on image and video retrieval. ACM, p 48Google Scholar
  38. 38.
    Liu W, Wang J, Ji R, Jiang Y, Chang S (2012) Supervised hashing with kernels. In: Proc. IEEE conference on computer vision and pattern recognitionGoogle Scholar
  39. 39.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Proc. advances in neural information processing systemsGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Guangdong Provincial Key Lab of Computational Intelligence & Cyberspace Information, School of Computer Science and EngineeringSouth China University of TechnologyGuangzhouChina
  2. 2.Department of Computer ScienceHong Kong City UniversityKowloon TongHong Kong
  3. 3.School of ComputingUlster UniversityJordanstownUK

Personalised recommendations