Advertisement

GrowBit: Incremental Hashing for Cross-Modal Retrieval

  • Devraj MandalEmail author
  • Yashas Annadani
  • Soma Biswas
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11364)

Abstract

Cross-modal retrieval using hashing techniques is gaining increasing importance due to its efficient storage, scalability and fast query processing speeds. In this work, we address a related and relatively unexplored problem: given a set of cross-modal data with their already learned hash codes, can we increase the number of bits to better represent the data without relearning everything? This scenario is especially important when the number of tags describing the data increases, necessitating longer hash codes for better representation. To tackle this problem, we propose a novel approach called GrowBit, which incrementally learns the bits in the hash code and thus utilizes all the bits learned so far. We develop a two-stage approach for learning the hash codes and hash functions separately, utilizing a recent formulation which decouples over the bits so that it can incorporate the incremental approach. Experiments on MirFlickr, IAPR-TC-12 and NUS-WIDE datasets show the usefulness of the proposed approach.

Keywords

Cross-modal retrieval Hashing Incremental learning 

References

  1. 1.
    Bronstein, M.M., Bronstein, A.M., Michel, F., Paragios, N.: Data fusion through cross-modality metric learning using similarity-sensitive hashing. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3594–3601 (2010)Google Scholar
  2. 2.
    Cakir, F., He, K., Sclaroff, S.: Hashing with binary matrix pursuit. arXiv preprint arXiv:1808.01990 (2018)
  3. 3.
    Chua, T.S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: NUS-WIDE: a real-world web image database from National University of Singapore. In: 2009 ACM International Conference on Image and Video Retrieval (ACM-CIVR), pp. 48–56 (2009)Google Scholar
  4. 4.
    Dai, Q., Li, J., Wang, J., Jiang, Y.G.: Binary optimized hashing. In: 2016 ACM on Multimedia Conference (ACM-MM), pp. 1247–1256 (2016)Google Scholar
  5. 5.
    Ding, G., Guo, Y., Zhou, J.: Collective matrix factorization hashing for multimodal data. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2075–2082 (2014)Google Scholar
  6. 6.
    Escalante, H.J., et al.: The segmented and annotated IAPR TC-12 benchmark. Comput. Vis. Image Underst. 114, 419–428 (2010)CrossRefGoogle Scholar
  7. 7.
    Fatih, C., He, K., Bargal, S.A., Sclaroff, S.: MIHash: online hashing with mutual information. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 437–445 (2017)Google Scholar
  8. 8.
    Hardoon, D.R., Szedmak, S., Shawe-Taylor, J.: Canonical correlation analysis: an overview with application to learning methods. Neural Comput. 16, 2639–2664 (2004)CrossRefGoogle Scholar
  9. 9.
    Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1990)zbMATHGoogle Scholar
  10. 10.
    Huiskes, M.J., Lew, M.S.: The MIR Flickr retrieval evaluation. In: 2008 ACM International Conference on Multimedia Information Retrieval (ACM-MIR), pp. 39–43 (2008)Google Scholar
  11. 11.
    Jiang, Q.Y., Li, W.J.: Deep cross-modal hashing. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3232–3240 (2017)Google Scholar
  12. 12.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: 2012 Advances in Neural Information Processing Systems (NIPS), pp. 1097–1105 (2012)Google Scholar
  13. 13.
    Kumar, S., Udupa, R.: Learning hash functions for cross-view similarity search. In: 2011 International Joint Conference on Artificial Intelligence (IJCAI), pp. 1360–1365 (2011)Google Scholar
  14. 14.
    Lin, G., Shen, C., Suter, D., van den Hengel, A.: A general two-step approach to learning-based hashing. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 2552–2559 (2013)Google Scholar
  15. 15.
    Lin, Z., Ding, G., Han, J., Wang, J.: Cross-view retrieval via probability-based semantics-preserving hashing. IEEE Trans. Cybern. 47, 4342–4355 (2017)CrossRefGoogle Scholar
  16. 16.
    Lin, Z., Ding, G., Hu, M., Wang, J.: Semantics-preserving hashing for cross-view retrieval. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3864–3872 (2015)Google Scholar
  17. 17.
    Liu, L., Lin, Z., Shao, L., Shen, F., Ding, G., Han, J.: Sequential discrete hashing for scalable cross-modality similarity retrieval. IEEE Trans. Image Process. 26, 107–118 (2017)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Long, M., Cao, Y., Wang, J., Yu, P.S.: Compositional correlation quantization for large-scale multimodal search. In: 2016 International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR), pp. 579–588 (2016)Google Scholar
  19. 19.
    Mandal, D., Biswas, S.: Label consistent matrix factorization based hashing for cross-modal retrieval. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 2901–2905 (2017)Google Scholar
  20. 20.
    Mandal, D., Chaudhury, K.N., Biswas, S.: Generalized semantic preserving hashing for n-label cross-modal retrieval. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2633–2641 (2017)Google Scholar
  21. 21.
    Paszke, A., et al.: Automatic differentiation in PyTorch. In: 2017 Advances in Neural Information Processing Systems Workshop (NIPS-W) (2017)Google Scholar
  22. 22.
    Shen, F., Zhou, X., Yang, Y., Song, J., Shen, H.T., Tao, D.: A fast optimization method for general binary code learning. IEEE Trans. Image Process. 25, 5610–5621 (2017)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Song, J., Yang, Y., Yang, Y., Huang, Z., Shen, H.T.: Inter-media hashing for large-scale retrieval from heterogeneous data sources. In: 2013 ACM SIGMOD International Conference on Management of Data (SIGMOD), pp. 785–796 (2013)Google Scholar
  24. 24.
    Wang, D., Gao, X., Wang, X., He, L.: Semantic topic multimodal hashing for cross-media retrieval. In: 2015 International Joint Conference on Artificial Intelligence (IJCAI), pp. 3890–3896 (2015)Google Scholar
  25. 25.
    Wang, J., Kumar, S., Chang, S.F.: Sequential projection learning for hashing with compact codes. In: 2010 International Conference on Machine Learning (ICML), pp. 1127–1134 (2010)Google Scholar
  26. 26.
    Wu, B., Yang, Q., Zheng, W.S., Wang, Y., Wang, J.: Quantized correlation hashing for fast cross-modal search. In: 2015 International Joint Conference on Artificial Intelligence (IJCAI), pp. 3946–3952 (2015)Google Scholar
  27. 27.
    Xia, R., Pan, Y., Lai, H., Liu, C., Yan, S.: Supervised hashing for image retrieval via image representation learning. In: 2014 AAAI Conference on Artificial Intelligence (AAAI), pp. 2156–2162 (2014)Google Scholar
  28. 28.
    Xie, L., Shen, J., Han, J., Zhu, L., Shao, L.: Dynamic multi-view hashing for online image retrieval. In: 2017 International Joint Conference on Artificial Intelligence (IJCAI), pp. 3133–3139 (2017)Google Scholar
  29. 29.
    Xu, X., Shen, F., Yang, Y., Shen, H.T., Li, X.: Learning discriminative binary codes for large-scale cross-modal retrieval. IEEE Trans. Image Process. 26, 2494–2507 (2017)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Yang, E., Deng, C., Liu, W., Liu, X., Tao, D., Gao, X.: Pairwise relationship guided deep hashing for cross-modal retrieval. In: 2017 AAAI Conference on Artificial Intelligence (AAAI), pp. 1618–1625 (2017)Google Scholar
  31. 31.
    Zhang, D., Li, W.J.: Large-scale supervised multimodal hashing with semantic correlation maximization. In: 2014 AAAI Conference on Artificial Intelligence (AAAI), pp. 2177–2183 (2014)Google Scholar
  32. 32.
    Zhang, J., Peng, Y., Yuan, M.: Unsupervised generative adversarial cross-modal hashing. In: 2018 AAAI Conference on Artificial Intelligence (AAAI), pp. 539–546 (2018)Google Scholar
  33. 33.
    Zhang, R., Lin, L., Zhang, R., Zuo, W., Zhang, L.: Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Trans. Image Process. 24, 4766–4779 (2015)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Zhang, T., Wang, J.: Collaborative quantization for cross-modal similarity search. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2036–2045 (2016)Google Scholar
  35. 35.
    Zhou, J., Ding, G., Guo, Y.: Latent semantic sparse hashing for cross-modal similarity search. In: 2014 International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR), pp. 415–424 (2014)Google Scholar
  36. 36.
    Zhou, J., Ding, G., Guo, Y., Liu, Q., Dong, X.: Kernel-based supervised hashing for cross-view similarity search. In: 2014 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2014)Google Scholar
  37. 37.
    Zhu, H., Long, M., Wang, J., Cao, Y.: Deep hashing network for efficient similarity retrieval. In: 2016 AAAI Conference on Artificial Intelligence (AAAI), pp. 2415–2421 (2016)Google Scholar
  38. 38.
    Zhuang, B., Lin, G., Shen, C., Reid, I.: Fast training of triplet-based deep binary embedding networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5955–5964 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Indian Institute of ScienceBangaloreIndia
  2. 2.ETHZurichSwitzerland

Personalised recommendations