Discrete Sparse Hashing for Cross-Modal Similarity Search

  • Lu Wang
  • Chao Ma
  • Enmei Tu
  • Jie YangEmail author
  • Nikola Kasabov
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11304)


Cross-modal hashing approaches have achieved great success on cross-modal similarity search. However, most existing cross-modal hashing methods relax the discrete constraints to solve the hashing model and determine the weights of different modalities manually, which can significantly degrade the performance of retrieval. Besides, they are sensitive to noises because of the widely-utilized \(l_2\)-norm loss function. To address above problems, in this paper, a novel hashing method is proposed to efficiently learn unified binary codes, namely Discrete Sparse Hashing (DSH). In DSH model, unified hash codes are directly learned by discrete sparse coding in sharing low-dimensional latent space for different modalities, where the large quantization error is avoided and the learned codes are robust owing to the sparsity of binary codes. Moreover, the weights of different modalities are adaptively adjusted for training data. Extensive experiments on three databases demonstrate superior performance of DSH over most state-of-the-art methods.


Sharing low-dimensional space Discrete sparse coding Linear classification framework Unsupervised hashing Cross-modal retrieval 



This research is partly supported by NSFC, China (No: 61572315, 6151101179) and 973 Plan, China (No. 2015CB856004).


  1. 1.
    Chua, T.S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: NUS-WIDE: a real-world web image database from national university of Singapore. In: ACM International Conference on Image and Video Retrieval, pp. 1–9 (2009)Google Scholar
  2. 2.
    Ding, G., Guo, Y., Zhou, J.: Collective matrix factorization hashing for multimodal data. In: Computer Vision and Pattern Recognition, pp. 2083–2090 (2014)Google Scholar
  3. 3.
    He, R., Zhang, M., Wang, L., Ji, Y., Yin, Q.: Cross-modal subspace learning via pairwise constraints. IEEE Trans. Image Process. 24(12), 5543–5556 (2015)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Jiang, Q.Y., Li, W.J.: Deep cross-modal hashing. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3270–3278 (2017)Google Scholar
  5. 5.
    Jin, Q., Grama, I., Kervrann, C., Liu, Q.: Nonlocal means and optimal weights for noise removal. SIAM J. Imaging Sci. 10(4), 1878–1920 (2017)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Kumar, S., Udupa, R.: Learning hash functions for cross-view similarity search. In: International Joint Conference on Artificial Intelligence, pp. 1360–1365 (2011)Google Scholar
  7. 7.
    Li, K., Qi, G., Ye, J., Hua, K.: Linear subspace ranking hashing for cross-modal retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 39(9), 1825–1838 (2017)CrossRefGoogle Scholar
  8. 8.
    Li, X., Hu, D., Nie, F.: Deep binary reconstruction for cross-modal hashing. In: Proceedings of the 2017 ACM on Multimedia Conference, pp. 1398–1406 (2017)Google Scholar
  9. 9.
    Lin, Z., Ding, G., Hu, M., Wang, J.: Semantics-preserving hashing for cross-view retrieval. In: Computer Vision and Pattern Recognition, pp. 3864–3872 (2015)Google Scholar
  10. 10.
    Long, M., Cao, Y., Wang, J., Yu, P.S.: Composite correlation quantization for efficient multimodal retrieval. Computer Science, pp. 579–588 (2016)Google Scholar
  11. 11.
    Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: a strategy employed by V1. Vis. Res. 37(23), 3311–3325 (1997)CrossRefGoogle Scholar
  12. 12.
    Rasiwasia, N., et al.: A new approach to cross-modal multimedia retrieval. In: International Conference on Multimedia, pp. 251–260 (2010)Google Scholar
  13. 13.
    Rastegari, M., Choi, J., Fakhraei, S., Daume Iii, H., Davis, L.S.: Predictable dual-view hashing. In: International Conference on International Conference on Machine Learning, pp. 1328–1336 (2013)Google Scholar
  14. 14.
    Schnemann, P.H.: A generalized solution of the orthogonal procrustes problem. Psychometrika 31(1), 1–10 (1966)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Shen, F., Shen, C., Liu, W., Shen, H.T.: Supervised discrete hashing. In: Computer Vision and Pattern Recognition, pp. 37–45 (2015)Google Scholar
  16. 16.
    Song, J., Yang, Y., Yang, Y., Huang, Z., Shen, H.T.: Inter-media hashing for large-scale retrieval from heterogeneous data sources. In: ACM SIGMOD International Conference on Management of Data, pp. 785–796 (2013)Google Scholar
  17. 17.
    Wang, D., Wang, Q., Gao, X.: Robust and flexible discrete hashing for cross-modal similarity search. IEEE Trans. Circuits Syst. Video Technol. 99, 1–1 (2017)Google Scholar
  18. 18.
    Wang, W., Ooi, B.C., Yang, X., Zhang, D., Zhuang, Y.: Effective multi-modal retrieval based on stacked auto-encoders. Proc. VLDB Endow. 7(8), 649–660 (2014)CrossRefGoogle Scholar
  19. 19.
    Wu, F., Jing, X.Y., You, X., Yue, D., Hu, R., Yang, J.Y.: Multi-view low-rank dictionary learning for image classification. Pattern Recognit. 50(C), 143–154 (2016)CrossRefGoogle Scholar
  20. 20.
    Wu, F., Jing, X.Y., Yue, D.: Multi-view Discriminant Dictionary Learning via Learning View-specific and Shared Structured Dictionaries for Image Classification. Kluwer Academic Publishers (2017)Google Scholar
  21. 21.
    Xu, X., Shen, F., Yang, Y., Shen, H.T., Li, X.: Learning discriminative binary codes for large-scale cross-modal retrieval. IEEE Trans. Image Process. 26(5), 2494–2507 (2017)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Zhang, D., Li, W.J.: Large-scale supervised multimodal hashing with semantic correlation maximization. In: Twenty-Eighth AAAI Conference on Artificial Intelligence, pp. 2177–2183 (2014)Google Scholar
  23. 23.
    Zhen, Y., Yeung, D.Y.: A probabilistic model for multimodal hash function learning. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 940–948 (2012)Google Scholar
  24. 24.
    Zhou, J., Ding, G., Guo, Y.: Latent semantic sparse hashing for cross-modal similarity search. In: Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 415–424 (2014)Google Scholar
  25. 25.
    Zhu, X., Huang, Z., Shen, H.T., Zhao, X.: Linear cross-modal hashing for efficient multimedia search. In: ACM Multimedia Conference, pp. 143–152 (2013)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Lu Wang
    • 1
  • Chao Ma
    • 1
  • Enmei Tu
    • 1
  • Jie Yang
    • 1
    Email author
  • Nikola Kasabov
    • 2
  1. 1.Institute of Image Processing and Pattern RecognitionShanghai Jiao Tong UniversityShanghaiChina
  2. 2.Knowledge Engineering and Discovery Research InstituteAuckland University of TechnologyAucklandNew Zealand

Personalised recommendations