Deep Supervised Hashing with Spherical Embedding

  • Stanislav PidhorskyiEmail author
  • Quinn Jones
  • Saeid Motiian
  • Donald Adjeroh
  • Gianfranco Doretto
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11364)


Deep hashing approaches are widely applied to approximate nearest neighbor search for large-scale image retrieval. We propose Spherical Deep Supervised Hashing (SDSH), a new supervised deep hashing approach to learn compact binary codes. The goal of SDSH is to go beyond learning similarity preserving codes, by encouraging them to also be balanced and to maximize the mean average precision. This is enabled by advocating the use of a different relaxation method, allowing the learning of a spherical embedding, which overcomes the challenge of maintaining the learning problem well-posed without the need to add extra binarizing priors. This allows the formulation of a general triplet loss framework, with the introduction of the spring loss for learning balanced codes, and of the ability to learn an embedding quantization that maximizes the mean average precision. Extensive experiments demonstrate that the approach compares favorably with the state-of-the-art while providing significant performance increase at more compact code sizes.



This material is based upon work supported in part by the Center for Identification Technology Research and the National Science Foundation under Grant No. 1650474.

Supplementary material

484519_1_En_26_MOESM1_ESM.pdf (143 kb)
Supplementary material 1 (pdf 142 KB)


  1. 1.
    Andoni, A., Indyk, P.: Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In: IEEE FOCS, pp. 459–468 (2006)Google Scholar
  2. 2.
    Cao, Y., Long, M., Liu, B., Wang, J., KLiss, M.: Deep cauchy hashing for hamming space retrieval. In: CVPR, pp. 1229–1237 (2018)Google Scholar
  3. 3.
    Cao, Y., Long, M., Wang, J., Liu, S.: Deep visual-semantic quantization for efficient image retrieval. In: CVPR (2017)Google Scholar
  4. 4.
    Cao, Y., Long, M., Wang, J., Zhu, H., Wen, Q.: Deep quantization network for efficient image retrieval. In: AAAI, pp. 3457–3463 (2016)Google Scholar
  5. 5.
    Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. In: BMVC (2014)Google Scholar
  6. 6.
    Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. arXiv preprint arXiv:1405.3531 (2014)
  7. 7.
    Chen, Z., Yuan, X., Lu, J., Tian, Q., Zhou, J.: Deep hashing via discrepancy minimization. In: CVPR, pp. 6838–6847 (2018)Google Scholar
  8. 8.
    Chua, T.S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: NUS-WIDE: a real-world web image database from National University of Singapore. In: ACM CIVR, pp. 48:1–48:9 (2009)Google Scholar
  9. 9.
    Erin Liong, V., Lu, J., Wang, G., Moulin, P., Zhou, J.: Deep hashing for compact binary codes learning. In: CVPR, pp. 2475–2483 (2015)Google Scholar
  10. 10.
    Ge, T., He, K., Ke, Q., Sun, J.: Optimized product quantization. IEEE TPAMI 36(4), 744–755 (2014)CrossRefGoogle Scholar
  11. 11.
    Ghasedi Dizaji, K., Zheng, F., Sadoughi, N., Yang, Y., Deng, C., Huang, H.: Unsupervised deep generative adversarial hashing network. In: CVPR, pp. 3664–3673 (2018)Google Scholar
  12. 12.
    Gionis, A., Indyk, P., Motwani, R., et al.: Similarity search in high dimensions via hashing. In: VLDB, vol. 99, 518–529 (1999)Google Scholar
  13. 13.
    Gong, Y., Lazebnik, S., Gordo, A., Perronnin, F.: Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE TPAMI 35(12), 2916–2929 (2013)CrossRefGoogle Scholar
  14. 14.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)Google Scholar
  15. 15.
    He, K., Wen, F., Sun, J.: K-means hashing: an affinity-preserving quantization method for learning binary compact codes. In: CVPR, pp. 2938–2945 (2013)Google Scholar
  16. 16.
    Heo, J., Lee, Y., He, J., Chang, S., Yoon, S.: Spherical hashing: binary code embedding with hyperspheres. IEEE Trans. Pattern Anal. Mach. Intell. 37(11), 2304–2316 (2015). Scholar
  17. 17.
    Krizhevsky, A.: Learning multiple layers of features from tiny images (2009)Google Scholar
  18. 18.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: NIPS, pp. 1097–1105 (2012)Google Scholar
  19. 19.
    Kulis, B., Darrell, T.: Learning to hash with binary reconstructive embeddings. In: NIPS, pp. 1042–1050 (2009)Google Scholar
  20. 20.
    Kulis, B., Grauman, K.: Kernelized locality-sensitive hashing for scalable image search. In: ICCV, pp. 2130–2137. IEEE (2009)Google Scholar
  21. 21.
    Lai, H., Pan, Y., Liu, Y., Yan, S.: Simultaneous feature learning and hash coding with deep neural networks. In: CVPR, pp. 3270–3278 (2015)Google Scholar
  22. 22.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  23. 23.
    Li, W.J., Wang, S., Kang, W.C.: Feature learning based deep supervised hashing with pairwise labels. arXiv preprint arXiv:1511.03855 (2015)
  24. 24.
    Liu, H., Wang, R., Shan, S., Chen, X.: Deep supervised hashing for fast image retrieval. In: CVPR, pp. 2064–2072 (2016)Google Scholar
  25. 25.
    Liu, W., Mu, C., Kumar, S., Chang, S.F.: Discrete graph hashing. In: NIPS, pp. 3419–3427 (2014)Google Scholar
  26. 26.
    Liu, W., Wang, J., Ji, R., Jiang, Y.G., Chang, S.F.: Supervised hashing with kernels. In: CVPR, pp. 2074–2081. IEEE (2012)Google Scholar
  27. 27.
    Liu, W., Wang, J., Kumar, S., Chang, S.F.: Hashing with graphs. In: ICML, pp. 1–8 (2011)Google Scholar
  28. 28.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)CrossRefGoogle Scholar
  29. 29.
    Mishkin, D., Matas, J.L.: All you need is a good init. CoRR abs/1511.06422 (2015)Google Scholar
  30. 30.
    Norouzi, M., Blei, D.M., Salakhutdinov, R.R.: Hamming distance metric learning. In: NIPS (2012)Google Scholar
  31. 31.
    Norouzi, M., Blei, D.M.: Minimal loss hashing for compact binary codes. In: ICML, pp. 353–360 (2011)Google Scholar
  32. 32.
    Ozols, M.: How to generate a random unitary matrix (2009)Google Scholar
  33. 33.
    Sablayrolles, A., Douze, M., Usunier, N., Jégou, H.: How should we evaluate supervised hashing? In: ICASSP, pp. 1732–1736 (2017)Google Scholar
  34. 34.
    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: CVPR, pp. 815–823 (2015)Google Scholar
  35. 35.
    Schwartz, R.E.: The five-electron case of Thomson’s problem. Exp. Math. 22(2), 157–186 (2013)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Strecha, C., Bronstein, A., Bronstein, M., Fua, P.: LDAHash: improved matching with smaller descriptors. IEEE TPAMI 34(1), 66–78 (2012)CrossRefGoogle Scholar
  37. 37.
    Szegedy, C., Toshev, A., Erhan, D.: Deep neural networks for object detection. In: NIPS, pp. 2553–2561 (2013)Google Scholar
  38. 38.
    Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: CVPR, pp. 1701–1708 (2014)Google Scholar
  39. 39.
    Wang, J., Kumar, S., Chang, S.F.: Semi-supervised hashing for scalable image retrieval. In: CVPR, pp. 3424–3431 (2010)Google Scholar
  40. 40.
    Wang, J., Zhang, T., Sebe, N., Shen, H.T., et al.: A survey on learning to hash. IEEE TPAMI 40(4), 769–790 (2018)CrossRefGoogle Scholar
  41. 41.
    Wang, N., Yeung, D.Y.: Learning a deep compact image representation for visual tracking. In: NIPS, pp. 809–817 (2013)Google Scholar
  42. 42.
    Wang, X., Shi, Y., Kitani, K.M.: Deep supervised hashing with triplet labels. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10111, pp. 70–84. Springer, Cham (2017). Scholar
  43. 43.
    Wang, X., Zhang, T., Qi, G.J., Tang, J., Wang, J.: Supervised quantization for similarity search. In: CVPR, pp. 2018–2026 (2016)Google Scholar
  44. 44.
    Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. In: NIPS, pp. 1753–1760 (2009)Google Scholar
  45. 45.
    Xia, R., Pan, Y., Lai, H., Liu, C., Yan, S.: Supervised hashing for image retrieval via image representation learning. In: AAAI, vol. 1, pp. 2156–2162 (2014)Google Scholar
  46. 46.
    Zhang, P., Zhang, W., Li, W.J., Guo, M.: Supervised hashing with latent factor models. In: ACM SIGIR, pp. 173–182 (2014)Google Scholar
  47. 47.
    Zhang, R., Lin, L., Zhang, R., Zuo, W., Zhang, L.: Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE TIP 24(12), 4766–4779 (2015)MathSciNetzbMATHGoogle Scholar
  48. 48.
    Zhang, T., Du, C., Wang, J.: Composite quantization for approximate nearest neighbor search. In: ICML, no. 2, pp. 838–846 (2014)Google Scholar
  49. 49.
    Zhao, F., Huang, Y., Wang, L., Tan, T.: Deep semantic ranking based hashing for multi-label image retrieval. In: CVPR, pp. 1556–1564 (2015)Google Scholar
  50. 50.
    Zhu, H., Long, M., Wang, J., Cao, Y.: Deep hashing network for efficient similarity retrieval. In: AAAI, pp. 2415–2421 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Stanislav Pidhorskyi
    • 1
    Email author
  • Quinn Jones
    • 1
  • Saeid Motiian
    • 2
  • Donald Adjeroh
    • 1
  • Gianfranco Doretto
    • 1
  1. 1.Lane Department of Computer Science and Electrical EngineeringWest Virginia UniversityMorgantownUSA
  2. 2.Adobe Applied ResearchSan FranciscoUSA

Personalised recommendations