Product Quantization Network for Fast Image Retrieval

  • Tan YuEmail author
  • Junsong Yuan
  • Chen Fang
  • Hailin Jin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11205)


Product quantization has been widely used in fast image retrieval due to its effectiveness of coding high-dimensional visual features. By extending the hard assignment to soft assignment, we make it feasible to incorporate the product quantization as a layer of a convolutional neural network and propose our product quantization network. Meanwhile, we come up with a novel asymmetric triplet loss, which effectively boosts the retrieval accuracy of the proposed product quantization network based on asymmetric similarity. Through the proposed product quantization network, we can obtain a discriminative and compact image representation in an end-to-end manner, which further enables a fast and accurate image retrieval. Comprehensive experiments conducted on public benchmark datasets demonstrate the state-of-the-art performance of the proposed product quantization network.



This work is supported in part by start-up grants of University at Buffalo, Computer Science and Engineering Department. This work is supported in part by Singapore Ministry of Education Academic Research Fund Tier 2 MOE2015-T2-2-114. This research was carried out at the ROSE Lab of Nanyang Technological University, Singapore. The ROSE Lab is supported by the National Research Foundation, Prime Ministers Office, Singapore. We gratefully acknowledge the support of NVAITC (NVIDIA AI Technology Centre) for their donation for our research.


  1. 1.
    Babenko, A., Lempitsky, V.: Additive quantization for extreme vector compression. In: CVPR, pp. 931–938 (2014)Google Scholar
  2. 2.
    Babenko, A., Lempitsky, V.: Aggregating local deep features for image retrieval. In: ICCV, pp. 1269–1277 (2015)Google Scholar
  3. 3.
    Babenko, A., Slesarev, A., Chigorin, A., Lempitsky, V.: Neural codes for image retrieval. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 584–599. Springer, Cham (2014). Scholar
  4. 4.
    Bai, S., Bai, X., Tian, Q., Latecki, L.J.: Regularized diffusion process on bidirectional context for object retrieval. TPAMI (2018)Google Scholar
  5. 5.
    Bai, S., Zhou, Z., Wang, J., Bai, X., Latecki, L.J., Tian, Q.: Ensemble diffusion for retrievalGoogle Scholar
  6. 6.
    Cao, Y., Long, M., Wang, J., Zhu, H., Wen, Q.: Deep quantization network for efficient image retrieval. In: AAAI (2016)Google Scholar
  7. 7.
    Charikar, M.S.: Similarity estimation techniques from rounding algorithms. In: Proceedings of the Thiry-Fourth Annual ACM Symposium on Theory of Computing, pp. 380–388 (2002)Google Scholar
  8. 8.
    Chua, T.S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: NUS-wide: a real-world web image database from national university of Singapore. In: Proceedings of the ACM International Conference on Image and Video Retrieval, p. 48 (2009)Google Scholar
  9. 9.
    Ge, T., He, K., Ke, Q., Sun, J.: Optimized product quantization for approximate nearest neighbor search. In: CVPR, pp. 2946–2953. IEEE (2013)Google Scholar
  10. 10.
    Gong, Y., Lazebnik, S., Gordo, A., Perronnin, F.: Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE T-PAMI 35(12), 2916–2929 (2013)CrossRefGoogle Scholar
  11. 11.
    Gordo, A., Almazán, J., Revaud, J., Larlus, D.: Deep image retrieval: learning global representations for image search. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 241–257. Springer, Cham (2016). Scholar
  12. 12.
    Hong, W., Meng, J., Yuan, J.: Distributed composite quantization. In: AAAI (2018)Google Scholar
  13. 13.
    Hong, W., Meng, J., Yuan, J.: Tensorized projection for high-dimensional binary embedding. In: AAAI (2018)Google Scholar
  14. 14.
    Hong, W., Yuan, J.: Fried binary embedding: from high-dimensional visual features to high-dimensional binary codes. IEEE Trans. Image Process. 27(10) (2018)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Hong, W., Yuan, J., Bhattacharjee, S.D.: Fried binary embedding for high-dimensional visual features. CVPR 11, 18 (2017)Google Scholar
  16. 16.
    Jain, H., Zepeda, J., Perez, P., Gribonval, R.: Subic: a supervised, structured binary code for image search. In: ICCV, pp. 833–842 (2017)Google Scholar
  17. 17.
    Jegou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE T-PAMI 33(1), 117–128 (2011)CrossRefGoogle Scholar
  18. 18.
    Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: CVPR, pp. 3304–3311 (2010)Google Scholar
  19. 19.
    Jiang, Q.Y., Li, W.J.: Asymmetric deep supervised hashing. In: AAAI (2018)Google Scholar
  20. 20.
    Klein, B., Wolf, L.: In defense of product quantization. arXiv preprint arXiv:1711.08589 (2017)
  21. 21.
    Krizhevsky, A.: Learning multiple layers of features from tiny images (2009)Google Scholar
  22. 22.
    Lai, H., Pan, Y., Liu, Y., Yan, S.: Simultaneous feature learning and hash coding with deep neural networks. arXiv preprint arXiv:1504.03410 (2015)
  23. 23.
    Li, Q., Sun, Z., He, R., Tan, T.: Deep supervised discrete hashing. In: NIPS, pp. 2479–2488 (2017)Google Scholar
  24. 24.
    Li, W.J., Wang, S., Kang, W.C.: Feature learning based deep supervised hashing with pairwise labels. arXiv preprint arXiv:1511.03855 (2015)
  25. 25.
    Liu, H., Wang, R., Shan, S., Chen, X.: Deep supervised hashing for fast image retrieval. In: CVPR, pp. 2064–2072 (2016)Google Scholar
  26. 26.
    Liu, W., Wang, J., Ji, R., Jiang, Y.G., Chang, S.F.: Supervised hashing with kernels. In: CVPR, pp. 2074–2081 (2012)Google Scholar
  27. 27.
    Martinez, J., Clement, J., Hoos, H.H., Little, J.J.: Revisiting additive quantization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 137–153. Springer, Cham (2016). Scholar
  28. 28.
    Ng, J.Y.H., Yang, F., Davis, L.S.: Exploiting local features from deep networks for image retrieval. arXiv preprint arXiv:1504.05133 (2015)
  29. 29.
    Norouzi, M., Fleet, D.J.: Cartesian k-means. In: CVPR, pp. 3017–3024 (2013)Google Scholar
  30. 30.
    Perronnin, F., Liu, Y., Sánchez, J., Poirier, H.: Large-scale image retrieval with compressed fisher vectors. In: CVPR, pp. 3384–3391 (2010)Google Scholar
  31. 31.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: CVPR, pp. 1–8 (2007)Google Scholar
  32. 32.
    Salakhutdinov, R., Hinton, G.: Semantic hashing. RBM 500(3), 500 (2007)Google Scholar
  33. 33.
    Shen, F., Shen, C., Liu, W., Shen, H.T.: Supervised discrete hashing. IEEE T-PAMI 35(12), 2916–2929 (2013)CrossRefGoogle Scholar
  34. 34.
    Wang, X., Shi, Y., Kitani, K.M.: Deep supervised hashing with triplet labels. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10111, pp. 70–84. Springer, Cham (2017). Scholar
  35. 35.
    Wang, X., Zhang, T., Qi, G.J., Tang, J., Wang, J.: Supervised quantization for similarity search. In: CVPR, pp. 2018–2026 (2016)Google Scholar
  36. 36.
    Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. In: NIPS, pp. 1753–1760 (2009)Google Scholar
  37. 37.
    Xia, R., Pan, Y., Lai, H., Liu, C., Yan, S.: Supervised hashing for image retrieval via image representation learning. In: AAAI, pp. 2156–2162. AAAI Press (2014)Google Scholar
  38. 38.
    Yu, T., Meng, J., Yuan, J.: Is my object in this video? reconstruction-based object search in videos. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 4551–4557. AAAI Press (2017)Google Scholar
  39. 39.
    Yu, T., Wang, Z., Yuan, J.: Compressive quantization for fast object instance search in videos. In: ICCV, pp. 833–842 (2017)Google Scholar
  40. 40.
    Yu, T., Wu, Y., Bhattacharjee, S.D., Yuan, J.: Efficient object instance search using fuzzy objects matching. In: AAAI (2017)Google Scholar
  41. 41.
    Yu, T., Wu, Y., Yuan, J.: Hope: hierarchical object prototype encoding for efficient object instance search in videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2424–2433 (2017)Google Scholar
  42. 42.
    Zhang, R., Lin, L., Zhang, R., Zuo, W., Zhang, L.: Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE TIP 24(12), 4766–4779 (2015)MathSciNetGoogle Scholar
  43. 43.
    Zhang, T., Du, C., Wang, J.: Composite quantization for approximate nearest neighbor search. In: ICML, no. 2, pp. 838–846 (2014)Google Scholar
  44. 44.
    Zhang, Z., Chen, Y., Saligrama, V.: Efficient training of very deep neural networks for supervised hashing. In: CVPR, pp. 1487–1495 (2016)Google Scholar
  45. 45.
    Zhao, F., Huang, Y., Wang, L., Tan, T.: Deep semantic ranking based hashing for multi-label image retrieval. In: CVPR, pp. 1556–1564 (2015)Google Scholar
  46. 46.
    Zhu, H., Long, M., Wang, J., Cao, Y.: Deep hashing network for efficient similarity retrieval. In: AAAI (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Nanyang Technological UniversitySingaporeSingapore
  2. 2.State University of New York at BuffaloBuffaloUSA
  3. 3.Adobe ResearchSan JoseUSA

Personalised recommendations