Skip to main content
SpringerLink
Log in

Retrieval by Classification: Discriminative Binary Embedding for Sketch-Based Image Retrieval

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11859))

Included in the following conference series:

Abstract

Sketch-based image retrieval (SBIR) intends to use free-hand sketch drawings as query to retrieve correlated real-world images from database. Hashing based methods gradually become the mainstream approaches in SBIR with its low memory usage and high query speed. Existing hashing based methods are incapable of guiding hash codes to preserve inter-class relationship and improving object recognition ability of hash functions simultaneously, which limits the higher performance. Hence, we propose Discriminative Binary Embedding (DBE), a novel algorithm of considering inter-class relationship and object recognition ability in a joint manner by treating retrieval as classification. Specifically, we apply NLP methods to encode category labels as binary embedding and then build classifiers for images and sketches, so as to obtain hash codes of instances based on binary embedding of predicted labels. Experimental results on two benchmarks show that DBE outperforms several state-of-the-arts.

Y. Shi—Student Author.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://code.google.com/archive/p/word2vec/.

References

  1. Cao, Y., Wang, H., Wang, C., Li, Z., Zhang, L., Zhang, L.: MindFinder: interactive sketch-based image search on millions of images. In: Proceedings of the 18th ACM International Conference on Multimedia, pp. 1605–1608. ACM (2010)

    Google Scholar 

  2. Charikar, M.S.: Similarity estimation techniques from rounding algorithms. In: Proceedings of the Thiry-Fourth Annual ACM Symposium on Theory of Computing, pp. 380–388. ACM (2002)

    Google Scholar 

  3. Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. arXiv preprint arXiv:1405.3531 (2014)

  4. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  5. Ding, G., Guo, Y., Zhou, J.: Collective matrix factorization hashing for multimodal data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2075–2082 (2014)

    Google Scholar 

  6. Eitz, M., Hays, J., Alexa, M.: How do humans sketch objects? ACM Trans. Graph. 31(4), 44–1 (2012)

    Google Scholar 

  7. Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: An evaluation of descriptors for large-scale image retrieval from sketched feature lines. Comput. Graph. 34(5), 482–498 (2010)

    Article  Google Scholar 

  8. Geladi, P., Kowalski, B.R.: Partial least-squares regression: a tutorial. Anal. Chim. Acta 185, 1–17 (1986)

    Article  Google Scholar 

  9. Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  10. Hardoon, D.R., Szedmak, S., Shawe-Taylor, J.: Canonical correlation analysis: an overview with application to learning methods. Neural Comput. 16(12), 2639–2664 (2004)

    Article  Google Scholar 

  11. Jiang, Q.Y., Li, W.J.: Asymmetric deep supervised hashing. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  12. Lin, Z., Ding, G., Hu, M., Wang, J.: Semantics-preserving hashing for cross-view retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3864–3872 (2015)

    Google Scholar 

  13. Liu, L., Shen, F., Shen, Y., Liu, X., Shao, L.: Deep sketch hashing: fast free-hand sketch-based image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2862–2871 (2017)

    Google Scholar 

  14. Liu, L., Qi, H.: Discriminative cross-view binary representation learning. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1736–1744. IEEE (2018)

    Google Scholar 

  15. Long, M., Cao, Y., Wang, J., Yu, P.S.: Composite correlation quantization for efficient multimodal retrieval. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 579–588. ACM (2016)

    Google Scholar 

  16. Maaten, L.V.D., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov), 2579–2605 (2008)

    MATH  Google Scholar 

  17. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)

    Google Scholar 

  18. Parui, S., Mittal, A.: Similarity-invariant sketch-based image retrieval in large databases. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 398–414. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_26

    Chapter  Google Scholar 

  19. Peng, Y., Huang, X., Zhao, Y.: An overview of cross-media retrieval: concepts, methodologies, benchmarks, and challenges. IEEE Trans. Circuits Syst. Video Technol. 28(9), 2372–2385 (2018)

    Article  Google Scholar 

  20. Qi, Y., Song, Y.Z., Zhang, H., Liu, J.: Sketch-based image retrieval via Siamese convolutional neural network. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 2460–2464. IEEE (2016)

    Google Scholar 

  21. Ravichandran, D., Pantel, P., Hovy, E.: Randomized algorithms and NLP: using locality sensitive hash functions for high speed noun clustering. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL 2005), pp. 622–629 (2005)

    Google Scholar 

  22. Saavedra, J.M., Barrios, J.M., Orand, S.: Sketch based image retrieval using learned keyshapes (LKS). In: BMVC, vol. 1, p. 7 (2015)

    Google Scholar 

  23. Sangkloy, P., Burnell, N., Ham, C., Hays, J.: The sketchy database: learning to retrieve badly drawn bunnies. ACM Trans. Graph. (TOG) 35(4), 119 (2016)

    Article  Google Scholar 

  24. Tseng, K.Y., Lin, Y.L., Chen, Y.H., Hsu, W.H.: Sketch-based image retrieval on mobile devices using compact hash bits. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 913–916. ACM (2012)

    Google Scholar 

  25. Wang, D., Gao, X., Wang, X., He, L.: Semantic topic multimodal hashing for cross-media retrieval. In: Twenty-Fourth International Joint Conference on Artificial Intelligence (2015)

    Google Scholar 

  26. Wang, D., Gao, X., Wang, X., He, L., Yuan, B.: Multimodal discriminative binary embedding for large-scale cross-modal retrieval. IEEE Trans. Image Process. 25(10), 4540–4554 (2016)

    Article  MathSciNet  Google Scholar 

  27. Wang, J., Zhang, T., Sebe, N., Shen, H.T., et al.: A survey on learning to hash. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 769–790 (2018)

    Article  Google Scholar 

  28. Wang, K., He, R., Wang, W., Wang, L., Tan, T.: Learning coupled feature spaces for cross-modal matching. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2088–2095 (2013)

    Google Scholar 

  29. Xu, X., Shen, F., Yang, Y., Shen, H.T.: Discriminant cross-modal hashing. In: Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval, pp. 305–308. ACM (2016)

    Google Scholar 

  30. Yu, Q., Liu, F., Song, Y.Z., Xiang, T., Hospedales, T.M., Loy, C.C.: Sketch me that shoe. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 799–807 (2016)

    Google Scholar 

  31. Zhang, D., Li, W.J.: Large-scale supervised multimodal hashing with semantic correlation maximization. In: Twenty-Eighth AAAI Conference on Artificial Intelligence (2014)

    Google Scholar 

  32. Zhang, J., et al.: Generative domain-migration hashing for sketch-to-image retrieval. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 297–314 (2018)

    Chapter  Google Scholar 

  33. Zhou, R., Chen, L., Zhang, L.: Sketch-based image retrieval on a large scale database. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 973–976. ACM (2012)

    Google Scholar 

  34. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

    Google Scholar 

Download references

Acknowledgments

This work was supported partially by the Key Science and Technology of Shenzhen (JCYJ20180305180637611), the Shenzhen Research Council (JCYJ20180305180804836 and JSGG20180507182030600), the Key Science and Technology Innovation Program of Hubei Province (2017AAA017), the Natural Science Foundation of Hubei Province (2018CFB691), the Special Projects for Technology Innovation of Hubei Province (2018ACA135), the National Natural Science Foundation of China (61571205 and 61772220) and the key research and development program of China (2016YFE0121200).

Author information

Authors and Affiliations

  1. School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, China

    Yufeng Shi, Xinge You, Wenjie Wang, Qinmu Peng & Shuo Wang

  2. Research Institute of Huazhong University of Science and Technology in Shenzhen, Shenzhen, China

    Xinge You

  3. Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China

    Feng Zheng

Authors
  1. Yufeng Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinge You
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenjie Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Feng Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qinmu Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shuo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinge You .

Editor information

Editors and Affiliations

  1. School of EECS, Peking University, Beijing, China

    Zhouchen Lin

  2. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Liang Wang

  3. Nanjing University of Science and Technology, Nanjing, China

    Jian Yang

  4. Xidian University, Xi’an, China

    Guangming Shi

  5. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Tieniu Tan

  6. Institute of Artificial Intelligence, Xi’an Jiaotong University, Xi’an, China

    Nanning Zheng

  7. Chinese Academy of Sciences, Beijing, China

    Xilin Chen

  8. Northwestern Polytechnical University, Xi’an, China

    Yanning Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shi, Y., You, X., Wang, W., Zheng, F., Peng, Q., Wang, S. (2019). Retrieval by Classification: Discriminative Binary Embedding for Sketch-Based Image Retrieval. In: Lin, Z., et al. Pattern Recognition and Computer Vision. PRCV 2019. Lecture Notes in Computer Science(), vol 11859. Springer, Cham. https://doi.org/10.1007/978-3-030-31726-3_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-31726-3_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-31725-6

  • Online ISBN: 978-3-030-31726-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation