Fine-Grained Color Sketch-Based Image Retrieval

  • Yu XiaEmail author
  • Shuangbu Wang
  • Yanran Li
  • Lihua You
  • Xiaosong Yang
  • Jian Jun Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11542)


We propose a novel fine-grained color sketch-based image retrieval (CSBIR) approach. The CSBIR problem is investigated for the first time using deep learning networks, in which deep features are used to represent color sketches and images. A novel ranking method considering both shape matching and color matching is also proposed. In addition, we build a CSBIR dataset with color sketches and images to train and test our method. The results show that our method has better retrieval performance.


Color sketch Image retrieval Deep learning Triplet network 



This research is supported by the PDE-GIR project which has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 778035. Yanran Li has received research grands from the South West Creative Technology Network.


  1. 1.
    Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: Sketch-based image retrieval: benchmark and bag-of-features descriptors. IEEE Trans. Vis. Comput. Graph. 17(11), 1624–1636 (2011)CrossRefGoogle Scholar
  2. 2.
    Hu, R., Collomosse, J.: A performance evaluation of gradient field hog descriptor for sketch based image retrieval. Comput. Vis. Image Underst. 117(7), 790–806 (2013)CrossRefGoogle Scholar
  3. 3.
    Kiran Yelamarthi, S., Krishna Reddy, S., Mishra, A., Mittal, A.: A zero-shot framework for sketch based image retrieval. In: ECCV, pp. 300–317 (2018)Google Scholar
  4. 4.
    Huang, F., Jin, C., Zhang, Y., Weng, K., Zhang, T., Fan, W.: Sketch-based image retrieval with deep visual semantic descriptor. Pattern Recogn. 76, 537–548 (2018)CrossRefGoogle Scholar
  5. 5.
    Li, Y., Hospedales, T.M., Song, Y.Z., Gong, S.: Fine-grained sketch-based image retrieval by matching deformable part models. In: British Machine Vision Conference (2014)Google Scholar
  6. 6.
    Yu, Q., Liu, F., Song, Y.Z., Xiang, T., Hospedales, T.M., Loy, C.C.: Sketch me that shoe. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 799–807 (2016)Google Scholar
  7. 7.
    Sangkloy, P., Burnell, N., Ham, C., Hays, J.: The sketchy database: learning to retrieve badly drawn bunnies. ACM Trans. Graph. (TOG) 35(4), 119 (2016)CrossRefGoogle Scholar
  8. 8.
    Song, J., Yu, Q., Song, Y.Z., Xiang, T., Hospedales, T.M.: Deep spatial-semantic attention for fine-grained sketch-based image retrieval. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5551–5560 (2017)Google Scholar
  9. 9.
    Bui, T., Collomosse, J.: Scalable sketch-based image retrieval using color gradient features. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1–8 (2015)Google Scholar
  10. 10.
    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)Google Scholar
  11. 11.
    Le Cam, L., Yang, G.L.: Asymptotics in Statistics: Some Basic Concepts. Springer, New York (2012). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yu Xia
    • 1
    Email author
  • Shuangbu Wang
    • 1
  • Yanran Li
    • 1
  • Lihua You
    • 1
  • Xiaosong Yang
    • 1
  • Jian Jun Zhang
    • 1
  1. 1.National Centre for Computer AnimationBournemouth UniversityBournemouthUK

Personalised recommendations