Skip to main content
SpringerLink
Log in

Coarse Label Refined Knowledge Reasoning for Fine-Grained Visual Categorization

  • Conference paper
  • First Online:
Intelligence Science and Big Data Engineering (IScIDE 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11266))

  • 1755 Accesses

Abstract

Fine-grained visual categorization (FGVC) is a challenging task due to large intra-class variance and small inter-class variance, aiming at recognizing hundreds of subcategories which belong to the same basic-level category. Humans have the ability to acquire knowledge about the world and use the acquired knowledge to reason about entities. When identifying an image, humans first reason out candidate subcategories of the object based on their prior knowledge. Then discover the characteristics of the object, which can be used as assistance for comparisons between candidate subcategories. Finally humans can determine the ultimate prediction. Inspired by this behavior, we propose a Coarse Label Refined Knowledge Reasoning (CLRKR) approach for fine-grained visual categorization. Its main novelties and advantages are as follows: (1) Knowledge Reasoning: We first construct the knowledge graph based on subcategory-attribute correlations, then reason out candidate subcategories with corresponding probabilities based on the knowledge graph. (2) Coarse Label Refinement: We incorporate coarse classes, which are pre-defined groups of attributes, to get corresponding coarse labels. These coarse labels reveal the most characteristic features of the entity in the image, which help us to re-rank the sequence of candidate subcategories and acquire the foremost subcategory as the final prediction. Our CLRKR approach achieves the best performance according to the experimental results on the widely-used CUB-200-2011 dataset for fine-grained visual categorization comparing with the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Lin, T.-Y., RoyChowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1449–1457 (2015)

    Google Scholar 

  2. Zhang, Y., et al.: Weakly supervised fine-grained categorization with part-based image representation. IEEE Trans. Image Process. 25(4), 1713–1725 (2016)

    Article  MathSciNet  Google Scholar 

  3. Marino, K., Salakhutdinov, R., Gupta, A.: The more you know: using knowledge graphs for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

    Google Scholar 

  4. Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The caltech-UCSD birds-200-2011 dataset (2011)

    Google Scholar 

  5. Li, Y., Tarlow, D., Brockschmidt, M., Zemel, R.: Gated graph sequence neural networks. In: International Conference on Learning Representations (ICLR) (2016). http://arxiv.org/abs/1511.05493

  6. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  7. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)

  8. Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. In: ACM International Conference on Multimedia (ACM MM), pp. 675–678. ACM (2014)

    Google Scholar 

  9. Szegedy, C., et al.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)

    Google Scholar 

  10. Yan, Y., Ni, B., Yang, X.: Fine-grained recognition via attribute-guided attentive feature aggregation. In: ACM International Conference on Multimedia (ACM MM), pp. 1032–1040. ACM (2017)

    Google Scholar 

  11. Zhou, F., Lin, Y.: Fine-grained image classification by exploring bipartite-graph labels. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1124–1133 (2016)

    Google Scholar 

  12. Fu, J., Zheng, H., Mei, T.: Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

    Google Scholar 

  13. He, X., Peng, Y., Zhao, J.: Fine-grained discriminative localization via saliency-guided faster R-CNN. In: ACM International Conference on Multimedia (ACM MM), pp. 627–635. ACM (2017)

    Google Scholar 

  14. He, X., Peng, Y.: Weakly supervised learning of part selection model with spatial constraints for fine-grained image classification. In: AAAI Conference on Artificial Intelligence (AAAI), pp. 4075–4081 (2017)

    Google Scholar 

  15. Zhang, X., Xiong, H., Zhou, W., Tian, Q.: Fused one-vs-all features with semantic alignments for fine-grained visual categorization. IEEE Trans. Image Process. 25(2), 878–892 (2016)

    Article  MathSciNet  Google Scholar 

  16. Zhang, X., Xiong, H., Zhou, W., Lin, W., Tian, Q.: Picking deep filter responses for fine-grained image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1134–1142 (2016)

    Google Scholar 

  17. Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. In: Neural Information Processing Systems (NIPS), pp. 2017–2025 (2015)

    Google Scholar 

  18. Wang, D., Shen, Z., Shao, J., Zhang, W., Xue, X., Zhang, Z.: Multiple granularity descriptors for fine-grained categorization. In: International Conference on Computer Vision (ICCV), pp. 2399–2406 (2015)

    Google Scholar 

  19. Simon, M., Rodner, E.: Neural activation constellations: unsupervised part model discovery with convolutional networks. In: International Conference of Computer Vision (ICCV), pp. 1143–1151 (2015)

    Google Scholar 

  20. Xiao, T., Xu, Y., Yang, K., Zhang, J., Peng, Y., Zhang, Z.: The application of two-level attention models in deep convolutional neural network for fine-grained image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 842–850 (2015)

    Google Scholar 

  21. Xu, Z., Tao, D., Huang, S., Zhang, Y.: Friend or Foe: fine-grained categorization with weak supervision. IEEE Trans. Image Process. (TIP) 26(1), 135–146 (2017)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgment

This work was supported by the National Natural Science Foundation of China under Grant 61771025.

Author information

Authors and Affiliations

  1. Institute of Computer Science and Technology, Peking University, Beijing, China

    Xiangyu Zhao & Yuxin Peng

Authors
  1. Xiangyu Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuxin Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuxin Peng .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Yuxin Peng

  2. Shanghai Jiao Tong University, Shanghai, China

    Kai Yu

  3. Tsinghua University, Beijing, China

    Jiwen Lu

  4. Central China Normal University, Wuhan, China

    Xingpeng Jiang

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, X., Peng, Y. (2018). Coarse Label Refined Knowledge Reasoning for Fine-Grained Visual Categorization. In: Peng, Y., Yu, K., Lu, J., Jiang, X. (eds) Intelligence Science and Big Data Engineering. IScIDE 2018. Lecture Notes in Computer Science(), vol 11266. Springer, Cham. https://doi.org/10.1007/978-3-030-02698-1_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-02698-1_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-02697-4

  • Online ISBN: 978-3-030-02698-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation