Fine-Grained Recognition of Vegetable Images Based on Multi-scale Convolution Neural Network

  • Xiu-Hong Yang
  • Ji-Xiang DuEmail author
  • Hong-Bo Zhang
  • Wen-Tao Fan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10955)


In recent years, deep learning has been widely used in various computer vision tasks. Because the task of solving vegetable pictures are different in the local critical areas, and solving the classification of vegetable categories to meet the needs of users has become an urgent problem to be solved. In this paper, we propose fine-grained image recognition based on Vegetable Dataset, and uses multi-scale iteration to extract critical area characteristics, which the learning at each scale consists of a classification subnetwork and the critical area. In addition, the multi-scale neural network is optimized by two loss functions, to learn accurate critical area and fine-grained feature. Finally, we further prove its scalability and effectiveness in comparing different datasets and different training methods, we get satisfactory results on Vegetable Dataset.


Deep learning Multi-scale Classification and ranking Critical area 



This paper is supported by the Grant of the National Science Foundation of China (No. 61673186, 61502182, 61502183).


  1. 1.
    Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 dataset. Technical report CNS-TR-2011-001, California Institute of Technology, USA (2011)Google Scholar
  2. 2.
    Aditya, K., Nityananda, J., Yao, B., Li, F.F.: Novel dataset for fine-grained image categorization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition on First Workshop on Fine-Grained Visual Categorization. IEEE, Springs, USA (2011)Google Scholar
  3. 3.
    Nilsback, M.E., Zisserman, A.: Automated flower classification over a large number of classes. In: Proceedings of the 6th Indian Conference on Computer Vision, Graphics and Image Processing, pp. 722−729. IEEE, Bhubaneswar, India (2008)Google Scholar
  4. 4.
    Krause, J., Stark, M., Deng, J., Li, F.F.: 3D object representations for fine-grained categorization. In: Proceedings of the 4th IEEE Workshop on 3D Representation and Recognition. IEEE International Conference on Computer Vision, pp. 554555. IEEE, Sydney, Australia (2013)Google Scholar
  5. 5.
    Maji, S., Kannala, J., Rahtu, E., Blaschko, M., Vedaldi, A.: Fine-grained visual classification of aircraft, 21 June 2013.
  6. 6.
    Huang, S., Xu, Z., Tao, D., Zhang, Y.: Part-stacked CNN for fine-grained visual categorization. In: CVPR, pp. 1173–1182 (2016)Google Scholar
  7. 7.
    Lin, D., Shen, X., Lu, C., Jia, J.: Deep LAC: deep localization, alignment and classification for fine-grained recognition. In: CVPR, pp. 1666–1674 (2015)Google Scholar
  8. 8.
    Parkhi, O.M., Vedaldi, A., Jawajar, C., Zisserman, A.: The truth about cats and dogs. In: ICCV, pp. 1427–1434 (2011)Google Scholar
  9. 9.
    Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., Perona, P.: Caltech-UCSD birds 200. Technical report CNS-TR-2010-001, California Institute of Technology (2010)Google Scholar
  10. 10.
    Zhang, H., Xu, T., Elhoseiny, M., Huang, X., Zhang, S., Elgammal, A., Metaxas, D.: SPDA-CNN: unifying semantic part detection and abstraction for fine-grained recognition. In: CVPR, pp. 1143–1152 (2016)Google Scholar
  11. 11.
    Zhang, N., Donahue, J., Girshick, R.B., Darrell, T.: Part-based R-CNNs for fine-grained category detection. In: ECCV, pp. 1173–1182 (2014)Google Scholar
  12. 12.
    Branson, S., Van Horn, G., Belongie, S., Perona, P.: Bird species categorization using pose normalized deep convolutional nets, 11 June 2014.
  13. 13.
    Branson, S., Beijbom, O., Belongie, S.: Efficient large-scale structured learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 18061813. IEEE, Portland, USA (2013)Google Scholar
  14. 14.
    Zhang, Y., Wei, X.S., Wu, J., Cai, J., Lu, J., Nguyen, V.A., Do, M.N.: Weakly supervised fine-grained categorization with part-based image representation. IEEE Trans. Image Process. 25(4), 1713–1725 (2016)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Lin, T.Y., RoyChowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the 15th IEEE International Conference on Computer Vision, pp. 14491457. IEEE, Santiago, Chile (2015)Google Scholar
  16. 16.
    Xiao, T., Xu, Y., Yang, K., Zhang, J., Peng, Y., Zhang, Z.: The application of two-level attention models in deep convolutional neural network for fine-grained image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 842850. IEEE, Boston, USA (2015)Google Scholar
  17. 17.
    Uijlings, J.R., van de Sande, K.E., Gevers, T., Smeulders, A.W.: Selective search for object recognition. Int. J. Comput. Vision 104(2), 154–171 (2013)CrossRefGoogle Scholar
  18. 18.
    Lin-Bo, Z., Chun-Heng, W., Bai-Hua, X., Yun-Xue, S.: Image representation using bag-of-phrases. Acta Automatica Sinica 38(1), 46–54 (2012)CrossRefGoogle Scholar
  19. 19.
    Wang-Sheng, Yu., Xiao-Hua, T., Zhi-Qiang, H.: A new image feature descriptor based on region edge statistical. Chin. J. Comput. 37(6), 1298–1410 (2014)Google Scholar
  20. 20.
    Xue-Jun, Y., Chun-Xia, Z., Xia, Y.: 2DPCA-SIFT: an efficient local feature descriptor. Acta Automatica Sinica 40(4), 675–682 (2014)Google Scholar
  21. 21.
    Fu, J., Mei, T., Yang, K., Lu, H., Rui, Y.: Tagging personal photos with transfer deep learning. In: WWW, pp. 344–354 (2015)Google Scholar
  22. 22.
    Fu, J., Wang, J., Rui, Y., Wang, X.-J., Mei, T., Lu, H.: Image tag refinement with view-dependent concept representations. IEEE T-CSVT 25(28), 1409–1422 (2015)Google Scholar
  23. 23.
    Fu, J., Wu, Y., Mei, T., Wang, J., Lu, H., Rui, Y.: Relaxing from vocabulary: robust weakly-supervised deep learning for vocabulary-free image tagging. In: ICCV (2015)Google Scholar
  24. 24.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS, pp. 1106–1114 (2012)Google Scholar
  25. 25.
    Wang, J., Fu, J., Mei, T., Xu, Y.: Beyond object recognition: visual sentiment analysis with deep coupled adjective and noun neural networks. In: IJCAI (2016)Google Scholar
  26. 26.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)Google Scholar
  27. 27.
    Zhang, X., Xiong, H., Zhou, W., Lin, W., Tian, Q.: Picking deep filter responses for fine-grained image recognition. In: CVPR, pp. 1134–1142 (2016)Google Scholar
  28. 28.
    Perronnin, F., Larlus, D.: Fisher vectors meet neural networks: a hybrid classification architecture. In: CVPR, pp. 3743–3752 (2015)Google Scholar
  29. 29.
    Girshick, R.B.: Fast R-CNN. In: ICCV, pp. 1440–1448 (2015)Google Scholar
  30. 30.
    Zhao, B., Wu, X., Feng, J., Peng, Q., Yan, S.: Diversified visual attention networks for fine-grained object classification. CoRR, abs/1606.08572 (2016)Google Scholar
  31. 31.
    Liu, X., Xia, T., Wang, J., Lin, Y.: Fully convolutional attention localization networks: efficient attention localization for fine-grained recognition. CoRR, abs/1603.06765 (2016)Google Scholar
  32. 32.
    Lin, T.-Y., RoyChowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: ICCV, pp. 1449–1457 (2015)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Xiu-Hong Yang
    • 1
  • Ji-Xiang Du
    • 1
    Email author
  • Hong-Bo Zhang
    • 1
  • Wen-Tao Fan
    • 1
  1. 1.Department of Computer Science and TechnologyHuaqiao UniversityXiamenChina

Personalised recommendations