Advertisement

Selective Zero-Shot Classification with Augmented Attributes

  • Jie Song
  • Chengchao Shen
  • Jie Lei
  • An-Xiang Zeng
  • Kairi Ou
  • Dacheng Tao
  • Mingli Song
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11213)

Abstract

In this paper, we introduce a selective zero-shot classification problem: how can the classifier avoid making dubious predictions? Existing attribute-based zero-shot classification methods are shown to work poorly in the selective classification scenario. We argue the under-complete human defined attribute vocabulary accounts for the poor performance. We propose a selective zero-shot classifier based on both the human defined and the automatically discovered residual attributes. The proposed classifier is constructed by firstly learning the defined and the residual attributes jointly. Then the predictions are conducted within the subspace of the defined attributes. Finally, the prediction confidence is measured by both the defined and the residual attributes. Experiments conducted on several benchmarks demonstrate that our classifier produces a superior performance to other methods under the risk-coverage trade-off metric.

Keywords

Zero-shot classification Selective classification Defined attributes Residual attributes Risk-coverage trade-off 

Notes

Acknowledgements

This work is supported by National Key Research and Development Program (2016YFB1200203), National Natural Science Foundation of China (61572428, U1509206), Fundamental Research Funds for the Central Universities (2017FZA5014) and Key Research, Development Program of Zhejiang Province (2018C01004) and ARC FL-170100117, DP-180103424 of Australia.

References

  1. 1.
    Lampert, C.H., Nickisch, H., Harmeling, S.: Learning to detect unseen object classes by between-class attribute transfer. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 951–958. IEEE (2009)Google Scholar
  2. 2.
    Akata, Z., Perronnin, F., Harchaoui, Z., Schmid, C.: Label-embedding for attribute-based classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 819–826 (2013)Google Scholar
  3. 3.
    Romera-Paredes, B., Torr, P.: An embarrassingly simple approach to zero-shot learning. In: International Conference on Machine Learning, pp. 2152–2161 (2015)Google Scholar
  4. 4.
    Akata, Z., Reed, S., Walter, D., Lee, H., Schiele, B.: Evaluation of output embeddings for fine-grained image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2927–2936 (2015)Google Scholar
  5. 5.
    Xian, Y., Schiele, B., Akata, Z.: Zero-shot learning - the good, the bad and the ugly. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  6. 6.
    Song, J., Shen, C., Yang, Y., Liu, Y., Song, M.: Transductive unbiased embedding for zero-shot learning. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018Google Scholar
  7. 7.
    Chow, C.K.: An optimum character recognition system using decision functions. IRE Trans. Electron. Comput. EC 6(4), 247–254 (1957)MathSciNetCrossRefGoogle Scholar
  8. 8.
    El-Yaniv, R., Wiener, Y.: On the foundations of noise-free selective classification. J. Mach. Learn. Res. 11, 1605–1641 (2010)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Peng, P., Tian, Y., Xiang, T., Wang, Y., Huang, T.: Joint learning of semantic and latent attributes. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 336–353. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46493-0_21CrossRefGoogle Scholar
  10. 10.
    Jiang, H., Wang, R., Shan, S., Yang, Y., Chen, X.: Learning discriminative latent attributes for zero-shot classification. In: The IEEE International Conference on Computer Vision (ICCV), October 2017Google Scholar
  11. 11.
    Kan, M., Shan, S., Chen, X.: Multi-view deep network for cross-view classification. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016Google Scholar
  12. 12.
    Fu, Y., Hospedales, T.M., Xiang, T., Fu, Z., Gong, S.: Transductive multi-view embedding for zero-shot recognition and annotation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 584–599. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10605-2_38CrossRefGoogle Scholar
  13. 13.
    Frome, A., Corrado, G.S., Shlens, J., Bengio, S., Dean, J., Mikolov, T., et al.: Devise: a deep visual-semantic embedding model. In: Advances in neural information processing systems, pp. 2121–2129 (2013)Google Scholar
  14. 14.
    Akata, Z., Perronnin, F., Harchaoui, Z., Schmid, C.: Label-embedding for image classification. IEEE Trans. Pattern Anal. Mach. Intell. 38(7), 1425–1438 (2016)CrossRefGoogle Scholar
  15. 15.
    Xian, Y., Akata, Z., Sharma, G., Nguyen, Q., Hein, M., Schiele, B.: Latent embeddings for zero-shot classification. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016Google Scholar
  16. 16.
    Socher, R., Ganjoo, M., Manning, C.D., Ng, A.: Zero-shot learning through cross-modal transfer. In: Advances in Neural Information Processing Systems, pp. 935–943 (2013)Google Scholar
  17. 17.
    Zhang, Z., Saligrama, V.: Zero-shot learning via semantic similarity embedding. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4166–4174 (2015)Google Scholar
  18. 18.
    Norouzi, M., et al.: Zero-shot learning by convex combination of semantic embeddings. In: Proceedings of ICLR. Citeseer (2014)Google Scholar
  19. 19.
    Changpinyo, S., Chao, W.L., Gong, B., Sha, F.: Synthesized classifiers for zero-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5327–5336 (2016)Google Scholar
  20. 20.
    Fu, Y., Xiang, T., Jiang, Y.G., Xue, X., Sigal, L., Gong, S.: Recent advances in zero-shot recognition: toward data-efficient understanding of visual content. IEEE Signal Process. Mag. 35(1), 112–125 (2018)CrossRefGoogle Scholar
  21. 21.
    Berg, T.L., Berg, A.C., Shih, J.: Automatic attribute discovery and characterization from noisy web data. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 663–676. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15549-9_48CrossRefGoogle Scholar
  22. 22.
    Sharmanska, V., Quadrianto, N., Lampert, C.H.: Augmented attribute representations. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 242–255. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33715-4_18CrossRefGoogle Scholar
  23. 23.
    Rastegari, M., Farhadi, A., Forsyth, D.: Attribute discovery via predictable discriminative binary codes. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 876–889. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33783-3_63CrossRefGoogle Scholar
  24. 24.
    Yu, F.X., Cao, L., Feris, R.S., Smith, J.R., Chang, S.F.: Designing category-level attributes for discriminative visual recognition. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, June 2013Google Scholar
  25. 25.
    Szegedy, C., et al.: Intriguing properties of neural networks. In: International Conference on Learning Representations (2014)Google Scholar
  26. 26.
    Mandelbaum, A., Weinshall, D.: Distance-based confidence score for neural network classifiers. CoRR abs/1709.09844 (2017)Google Scholar
  27. 27.
    Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: Proceedings of the 33rd International Conference on Machine Learning (ICML2016) (2016)Google Scholar
  28. 28.
    Geifman, Y., El-Yaniv, R.: Selective classification for deep neural networks. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 4885–4894. Curran Associates, Inc. (2017)Google Scholar
  29. 29.
    Farhadi, A., Endres, I., Hoiem, D., Forsyth, D.: Describing objects by their attributes. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1778–1785. IEEE (2009)Google Scholar
  30. 30.
    Welinder, P., et al.: Caltech-UCSD Birds 200. Technical report CNS-TR-2010-001, California Institute of Technology (2010)Google Scholar
  31. 31.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  32. 32.
    Morgado, P., Vasconcelos, N.: Semantically consistent regularization for zero-shot recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  33. 33.
    Kodirov, E., Xiang, T., Gong, S.: Semantic autoencoder for zero-shot learning. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  34. 34.
    Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136 (2016)

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.College of Computer Science and TechnologyZhejiang UniversityHangzhouChina
  2. 2.Alibaba GroupHangzhouChina
  3. 3.UBTECH Sydney AI Centre, SIT, FEITUniversity of SydneyCamperdownAustralia

Personalised recommendations