Deep Sparse Informative Transfer SoftMax for Cross-Domain Image Classification

  • Hanfang Yang
  • Xiangdong ZhouEmail author
  • Lan Lin
  • Bo Yao
  • Zijing Tan
  • Haocheng Tang
  • Yingjie Tian
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10828)


In many real applications, it is often encountered that the models trained on source domain cannot fit the related target images very well, due to the variants and changes of the imaging background, lighting of environment, viewpoints and so forth. Therefore cross-domain image classification becomes a very interesting research problem. Lots of research efforts have been conducted on this problem, where many of them focus on exploring the cross-domain image features. Recently transfer learning based methods become the main stream. In this paper, we present a novel transfer SoftMax model called Sparse Informative Transfer SoftMax (SITS) to deal with the problem of cross-domain image classification. SITS is a flexible classification framework. Specifically, the principle eigenvectors of the target domain feature space are introduced into our objective function, hence the informative features of the target domain are exploited in the process of the model training. The sparse regularization for feature selection and the SoftMax classification are also employed in our framework. On this basis, we developed Deep SITS network to efficiently learn informative transfer model and enhance the transferable ability of deep neural network. Extensive experiments are conducted on several commonly used benchmarks. The experimental results show that comparing with the state-of-the-art methods, our method achieves the best performance.


Transfer learning Neural network Deep learning Image classification Sparse regularization 



This work was supported by the National High Technology Research and Development Program (863 Program) of China (2015AA050203), NSFC grant no. 61370157, NSFC grant no. 61373106, NSFC grant no. 61572135 and State Grid Shanghai Company Project No. 52094016001Z.


  1. 1.
    Azizpour, H., Razavian, A.S., Sullivan, J., Maki, A., Carlsson, S.: From generic to specific deep representations for visual recognition. CoRR abs/1406.5774 (2014)Google Scholar
  2. 2.
    Borgwardt, K.M., Gretton, A., Rasch, M.J., Kriegel, H.-P., Schölkopf, B., Smola, A.J.: Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14), e49–e57 (2006)CrossRefGoogle Scholar
  3. 3.
    Bruzzone, L., Marconcini, M.: Domain adaptation problems: a DASVM classification technique and a circular validation strategy. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 770–787 (2010)CrossRefGoogle Scholar
  4. 4.
    Chen, M., Weinberger, K.Q., Blitzer, J: Co-training for domain adaptation. In: Advances in Neural Information Processing Systems, pp. 2456–2464 (2011)Google Scholar
  5. 5.
    Ding, Z., Shao, M., Fu, Y.: Deep low-rank coding for transfer learning. In: IJCAI, pp. 3453–3459 (2015)Google Scholar
  6. 6.
    Lixin, D., Tsang, I.W., Xu, D.: Domain transfer multiple kernel learning. IEEE Trans. Pattern Anal. Mach. Intell. 34(3), 465–479 (2012)CrossRefGoogle Scholar
  7. 7.
    Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. arXiv preprint arXiv:1409.7495 (2014)
  8. 8.
    Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., Lempitsky, V.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(59), 1–35 (2016)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Gong, B., Shi, Y., Sha, F., Grauman, K.: Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2066–2073. IEEE (2012)Google Scholar
  10. 10.
    Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset (2007)Google Scholar
  11. 11.
    Hastie, T., Tibshirani, R., Wainwright, M.: Statistical Learning with Sparsity. CRC Press, Boca Raton (2015)CrossRefGoogle Scholar
  12. 12.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014)Google Scholar
  13. 13.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  14. 14.
    Long, M., Cao, Y., Wang, J., Jordan, M.I.: Learning transferable features with deep adaptation networks. In: ICML, pp. 97–105 (2015)Google Scholar
  15. 15.
    Long, M., Wang, J., Ding, G., Sun, J., Yu, P.S.: Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2200–2207 (2013)Google Scholar
  16. 16.
    Long, M., Wang, J., Ding, G., Sun, J., Yu, P.S.: Transfer joint matching for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1410–1417 (2014)Google Scholar
  17. 17.
    Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22(2), 199–210 (2011)CrossRefGoogle Scholar
  18. 18.
    Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 213–226. Springer, Heidelberg (2010). Scholar
  19. 19.
    Razavian, A.S., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 806–813 (2014)Google Scholar
  20. 20.
    Tzeng, E., Hoffman, J., Darrell, T., Saenko, K.: Simultaneous deep transfer across domains and tasks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4068–4076 (2015)Google Scholar
  21. 21.
    Tzeng, E., Hoffman, J., Saenko, K., Darrell, T.: Adversarial discriminative domain adaptation. In: ICLR Workshop (2017)Google Scholar
  22. 22.
    Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., Darrell, T.: Deep domain confusion: maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014)
  23. 23.
    Wang, H., Wang, W., Zhang, C., Xu, F.: Cross-domain metric learning based on information theory. In: AAAI, pp. 2099–2105 (2014)Google Scholar
  24. 24.
    Wang, W., Wang, H., Zhang, C., Xu, F.: Transfer feature representation via multiple kernel learning. In: AAAI, pp. 3073–3079 (2015)Google Scholar
  25. 25.
    Weston, J., Ratle, F., Mobahi, H., Collobert, R.: Deep learning via semi-supervised embedding. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 639–655. Springer, Heidelberg (2012). Scholar
  26. 26.
    Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)Google Scholar
  27. 27.
    Zhu, G., Yang, H., Lin, L., Zhou, G., Zhou, X.: An informative logistic regression for cross-domain image classification. In: Nalpantidis, L., Krüger, V., Eklundh, J.-O., Gasteratos, A. (eds.) ICVS 2015. LNCS, vol. 9163, pp. 147–156. Springer, Cham (2015). Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Hanfang Yang
    • 1
  • Xiangdong Zhou
    • 1
    Email author
  • Lan Lin
    • 2
  • Bo Yao
    • 1
  • Zijing Tan
    • 1
  • Haocheng Tang
    • 1
  • Yingjie Tian
    • 3
  1. 1.School of Computer ScienceFudan UniversityShanghaiChina
  2. 2.School of Electronics and Information EngineeringTongji UniversityShanghaiChina
  3. 3.State Grid Shanghai Municipal Electric Power CompanyShanghaiChina

Personalised recommendations