The Effect of Task Similarity on Deep Transfer Learning

  • Wei Zhang
  • Yuchun FangEmail author
  • Zhengyan Ma
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10635)


In recent years, with deep learning achieving a great success, deep transfer learning gradually becomes a new issue. Fine-tuning as a simple transfer learning method can be used to help train deep network and improve the performance of network. In our paper, we use two fine-tuning strategies on deep convolutional neural network and compare their results. There are many influencing factors, such as the depth and width of the network, the amount of data, the similarity of the source and target domain, and so on. Then we keep the network structure and other related factors consistent and use the fine fine-tuning strategy to find the effect of cross-domain factor and similarity of task. Specifically, we use source network and target test data to calculate the similarity. The results of experiments show that when we use fine-tune strategy, using different dataset in source and target domain would affect the target task a lot. Besides the similarity of tasks has direction, and to some extent the similarity would reflect the increment of performance of target task when the source and target task use the same dataset.


Deep learning Transfer learning CNN 



The work is funded by the National Natural Science Foundation of China (No. 61170155), Shanghai Innovation Action Plan Project (No. 16511101200) and the Open Project Program of the National Laboratory of Pattern Recognition (No. 201600017).


  1. 1.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) NIPS 2012, pp. 1097–1105 (2012)Google Scholar
  2. 2.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: IEEE CVPR 2015 (2015)Google Scholar
  3. 3.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng., 1345–1359 (2010)Google Scholar
  4. 4.
    Ge, W., Yu, Y.: Borrowing treasures from the wealthy: deep transfer learning through selective joint fine-tuning. arXiv preprint arXiv:1702.08690 (2017)
  5. 5.
    Xu, Z., Huang, S., Zhang, Y., Tao, D.: Webly-supervised fine-grained visual categorization via deep domain adaptation. IEEE Trans. Pattern Anal. Mach. Intell. (2016)Google Scholar
  6. 6.
    Long, M., Wang, J., Jordan, M.I.: Deep transfer learning with joint adaptation networks. arXiv preprint arXiv:1605.06636 (2016)
  7. 7.
    Ding, Z., Nasrabadi, N.M., Fu, Y.: Task-driven deep transfer learning for image classification. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2414–2418. IEEE (2016)Google Scholar
  8. 8.
    Azizpour, H., Razavian, A. S., Sullivan, J., Maki, A., Carlsson, S.: Factors of transferability for a generic convNet representation. IEEE Trans. Pattern Anal. Mach. Intell., 1790–1802 (2016)Google Scholar
  9. 9.
    A visual proof that neural nets can compute any function, Accessed 20 May 2017
  10. 10.
    CS231n Convolutional Neural Networks for Visual Recognition, Accessed 15 June 2017

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of Computer Engineering and ScienceShanghai UniversityShanghaiChina

Personalised recommendations