Advertisement

Web-Induced Heterogeneous Transfer Learning with Sample Selection

  • Sanatan SukhijaEmail author
  • Narayanan C. Krishnan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11052)

Abstract

Transfer learning algorithms utilize knowledge from a data-rich source domain to learn a model in the target domain where labeled data is scarce. This paper presents a novel solution for the challenging and interesting problem of Heterogeneous Transfer Learning (HTL) where the source and target task have heterogeneous feature and label spaces. Contrary to common space based HTL algorithms, the proposed HTL algorithm adapts source data for the target task. The correspondence required for aligning the heterogeneous features of the source and target domain is obtained through labels across two domains that are semantically aligned using web-induced knowledge. The experimental results suggest that the proposed algorithm performs significantly better than state-of-the-art transfer approaches on three diverse real-world transfer tasks.

Keywords

Heterogeneous Transfer Learning Sample Selection 

Notes

Acknowledgment

This research is supported by the Department of Science and Technology, India under grant YSS/2015/001206, and by the Indian Institute of Technology Ropar under the ISIRD grant.

References

  1. 1.
    Sukhija, S.: Label space driven heterogeneous transfer learning with web induced alignment. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence (2018)Google Scholar
  2. 2.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556 (2014)
  3. 3.
    Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images, Citeseer (2009)Google Scholar
  4. 4.
    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Li, F.-F.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009)Google Scholar
  5. 5.
    Zhu,Y., et al.: Heterogeneous transfer learning for image classification. In: Proceedings of the AAAI National Conference on Artificial Intelligence (2011)Google Scholar
  6. 6.
    Prettenhofer, P., Stein, B.: Cross-language text classification using structural correspondence learning. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL) (2010)Google Scholar
  7. 7.
    Cilibrasi, R.L., Vitanyi, P.M.B.: The google similarity distance. In: IEEE Transactions on Knowledge and Data Engineering (2007)Google Scholar
  8. 8.
    Wei, Y., Zhu, Y., Leung, C.W.-k., Song, Y., Yang, Q.: Instilling social to physical: co-regularized heterogeneous transfer learning. In: Proceedings of the AAAI National Conference on Artificial Intelligence (2016)Google Scholar
  9. 9.
    Branch, M.A., Coleman, T.F., Li, Y.: A subspace, interior, and conjugate gradient method for large-scale bound-constrained minimization problems. SIAM J. Sci. Comput. 21(1), 1–23 (1999)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Hu, D.H., Yang, Q.: Transfer learning for activity recognition via sensor mapping. In: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence (IJCAI) (2011)Google Scholar
  11. 11.
    Lichman, M.: UCI Machine Learning Repository. University of California, Irvine, School of Information and Computer Sciences (2013). http://archive.ics.uci.edu/ml
  12. 12.
    Cook, D.J., Crandall, A.S., Thomas, B.L., Krishnan, N.C.: CASAS: a smart home in a box. Computer 46(7), 62–69 (2013)CrossRefGoogle Scholar
  13. 13.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)CrossRefGoogle Scholar
  14. 14.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3(Jan), 993–1022 (2003)zbMATHGoogle Scholar
  15. 15.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems (2013)Google Scholar
  16. 16.
    Zhou, J.T., Tsang, I.W., Pan, S.J., Tan, M.: Heterogeneous domain adaptation for multiple classes. In: Proceedings of the International Conference on Artificial Intelligence and Statistics (2014)Google Scholar
  17. 17.
    Li, W., Duan, L., Xu, D., Tsang, I.W.: Learning with augmented features for supervised and semi-supervised heterogeneous domain adaptation. IEEE Trans. Pattern Anal. Mach. Intell. 36(6), 1134–1148 (2014)CrossRefGoogle Scholar
  18. 18.
    van Kasteren, T.L.M., Englebienne, G., Kröse, B.J.A.: Transferring knowledge of activity recognition across sensor networks. In: Floréen, P., Krüger, A., Spasojevic, M. (eds.) Pervasive 2010. LNCS, vol. 6030, pp. 283–300. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-12654-3_17CrossRefGoogle Scholar
  19. 19.
    Borchani, H., Varando, G., Bielza, C., Larrañaga, P.: A survey on multi-output regression. Wiley Interdisc. Rev. Data Min. Knowl. Discovery 5(5), 216–233 (2015)CrossRefGoogle Scholar
  20. 20.
    He, J., Liu, Y., Yang, Q.: Linking heterogeneous input spaces with pivots for multi-task learning. In: Proceedings of the SIAM International Conference on Data Mining (2014)Google Scholar
  21. 21.
    Wang, C., Mahadevan, S.: Manifold alignment using procrustes analysis. In: Proceedings of the International Conference on Machine learning (2008)Google Scholar
  22. 22.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefGoogle Scholar
  23. 23.
    Sukhija, S.: Label space driven feature space remapping. In: Proceedings of the ACM India Joint International Conference on Data Science and Management of Data (2018)Google Scholar
  24. 24.
    Wang, C., Mahadevan, S.: Manifold alignment without correspondence. In: Proceedings of the International Joint Conference on Artificial Intelligence (2009)Google Scholar
  25. 25.
    Rosenstein, M.T., Marx, Z., Kaelbling, L.P., Dietterich, T.G.: To transfer or not to transfer. In: NIPS 2005 Workshop on Transfer Learning (2005)Google Scholar
  26. 26.
    Hubert Tsai, Y.-H., Yeh, Y.R., Frank Wang, Y.-C.: Learning cross-domain landmarks for heterogeneous domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  27. 27.
    Yan, Y., et al.: Learning discriminative correlation subspace for heterogeneous domain adaptation. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence (2017)Google Scholar
  28. 28.
    Sukhija, S., Krishnan, N.C., Kumar, D.: Supervised heterogeneous transfer learning using random forests. In: Proceedings of the ACM India Joint International Conference on Data Science and Management of Data (2018)Google Scholar
  29. 29.
    Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2006)Google Scholar
  30. 30.
    Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32(2), 407–499 (2004)Google Scholar
  31. 31.
    Wang, C., Mahadevan, S.: Heterogeneous domain adaptation using manifold alignment. In: Proceedings of the International Joint Conference on Artificial Intelligence (2011)Google Scholar
  32. 32.
    Kyrillidis, A., Zouzias, A.: Non-uniform feature sampling for decision tree ensembles. In: Proceedings of the IEEE International Conference on Acoustics Speech and Signal Processing (2014)Google Scholar
  33. 33.
    Sukhija, S., Krishnan, N.C., Singh, G.: Supervised heterogeneous domain adaptation via random forests. In: Proceedings of the International Joint Conference on Artificial Intelligence (2016)Google Scholar
  34. 34.
    Wang, C., Mahadevan, S.: A general framework for manifold alignment. In: Proceedings of the AAAI Fall Symposium: Manifold Learning and Its Applications (2009)Google Scholar
  35. 35.
    Yang, Q., Chen, Y., Xue, G.-R., Dai, W., Yu, Y.: Heterogeneous transfer learning for image clustering via the social web. In: Proceedings of the International Joint Conference on Natural Language Processing (2009)Google Scholar
  36. 36.
    Amini, M., Usunier, N., Goutte, C.: Learning from multiple partially observed views-an application to multilingual text categorization. In: Advances in Neural Information Processing Systems (2009)Google Scholar
  37. 37.
    Xiao, M., Guo, Y.: Semi-supervised subspace co-projection for multi-class heterogeneous domain adaptation. In: Appice, A., Rodrigues, P.P., Santos Costa, V., Gama, J., Jorge, A., Soares, C. (eds.) ECML PKDD 2015. LNCS (LNAI), vol. 9285, pp. 525–540. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-23525-7_32CrossRefGoogle Scholar
  38. 38.
    Zhou, G., He, T., Wu, W., Hu, X.T.: Linking heterogeneous input features with pivots for domain adaptation. Proceedings of the 24th International Conference on Artificial Intelligence (IJCAI) (2015)Google Scholar
  39. 39.
    Zhou, J.T., Pan, S.J., Tsang, I.W., Yan, Y.: Hybrid heterogeneous transfer learning through deep learning. In: Proceedings of the AAAI National Conference on Artificial Intelligence (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Indian Institute of Technology RoparPunjabIndia

Personalised recommendations