Abstract
The common assumption that training and testing samples share the same distribution is often violated in practice. When this happens, traditional learning models may not generalize well. To solve this problem, domain adaptation and transfer learning try to employ training data from other related source domains. We propose a multiple sources regularization framework for this problem. The framework extends classification model with regularization by adding a special regularization term, which penalizes the target classifier far from the convex combination of source classifiers. Then this framework guarantees the target classifier minimizes the empirical risk in target domain and the distance from the convex combination of source classifier simultaneously. By the way, the weights of the convex combination of source classifiers are embedded into the learning model as parameters, and will be learned through optimization algorithm automatically, which means our framework can identify similar or related domains adaptively. We apply our framework to SVM classification model and develop an optimization algorithm to solve this problem in iterative manner. Empirical study demonstrates the proposed algorithm outperforms some state-of-art related algorithms on real-world datasets, such as text categorization and optical recognition.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Crammer, K., Kearns, M., Wortman, J.: Learning from multiple sources. Journal of Machine Learning Research 9, 1757–1774 (2008)
Bruzzone, L., Marconcini, M.: Domain adaptation problems: A dasvm classification technique and a circular validation strategy. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 770–787 (2010)
Jiang, J., Zhai, C.: Instance weighting for domain adaptation in nlp. In: ACL. The Association for Computer Linguistics (2007)
Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. In: IJCAI, pp. 1187–1192 (2009)
Duan, L., Tsang, I.W.-H., Xu, D., Maybank, S.J.: Domain transfer svm for video concept detection. In: CVPR, pp. 1375–1381. IEEE (2009)
Gao, J., Fan, W., Jiang, J., Han, J.: Knowledge transfer via multiple model local structure mapping. In: KDD, pp. 283–291 (2008)
Zhong, E., Fan, W., Peng, J., Zhang, K., Ren, J., Turaga, D.S., Verscheure, O.: Cross domain distribution adaptation via kernel mapping. In: KDD, pp. 1027–1036 (2009)
Schweikert, G., Widmer, C., Schölkopf, B., Rätsch, G.: An empirical analysis of domain adaptation algorithms for genomic sequence analysis. In: NIPS, pp. 1433–1440 (2008)
Yang, J., Yan, R., Hauptmann, A.G.: Cross-domain video concept detection using adaptive svms. ACM Multimedia, 188–197 (2007)
Luo, P., Zhuang, F., Xiong, H., Xiong, Y., He, Q.: Transfer learning from multiple source domains via consensus regularization. In: CIKM, pp. 103–112 (2008)
Duan, L., Tsang, I.W., Xu, D., Chua, T.-S.: Domain adaptation from multiple sources via auxiliary classifiers. In: ICML, p. 37 (2009)
Vapnik, V.: Statistical Learning Theory. JohnWiley, NewYork (1998)
Bach, F., Harchaoui, Z.: Diffrac: a discriminative and flexible framework for clustering. In: NIPS (2007)
Zhuang, J., Tsang, I.W., Hoi, S.C.H.: Simplenpkl: simple non-parametric kernel learning. In: ICML, p. 160 (2009)
Szafranski, M., Grandvalet, Y., Rakotomamonjy, A.: Composite kernel learning. Machine Learning 79(1-2), 73–103 (2010)
Boyd, S., Xiao, L.: Least-squaures covariance matrix adjustment. SIAM Journal of Matrix Anal. Appl. 27, C532–C546 (2005)
Asuncion, A., Newman, D.J.: UCI machine learning repository (2007), http://www.ics.uci.edu/mlearn/ML-Repository.html
Davidov, D., Gabrilovich, E., Markovitch, S.: Parameterized generation of labeled datasets for text categorization based on a hierarchical directory. In: SIGIR, pp. 250–257 (2004)
Daumé III, H.: Frustratingly easy domain adaptation. In: Conference of the Association for Computational Linguistics (ACL), Prague, Czech Republic (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Hu, S., Ren, J., Zhang, C., Zhang, C. (2014). Domain Transfer via Multiple Sources Regularization. In: Tseng, V.S., Ho, T.B., Zhou, ZH., Chen, A.L.P., Kao, HY. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2014. Lecture Notes in Computer Science(), vol 8444. Springer, Cham. https://doi.org/10.1007/978-3-319-06605-9_43
Download citation
DOI: https://doi.org/10.1007/978-3-319-06605-9_43
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-06604-2
Online ISBN: 978-3-319-06605-9
eBook Packages: Computer ScienceComputer Science (R0)