Abstract
Various instance weighting methods have been proposed for instance-based transfer learning. Kernel Mean Matching (KMM) is one of the typical instance weighting approaches which estimates the instance importance by matching the two distributions in the universal reproducing kernel Hilbert space (RKHS). However, KMM is an unsupervised learning approach which does not utilize the class label knowledge of the source data. In this paper, we extended KMM by leveraging the class label knowledge and integrated KMM and SVM into an unified optimization framework called KMM-LM (Large Margin). The objective of KMM-LM is to maximize the geometric soft margin, and minimize the empirical classification error together with the domain discrepancy based on KMM simultaneously. KMM-LM utilizes an iterative minimization algorithm to find the optimal weight vector of the classification decision hyperplane and the importance weight vector of the instances in the source domain. The experiments show that KMM-LM outperforms the state-of-the-art baselines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sarinnapakorn, K., Kubat, M.: Combining Sub-classifiers in Text Categorization: A DST-Based Solution and a Case Study. IEEE Transactions on Knowledge and Data Engineering 19(12), 1638–1651 (2007)
Blitzer, J., Dredze, M., Pereira, F.: Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification. In: ACL 2007, pp. 440–447 (2007)
Blitzer, J., Kakade, S., Foster, D.P.: Domain Adaptation with Coupled Subspaces. In: AISTATS 2011, pp. 173–181 (2011)
Pan, W., Xiang, E.W., Liu, N.N., Yang, Q.: Transfer Learning in Collaborative Filtering for Sparsity Reduction. In: AAAI 2010, pp. 230–235 (2010)
Ma, H., Zhou, D., Liu, C., Lyu, M.R., King, I.: Recommender Systems with Social Regularization. In: WSDM 2011, pp. 287–296 (2011)
Gao, W., Cai, P., Wong, K.-F., Zhou, A.: Learning to Rank only using Training Data from related Domain. In: SIGIR 2010, pp. 162–169 (2010)
Huang, J., Smola, A.J., Gretton, A., Borgwardt, K.M., Schölkopf, B.: Correcting Sample Selection Bias by Unlabeled Data. In: NIPS 2006, pp. 601–608 (2006)
Joachims, T.: Transductive Inference for Text Classification using Support Vector Machines. In: IMCL 1999, pp. 200–209 (1999)
Pan, S.J., Yang, Q.: A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering 22(10), 1345–1359 (2010)
Jiang, J., Zhai, C.: Instance Weighting for Domain Adaptation in NLP. In: ACL 2007, pp. 264–271 (2007)
Dai, W., Yang, Q., Xue, G., Yu, Y.: Boosting for Transfer Learning. In: ICML 2007, pp. 193–200 (2007)
Dai, W., Xue, G., Yang, Q., Yu, Y.: Co-clustering Based Classification for Out-of-domain Documents. In: KDD 2007, pp. 210–219 (2007)
Dayanik, A.A., Lewis, D.D., Madigan, D., Menkov, V., Genkin, A.: Constructing Informative Prior Distributions from Domain Knowledge in Text Classification. In: SIGIR 2006, pp. 493–500 (2006)
Kanamori, T., Suzuki, T., Sugiyama, M.: Statistical Analysis of Kernel-based Least-squares Density-ratio Estimation. Machine Learning (ML) 86(3), 335–367 (2012)
Sugiyama, M., Nakajima, S., Kashima, H., von Bnau, P., Kawanabe, M.: Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation. In: NIPS 2007 (2007)
Tsuboi, Y., Kashima, H., Hido, S., Bickel, S., Sugiyama, M.: Direct Density Ratio Estimation for Large-scale Covariate Shift Adaptation. In: SDM 2008, pp. 443–454 (2008)
Kanamori, T., Hido, S., Sugiyama, M.: A Least-squares Approach to Direct Importance Estimation. Journal of Machine Learning Research (JMLR) 10, 1391–1445 (2009)
Bickel, S., Brckner, M., Scheffer, T.: Discriminative Learning for Differing Training and Test Distributions. In: ICML 2007, pp. 81–88 (2007)
Sun, Q., Chattopadhyay, R., Panchanathan, S., Ye, J.: A Two-Stage Weighting Framework for Multi-Source Domain Adaptation. In: NIPS 2011, pp. 505–513 (2011)
McCallum, A.K., Nigam, K., Rennie, J., Seymore, K.: Automating the Construction of Internet Portals with Machine Learning. Information Retrieval 3(2), 127–163 (2000)
Lewis, D.D.: Reuters-21578 Test Collection, http://www.daviddlewis.com/
Salton, G., Buckley, C.: Term-weighting Approaches in Automatic Text Retrieval. Information Processing & Management 24(5), 513–523 (1988)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tan, Q., Deng, H., Yang, P. (2012). Kernel Mean Matching with a Large Margin. In: Zhou, S., Zhang, S., Karypis, G. (eds) Advanced Data Mining and Applications. ADMA 2012. Lecture Notes in Computer Science(), vol 7713. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35527-1_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-35527-1_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35526-4
Online ISBN: 978-3-642-35527-1
eBook Packages: Computer ScienceComputer Science (R0)