Advertisement

Adaptive Boosting for Transfer Learning Using Dynamic Updates

  • Samir Al-Stouhi
  • Chandan K. Reddy
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6911)

Abstract

Instance-based transfer learning methods utilize labeled examples from one domain to improve learning performance in another domain via knowledge transfer. Boosting-based transfer learning algorithms are a subset of such methods and have been applied successfully within the transfer learning community. In this paper, we address some of the weaknesses of such algorithms and extend the most popular transfer boosting algorithm, TrAdaBoost. We incorporate a dynamic factor into TrAdaBoost to make it meet its intended design of incorporating the advantages of both AdaBoost and the “Weighted Majority Algorithm”. We theoretically and empirically analyze the effect of this important factor on the boosting performance of TrAdaBoost and we apply it as a “correction factor” that significantly improves the classification performance. Our experimental results on several real-world datasets demonstrate the effectiveness of our framework in obtaining better classification results.

Keywords

Transfer learning AdaBoost TrAdaBoost Weighted Majority Algorithm 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Borgwardt, K.M., Gretton, A., Rasch, M.J., Kriegel, H.P., Schlkopf, B., Smola, A.J.: Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14), e49–e57 (2006)CrossRefGoogle Scholar
  2. 2.
    Cao, B., Pan, S.J., Zhang, Y., Yeung, D., Yang, Q.: Adaptive transfer learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 407–412 (2010)Google Scholar
  3. 3.
    Chawla, N.V., Lazarevic, A., Hall, L.O., Bowyer, K.W.: SMOTEBoost: Improving prediction of the minority class in boosting. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) PKDD 2003. LNCS (LNAI), vol. 2838, pp. 107–119. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  4. 4.
    Dai, W., Xue, G.R., Yang, Q., Yu, Y.: Co-clustering based classification for out-of-domain documents. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 210–219 (2007)Google Scholar
  5. 5.
    Dai, W., Yang, Q., Xue, G.R., Yu, Y.: Boosting for transfer learning. In: Proceedings of the International Conference on Machine Learning, pp. 193–200 (2007)Google Scholar
  6. 6.
    Eaton, E., desJardins, M.: Set-based boosting for instance-level transfer. In: Proceedings of the 2009 IEEE International Conference on Data Mining Workshops, pp. 422–428 (2009)Google Scholar
  7. 7.
    Eaton, E.: Selective Knowledge Transfer for Machine Learning. Ph.D. thesis, University of Maryland Baltimore County (2009)Google Scholar
  8. 8.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. In: Proceedings of the Second European Conference on Computational Learning Theory, pp. 23–37 (1995)Google Scholar
  9. 9.
    Lang, K.: Newsweeder: Learning to filter netnews. In: Proceedings of the 12th International Machine Learning Conference, pp. 331–339 (1995)Google Scholar
  10. 10.
    Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. In: Proceedings of the 30th Annual Symposium on Foundations of Computer Science, pp. 256–261 (1989)Google Scholar
  11. 11.
    Pan, S.J., Kwok, J.T., Yang, Q.: Transfer learning via dimensionality reduction. In: Proceedings of the National Conference on Artificial Intelligence, pp. 677–682 (2008)Google Scholar
  12. 12.
    Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. In: Proceedings of the 21st International Jont Conference on Artifical Intelligence, pp. 1187–1192 (2009)Google Scholar
  13. 13.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22(10), 1345–1359 (2010)CrossRefGoogle Scholar
  14. 14.
    Pardoe, D., Stone, P.: Boosting for regression transfer. In: Proceedings of the 27th International Conference on Machine Learning, pp. 863–870 (2010)Google Scholar
  15. 15.
    Reddy, C.K., Park, J.H.: Multi-resolution boosting for classification and regression problems. Knowledge and Information Systems (2011)Google Scholar
  16. 16.
    Sun, Y., Kamel, M.S., Wong, A.K., Wang, Y.: Cost-sensitive boosting for classification of imbalanced data. Pattern Recognition 40(12), 3358–3378 (2007)CrossRefzbMATHGoogle Scholar
  17. 17.
    Venkatesan, A., Krishnan, N., Panchanathan, S.: Cost-sensitive boosting for concept drift. In: Proceedings of the 2010 International Workshop on Handling Concept Drift in Adaptive Information Systems (2010)Google Scholar
  18. 18.
    Wu, P., Dietterich, T.G.: Improving svm accuracy by training on auxiliary data sources. In: Proceedings of the Twenty-First International Conference on Machine Learning, pp. 871–878 (2004)Google Scholar
  19. 19.
    Yao, Y., Doretto, G.: Boosting for transfer learning with multiple sources. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1855–1862 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Samir Al-Stouhi
    • 1
  • Chandan K. Reddy
    • 2
  1. 1.Department of Computer EngineeringWayne State UniversityDetroitUSA
  2. 2.Department of Computer ScienceWayne State UniversityDetroitUSA

Personalised recommendations