Encyclopedia of Database Systems

2018 Edition
| Editors: Ling Liu, M. Tamer Özsu

Boosting

  • Zhi-Hua Zhou
Reference work entry
DOI: https://doi.org/10.1007/978-1-4614-8265-9_568

Definition

Boosting is a kind of ensemble methods [13] which produces a strong learner that is capable of making very accurate predictions by combining rough and moderately inaccurate learners (which are called as base learners or weak learners). In particular, boosting sequentially trains a series of base learners by using a base learning algorithm, where the training examples wrongly predicted by a base learner will receive more attention from the successive base learner. After that, it generates a final strong learner through a weighted combination of these base learners.

Historical Background

In 1989, Kearns and Valiant posed an interesting theoretical question, i.e., whether two complexity classes, weakly learnable and strongly learnable problems, are equal. In other words, whether a weak learning algorithm that performs just slightly better than random guess can be boosted into an arbitrarily accurate strong learning algorithm. In 1990, Schapire [9] proved that the answer to the...

This is a preview of subscription content, log in to check access.

Recommended Reading

  1. 1.
    Bauer E, Kohavi R. An empirical comparison of voting classification algorithms: bagging, boosting, and variants. Mach Learn. 1999;36(1–2):105–39.CrossRefGoogle Scholar
  2. 2.
    Breiman L. Prediction games and arcing classifiers. Neural Comput. 1999;11(7):1493–517.CrossRefGoogle Scholar
  3. 3.
    Freund Y, Schapire RE. A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci. 1997;55(1):119–39 (A short version appeared in the Proceedings of EuroCOLT’95).MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Friedman J, Hastie T, Tibshirani R. Additive logistic regression: a statistical view of boosting with discussions. Ann Stat. 2000;28(2):337–407.zbMATHCrossRefGoogle Scholar
  5. 5.
    Gao W, Zhou Z-H. On the doubt about margin explanation of boosting. Artif Intell. 2013;203:1–18.MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Meir R, Rätsch G. An introduction to boosting and leveraging. In: Mendelson S, Smola AJ, editors. Advanced lectures in machine learning. LNCS vol. 2600. Berlin: Springer; 2003. p. 118–83.CrossRefGoogle Scholar
  7. 7.
    Opitz D, Maclin R. Popular ensemble methods: an empirical study. J Artif Intell Res. 1999;11(1):169–98.zbMATHCrossRefGoogle Scholar
  8. 8.
    Reyzin L, Schapire RE. How boosting the margin can also boost classifier complexity. In: Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh; 2006. p. 753–60.Google Scholar
  9. 9.
    Schapire RE. The strength of weak learn ability. Mach Learn. 1990;5(2):197–227.Google Scholar
  10. 10.
    Schapire RE, Freund Y, Bartlett P, Lee WS. Boosting the margin: a new explanation for the effectiveness of voting methods. Ann Stat. 1998;26(5):1651–86.MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; 2001. p. 511–8.Google Scholar
  12. 12.
    Wang L, Sugiyama M, Yang C, Zhou Z.H, Feng J. On the margin explanation of boosting algorithm. In: Proceedings of the 21st Annual Conference on Learning Theory; 2008. p. 479–90.Google Scholar
  13. 13.
    Zhou Z-H. Ensemble methods: foundations and algorithms. Boca Raton: CRC Press; 2012.CrossRefGoogle Scholar
  14. 14.
    Zhou Z-H. Large margin distribution learning. In: Proceedings of Artificial Neural Networks in Pattern Recognition; 2014.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.National Key Lab for Novel Software TechnologyNanjing UniversityNanjingChina

Section editors and affiliations

  • Kyuseok Shim
    • 1
  1. 1.School of Elec. Eng. and Computer ScienceSeoul National Univ.SeoulRepublic of Korea