Advertisement

Gradient Boosting with Neural Networks

  • Michel Denuit
  • Donatien Hainaut
  • Julien Trufin
Chapter
Part of the Springer Actuarial book series (SPACT)

Abstract

Gradient boosting machines form a family of powerful machine learning techniques that have been applied with success in a wide range of practical applications. Ensemble techniques rely on simple averaging of models in the ensemble. The family of boosting methods adopts a different strategy to construct ensembles. In boosting algorithms, new models are sequentially added to the ensemble. At each iteration, a new weak base-learner is trained with respect to the error of the whole ensemble built so far.

References

  1. Bissacco A, Yang MH, Soatto S (2007) Fast human pose estimation using appearance and motion via multi-dimensional boosting regression. In: IEEE conference on computer vision and pattern recognition, CVPR’07Google Scholar
  2. Chen T, Guestrin C (2016) XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, San Francisco, August 13–17, 2016. ACM, New York, pp 785–794Google Scholar
  3. Freund Y, Schapire R (1996) Experiments with a new boosting algorithm. In: Machine learning: proceedings of the thirteenth international conference, pp 148–156Google Scholar
  4. Freund Y, Schapire R (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55:119–139MathSciNetCrossRefGoogle Scholar
  5. Friedman JH (1999) Greedy function approximation: a gradient boosting machine. Technical report, Dept. of Statistics, Stanford UniversityGoogle Scholar
  6. Friedman JH, Hastie T, Tibshirani R (1998) Additive logistic regression: a statistical view of boosting. Technical report, Dept. of Statistics, Stanford UniversityGoogle Scholar
  7. Hutchinson RA, Liu LP, Dietterich TG (2011) Incorporating boosted regression trees into ecological latent variable models. In: Twenty-fifth conference on artificial intelligence, AAAI’11, San Francisco, pp 1343–1348Google Scholar
  8. Mason L, Baxter J, Bartlett PL, Frean M (1999) Boosting algorithms as gradient descent. In: Solla SA, Leen TK, Muller K (eds) Advances in neural information processing system, vol 12. MIT Press, Cambridge, pp 512–518Google Scholar
  9. Pittman SJ, Brown KA (2011) Multi-scale approach for predicting fish species distributions across coral reef seascapes. PLoS ONE 6(5):e20583.  https://doi.org/10.1371/journal.pone.0020583 CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Michel Denuit
    • 1
  • Donatien Hainaut
    • 2
  • Julien Trufin
    • 3
  1. 1.Université Catholique LouvainLouvain-la-NeuveBelgium
  2. 2.Université Catholique de LouvainLouvain-la-NeuveFrance
  3. 3.Université Libre de BruxellesBrusselsBelgium

Personalised recommendations