Boosted Projection: An Ensemble of Transformation Models

  • Ricardo Barbosa Kloss
  • Artur Jordão
  • William Robson Schwartz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10657)


Computer vision problems usually suffer from a very high dimensionality, which can make it hard to learn classifiers. A way to overcome this problem is to reduce the dimensionality of the input. This work presents a novel method for tackling this problem, referred to as Boosted Projection. It relies on the use of several projection models based on Principal Component Analysis or Partial Least Squares to build a more compact and richer data representation. We conducted experiments in two important computer vision tasks: pedestrian detection and image classification. Our experimental results demonstrate that the proposed approach outperforms many baselines and provides better results when compared to the original dimensionality reduction techniques of partial least squares.


Dimensionality reduction Machine learning Ensemble Partial Least Squares Computer vision 



The authors would like to thank the Brazilian National Research Council – CNPq, the Minas Gerais Research Foundation – FAPEMIG (Grants APQ-00567-14 and PPM-00540-17) and the Coordination for the Improvement of Higher Education Personnel – CAPES (DeepEyes Project). The authors gratefully acknowledge the support of NVIDIA Corporation with the donation of the GeForce Titan X GPU used for this research.


  1. 1.
    Akata, Z., Perronnin, F., Harchaoui, Z., Schmid, C.: Good practice in large-scale learning for image classification. PAMI 36(3), 507–520 (2014)CrossRefGoogle Scholar
  2. 2.
    Avidan, S.: Ensemble tracking. PAMI 29(2), 261–271 (2007)CrossRefGoogle Scholar
  3. 3.
    Benenson, R., Omran, M., Hosang, J., Schiele, B.: Ten years of pedestrian detection, what have we learned? In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8926, pp. 613–627. Springer, Cham (2015). Google Scholar
  4. 4.
    Brown, G., Kuncheva, L.I.: “Good” and “Bad” diversity in majority vote ensembles. In: El Gayar, N., Kittler, J., Roli, F. (eds.) MCS 2010. LNCS, vol. 5997, pp. 124–133. Springer, Heidelberg (2010). CrossRefGoogle Scholar
  5. 5.
    Chollet, F.: Keras (2015).
  6. 6.
    Cogranne, R., Fridrich, J.: Modeling and extending the ensemble classifier for steganalysis of digital images using hypothesis testing theory. IEEE Trans. Inf. Forensics Secur. 10(12), 2627–2642 (2015)CrossRefGoogle Scholar
  7. 7.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR (2005)Google Scholar
  8. 8.
    Dollár, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: an evaluation of the state of the art. PAMI 34, 743–761 (2012)CrossRefGoogle Scholar
  9. 9.
    Freund, Y., Schapire, R.E.: A desicion-theoretic generalization of on-line learning and an application to boosting. In: Vitányi, P. (ed.) EuroCOLT 1995. LNCS, vol. 904, pp. 23–37. Springer, Heidelberg (1995). CrossRefGoogle Scholar
  10. 10.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3(Mar), 1157–1182 (2003)MATHGoogle Scholar
  11. 11.
    Jordão, A., Schwartz, W.R.: Oblique random forest based on partial least squares applied to pedestrian detection. In: ICIP (2016)Google Scholar
  12. 12.
    Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)Google Scholar
  13. 13.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  14. 14.
    Martis, R.J., Acharya, U.R., Min, L.C.: ECG beat classification using PCA, LDA, ICA and discrete wavelet transform. Biomed. Sig. Process. Control 8(5), 437–448 (2013)CrossRefGoogle Scholar
  15. 15.
    Rosipal, R., Krämer, N.: Overview and recent advances in partial least squares. In: Saunders, C., Grobelnik, M., Gunn, S., Shawe-Taylor, J. (eds.) SLSFS 2005. LNCS, vol. 3940, pp. 34–51. Springer, Heidelberg (2006). CrossRefGoogle Scholar
  16. 16.
    Sánchez, J., Perronnin, F., Mensink, T., Verbeek, J.: Image classification with the fisher vector: theory and practice. Int. J. Comput. Vis. 105(3), 222–245 (2013)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Schwartz, W., Kembhavi, A., Harwood, D., Davis, L.: Human detection using partial least squares analysis. In: ICCV (2009)Google Scholar
  18. 18.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  19. 19.
    Takemura, A., Shimizu, A., Hamamoto, K.: Discrimination of breast tumors in ultrasonic images using an ensemble classifier based on the adaboost algorithm with feature selection. IEEE Trans. Med. Imaging 29(3), 598–609 (2010)CrossRefGoogle Scholar
  20. 20.
    Uzair, M., Mahmood, A., Mian, A.: Hyperspectral face recognition with spatiospectral information fusion and PLS regression. TIP 24(3), 1127–1137 (2015)MathSciNetGoogle Scholar
  21. 21.
    Wang, X., Han, T.X., Yan, S.: An HOG-LBP human detector with partial occlusion handling. In: ICCV (2009)Google Scholar
  22. 22.
    Wold, H.: Partial least squares. In: Encyclopedia of Statistical Sciences (1985)Google Scholar
  23. 23.
    Wold, S., Esbensen, K., Geladi, P.: Principal component analysis. Chemometr. Intell. Lab. Syst. 2(1–3), 37–52 (1987)CrossRefGoogle Scholar
  24. 24.
    Woźniak, M., Graña, M., Corchado, E.: A survey of multiple classifier systems as hybrid systems. Inf. Fusion 16, 3–17 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Smart Surveillance Interest Group, Department of Computer ScienceUniversidade Federal de Minas GeraisBelo HorizonteBrazil

Personalised recommendations