A Comparison of Generic Machine Learning Algorithms for Image Classification

  • Raphaël Marée
  • Pierre Geurts
  • Giorgio Visimberga
  • Justus Piater
  • Louis Wehenkel


In this paper, we evaluate 7 machine learning algorithms for image classification including our recent approach that combines building of ensembles of extremely randomized trees and extraction of sub-windows from the original images. For the approach to be generic, all these methods are applied directly on pixel values without any feature extraction. We compared them on four publicly available datasets corresponding to representative applications of image classification problems: handwritten digits (MNIST), faces (ORL), 3D objects (COIL-IOO), and textures (OUTEX). A comparison with studies from the computer vision literature shows that generic methods can come remarkably close to specialized methods. In particular, our sub-window algorithm is competitive with the state of the art, a remarkable result considering its generality and conceptual simplicity.


Support Vector Machine Random Forest Face Recognition Image Classification Machine Learning Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    L. Breiman. Bagging predictors. Machine Learning, 24(2):123–140, 1996.MathSciNetMATHGoogle Scholar
  2. [2]
    L. Breiman. Random forests. Machine learning, 45:5–32, 2001.MATHCrossRefGoogle Scholar
  3. [3]
    L. Breiman, J.H. Friedman, R.A. Olsen, and C.J. Stone. Classification and Regression Trees. Wadsworth International (California), 1984.MATHGoogle Scholar
  4. [4]
    C.J.C. Burges and B. Schölkopf, Improving the accuracy and speed of support vector machines. In M. C. Mozer, M. I. Jordan, and Thomas Petsche, editors, Advances in Neural Information Processing Systems, volume 9, page 375. The MIT Press, 1997.Google Scholar
  5. [5]
    Chih-Chung Chang and Chih-Jen Lin. Libsvm: a library for support vector machines. Technical report, Computer Science and Information Engineering, National Taiwan University, 2003.Google Scholar
  6. [6]
    J. Dahmen, D. Keysers, and H. Ney. Combined classification of handwritten digits using the ‘virtual test sample method’. In Proc. Second International Workshop, MCS 2001 Cambridge, UK, pages 99–108, July 2001.Google Scholar
  7. [7]
    H. Drucker. Fast decision tree ensembles for optical character recognition. In Proc. Fifth Annual Symposium on Document Analysis and Information Retrieval, pages 137–147, 1996.Google Scholar
  8. [8]
    Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. In Proc. Thirteenth International Conference on Machine Learning, pages 148–156, 1996.Google Scholar
  9. [9]
    P. Geurts. Contributions to decision tree induction: bias/variance tradeoff and time series classification. Phd. thesis, Department of Electrical Engineering and Computer Science, University of Liège, May 2002.Google Scholar
  10. [10]
    P. Geurts. Extremely randomized trees. Technical report, Department of Electrical Engineering and Computer Science, University of Liège, 2003.Google Scholar
  11. [11]
    G.-D. Guo, S. Li, and K. Chan. Face recognition by support vector machines. In Proc. International Conference on Automatic Face and Gesture Recognition, 196–201., 2000.Google Scholar
  12. [12]
    G.-D. Guo and H.-J. Zhang. Boosting for fast face recognition. In Proc. IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pages 96–100, 200l.Google Scholar
  13. [13]
    M.S. Hoque and M. C. Fairhurst. A moving window classifier for offline character recognition. In Proc. of the Seventh International Workshop on Frontiers in Handwriting Recognition, Amsterdam, pages 595–600, September 2000.Google Scholar
  14. [14]
    T.O. Kvalseth. Entropy and correlation: Some comments. IEEE Trans. on Systems, Man and Cybernetics, SMC-17(3):517–519, 1987.CrossRefGoogle Scholar
  15. [15]
    S. Lawrence, C. Lee Giles, A. C. Tsoi, and A. D. Back. Face recognition: A convolutional neural network approach. IEEE Transactions on Neural Networks, 8(1):98–113, 1997.CrossRefGoogle Scholar
  16. [16]
    Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proc. of the IEEE, 86(11):2278–2324, 1998.CrossRefGoogle Scholar
  17. [17]
    R. Maree, Geurts P., Piater J., and Wehenkel L. A generic approach for image classification based on decision tree ensembles. Submitted.Google Scholar
  18. [18]
    T. Menp, M. Pietikinen, and J. Viertola. Separating color and pattern information for color texture discrimination. In Proc. 16th International Conference on Pattern Recognition, 2002.Google Scholar
  19. [19]
    A. Nefian and M. Hayes. Face recognition using an embedded HMM. In Proc. IEEE Conference on Audio and Video-based Biometric Person Authentication, pages 19–24, March 1999.Google Scholar
  20. [20]
    S. Obrzalek and J. Matas. Object recognition using local affine frames on distinguished regions. In Electronic Proceedings of the 13th British Machine Vision Conference, University of Cardiff, 2002.Google Scholar
  21. [21]
    T. Ojala, T. Menp, M. Pietikinen, J. Viertola, J. Kyllnen, and S. Huovinen. Outex-new framework for empirical evaluation of texture analysis algorithms. Proc. 16th International Conference on Pattern Recognition, Quebec, Canada, 1:701–706, 2002.Google Scholar
  22. [22]
    R. Paredes and A. Perez-Cortes. Local representations and a direct voting scheme for face recognition. In Pattern Recognition in Information Systems, Proc. 1st International Workshop on Pattern Recognition in Information Systems, pages 71–79, July 2001.Google Scholar
  23. [23]
    M. Pontil and A. Verri. Support vector machines for 3d object recognition. IEEE 1hmsactions on Pattern Analysis and Machine Intelligence, 20(6):637–646, 1998.CrossRefGoogle Scholar
  24. [24]
    V.N. Vapnik. The nature of statistical learning theory. Springer Verlag, 1995.Google Scholar
  25. [25]
    L. Wehenkel. Automatic learning techniques in power systems. Kluwer Academic, Boston, 1998.MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2004

Authors and Affiliations

  • Raphaël Marée
    • 1
  • Pierre Geurts
    • 1
  • Giorgio Visimberga
    • 2
  • Justus Piater
    • 1
  • Louis Wehenkel
    • 1
  1. 1.Montefiore InstituteUniversity of LiègeLiègeBelgium
  2. 2.Politecnico Di BariBariItaly

Personalised recommendations