A New Random Forest Method for One-Class Classification

  • Chesner Désir
  • Simon Bernard
  • Caroline Petitjean
  • Laurent Heutte
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7626)

Abstract

We propose a new one-class classification method, called One Class Random Forest, that is able to learn from one class of samples only. This method, based on a random forest algorithm and an original outlier generation procedure, makes use of the ensemble learning mechanisms offered by random forest algorithms to reduce both the number of artificial outliers to generate and the size of the feature space in which they are generated. We show that One Class Random Forests perform well on various UCI public datasets in comparison to few other state-of-the-art one class classification methods (gaussian density models, Parzen estimators, gaussian mixture models and one-class SVMs).

Keywords

One-class classification decision trees ensemble methods random forests outlier generation 

References

  1. 1.
    Hempstalk, K., Frank, E., Witten, I.: One-Class Classification by Combining Density and Class Probability Estimation. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part I. LNCS (LNAI), vol. 5211, pp. 505–519. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  2. 2.
    Scholkopf, B., Platt, J., Shawe-Taylor, J., Smola, A., Williamson, R.: Estimating the support of a high-dimensional distribution. Neural Computation 13(7), 1443–1471 (2001)CrossRefGoogle Scholar
  3. 3.
    Tarassenko, L., Clifton, D., Bannister, P., King, S., King, D.: Novelty detection. Encyclopedia of Structural Health Monitoring (2009)Google Scholar
  4. 4.
    Khan, S., Madden, M.: A survey of recent trends in one class classification. Artificial Intelligence and Cognitive Science, 188–197 (2010)Google Scholar
  5. 5.
    Tax, D., Duin, R.: Combining One-Class Classifiers. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 299–308. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  6. 6.
    Dietterich, T.: Ensemble Methods in Machine Learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  7. 7.
    Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)MATHCrossRefGoogle Scholar
  8. 8.
    Robnik-Sikonja, M.: Improving Random Forests. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS (LNAI), vol. 3201, pp. 359–370. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Bernard, S., Heutte, L., Adam, S.: Forest-rk: A new random forest induction method. In: Advanced Intelligent Computing Theories and Applications. With Aspects of Artificial Intelligence, pp. 430–437 (2008)Google Scholar
  10. 10.
    Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Machine Learning 63(1), 3–42 (2006)MATHCrossRefGoogle Scholar
  11. 11.
    Ho, T.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)CrossRefGoogle Scholar
  12. 12.
    Tax, D., Ypma, A., Duin, R.: Support vector data description applied to machine vibration analysis. In: Proc. 5th Annual Conference of the Advanced School for Computing and Imaging, Heijen, NL, Citeseer (1999)Google Scholar
  13. 13.
    Blake, C., Merz, C.: Uci repository of machine learning databases. Department of Information and Computer Science, vol. 55. University of California, Irvine (1998), http://www.ics.uci.edu/~mlearn/mlrepository.html
  14. 14.
    Baldi, P., Brunak, S., Chauvin, Y., Andersen, C., Nielsen, H.: Assessing the accuracy of prediction algorithms for classification: an overview. Bioinformatics 16(5), 412–424 (2000)CrossRefGoogle Scholar
  15. 15.
    Duin, R.: PRTools version 3.0: A matlab toolbox for pattern recognition. In: Proc. of SPIE, Citeseer (2000)Google Scholar
  16. 16.
    Bernard, S., Heutte, L., Adam, S.: Influence of Hyperparameters on Random Forest Accuracy. In: Benediktsson, J.A., Kittler, J., Roli, F. (eds.) MCS 2009. LNCS, vol. 5519, pp. 171–180. Springer, Heidelberg (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Chesner Désir
    • 1
  • Simon Bernard
    • 2
  • Caroline Petitjean
    • 1
  • Laurent Heutte
    • 1
  1. 1.Université de Rouen, LITIS EA 4108Saint-Etienne-du-RouvrayFrance
  2. 2.Department of EECS et GIGA-ResearchUniversité de LiègeLiègeBelgium

Personalised recommendations