Skip to main content
SpringerLink
Log in

A Qualitative Evaluation of Random Forest Feature Learning

  • Conference paper
Recent Advances on Soft Computing and Data Mining

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 287))

Abstract

Feature learning is a hot trend in the machine learning community now. Using a random forest in feature learning is a relatively unexplored area compared to its application in classification and regression. This paper aims to show the characteristics of the features learned by a random forest and its connections with other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 25, pp. 1106–1114 (2012)

    Google Scholar 

  2. Seide, F., Li, G., Yu, D.: Conversational speech transcription using context-dependent deep neural networks. In: Proc. Interspeech, vol. 11, pp. 437–440 (2011)

    Google Scholar 

  3. Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  4. Criminisi, A.: Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning. Foundations and Trends in Computer Graphics and Vision 7(2-3), 81–227 (2011)

    Article  MATH  Google Scholar 

  5. Lee, H., et al.: Efficient sparse coding algorithms. In: Advances in Neural Information Processing Systems, pp. 801–808 (2006)

    Google Scholar 

  6. Raina, R., et al.: Self-taught learning: Transfer learning from unlabeled data. In: Proceedings of the Twenty-fourth International Conference on Machine Learning (2007)

    Google Scholar 

  7. Bengio, Y.: Learning deep architectures for AI. Foundations and Trends in Machine Learning 2(1), 1–127 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  8. Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Computation 18(7), 1527–1554 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  9. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition Proceedings of the IEEE, Vol. Proceedings of the IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  10. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. Tech. rep. DTIC Document (1985)

    Google Scholar 

  11. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  12. Salakhutdinov, R., Mnih, A., Hinton, G.: Restricted Boltzmann machines for collaborative filtering. In: Proceedings of the 24th International Conference on Machine Learning, pp. 791–798. ACM (2007)

    Google Scholar 

  13. Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research 37(23), 3311–3325 (1997)

    Article  Google Scholar 

  14. Hinton, G.E.: Learning distributed representations of concepts. In: Proceedings of the Eighth Annual Conference of the Cognitive Science Society, Amherst, MA, pp. 1–12 (1986)

    Google Scholar 

  15. Bengio, Y., et al.: Neural probabilistic language models. In: Holmes, D.E., Jain, L.C. (eds.) Innovations in Machine Learning. STUDFUZZ, vol. 194, pp. 137–186. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  16. Bengio, Y., Delalleau, O., Simard, C.: Decision trees do not generalize to new variations. Computational Intelligence 26(4), 449–467 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  17. Moosmann, F., Nowak, E., Jurie, F.: Randomized clustering forests for image classification. Pattern Analysis and Machine Intelligence 30(9), 1632–1646 (2008)

    Article  Google Scholar 

  18. Vens, C., Costa, F.: Random Forest Based Feature Induction. In: 2011 IEEE 11th International Conference on Data Mining (ICDM), pp. 744–753. IEEE (2011)

    Google Scholar 

  19. Breiman, L.: Classification and regression trees. CRC Press (1993)

    Google Scholar 

  20. Quinlan, J.R.: C4. 5: Programs for machine learning, vol. 1. Morgan Kaufmann (1993)

    Google Scholar 

  21. Amit, Y., Geman, D.: Randomized Inquiries About Shape: An Application to Handwritten Digit Recognition. Tech. rep. DTIC Document (1994)

    Google Scholar 

  22. Amit, Y., Geman, D.: Shape quantization and recognition with randomized trees. Neural Computation 9(7), 1545–1588 (1997)

    Article  Google Scholar 

  23. Ho, T.K.: Random decision forests. In: Proceedings of the Third International Conference on Document Analysis and Recognition, vol. 1, pp. 278–282. IEEE (1995)

    Google Scholar 

  24. Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)

    MATH  MathSciNet  Google Scholar 

  25. Bosch, A., Zisserman, A., Muoz, X.: Image classification using random forests and ferns. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–8. IEEE (2007)

    Google Scholar 

  26. Criminisi, A., Shotton, J., Robertson, D., Konukoglu, E.: Regression forests for efficient anatomy detection and localization in CT studies. In: Menze, B., Langs, G., Tu, Z., Criminisi, A., et al. (eds.) MICCAI 2010. LNCS, vol. 6533, pp. 106–117. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  27. Fanelli, G., Gall, J., Van Gool, L.: Real time head pose estimation with random regression forests. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 617–624. IEEE (2011)

    Google Scholar 

  28. Geremia, E., Menze, B.H., Clatz, O., Konukoglu, E., Criminisi, A., Ayache, N.: Spatial decision forests for MS lesion segmentation in multi-channel MR images. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A., et al. (eds.) MICCAI 2010, Part I. LNCS, vol. 6361, pp. 111–118. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  29. Leistner, C., et al.: Semi-supervised random forests. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 506–513. IEEE (2009)

    Google Scholar 

  30. Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Machine Learning 63(1), 3–42 (2006)

    Article  MATH  Google Scholar 

  31. Coates, A., Ng, A.: The Importance of Encoding Versus Training with Sparse Coding and Vector Quantization. In: Proceedings of the 28th International Conference on Machine Learning, pp. 921–928 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Computer Science & Networked Systems, Sunway University, No. 5 Jalan Universiti, Bandar Sunway, 46150, Petaling Jaya, Selangor Darul Ehsan, Malaysia

    Adelina Tang & Joan Tack Foong

Authors
  1. Adelina Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joan Tack Foong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adelina Tang .

Editor information

Editors and Affiliations

  1. Department of Information System Faculty of Comp. Sci. & Info. Tech., University of Malaya, Kuala Lumpur, Malaysia

    Tutut Herawan

  2. Faculty of Comp. Sci. and Info. Tech, Universiti Tun Hussein Onn Malaysia, Parit Raja, Malaysia

    Rozaida Ghazali

  3. Faculty of Comp. Sci. and Info. Tech., Universiti Tun Hussein Onn Malaysia, Parit Raja, Malaysia

    Mustafa Mat Deris

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Tang, A., Foong, J.T. (2014). A Qualitative Evaluation of Random Forest Feature Learning. In: Herawan, T., Ghazali, R., Deris, M. (eds) Recent Advances on Soft Computing and Data Mining. Advances in Intelligent Systems and Computing, vol 287. Springer, Cham. https://doi.org/10.1007/978-3-319-07692-8_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-07692-8_34

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-07691-1

  • Online ISBN: 978-3-319-07692-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation