Advertisement

1-vs-Others Rough Decision Forest

  • Jinmao Wei
  • Shuqin Wang
  • Guoying Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6401)

Abstract

Bootstrap, boosting and subspace are popular techniques for inducing decision forests. In all the techniques, each single decision tree is induced in the same way as that for inducing a decision tree on the whole data, in which all possible classes are dealt with together. In such induced trees, some minority classes may be covered up by others when some branches grow or are pruned. For a multi-class problem, this paper proposes to induce individually the 1-vs-others rough decision trees for all classes, and finally construct a rough decision forest, intending to reduce the possible side effects of imbalanced class distribution. Since all training samples are reused to construct the rough decision trees for all classes, the method also tends to have the merits of bootstrap, boosting and subspace. Experimental results and comparisons on some hard gene expression data show the attractiveness of the method.

Keywords

Classification Decision tree Decision forest Rough set 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Technical report. Wadsworth International, Monterey, CA (1984)Google Scholar
  2. 2.
    Quinlan, J.R.: Introduction of Decision Trees. Machine Learning 3, 81–106 (1986)Google Scholar
  3. 3.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
  4. 4.
    Quinlan, J.R., Rivest, R.: Inferring Decision Trees Using the Minimum Description Length Principle. Information and Computation 80, 227–248 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Fayyad, U.M., Irani, K.B.: On the Handling of Continuous-valued Attributes in Decision Tree Generation. Machine Learning 8, 87–102 (1992)zbMATHGoogle Scholar
  6. 6.
    Breiman, L.: Bagging Predictors. Machine Learning 24, 123–140 (1996)zbMATHMathSciNetGoogle Scholar
  7. 7.
    Breiman, L.: Random Forests. Machine Learning 45, 5–32 (2001)zbMATHCrossRefGoogle Scholar
  8. 8.
    Dietterich, T.G., Kong, E.B.: Machine Learning Bias, Statistical Bias and Statistical Variance of Decision Tree Algorithms. Technical Report. Oregon State University (1995)Google Scholar
  9. 9.
    Freund, Y., Schapire, R.E.: A Decision-theoretic Generalization of On-line Learning and an Application to Boosting. Journal of Computer and System Sciences 55, 119–139 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Quinlan, J.R.: Bagging, Boosting, and C4.5. In: Proceedings, Fourteenth National Conference on Artificial Intelligence (1996)Google Scholar
  11. 11.
    Ho, T.K.: The Random Subspace Method for Constructing Decision Forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 832–844 (1998)CrossRefGoogle Scholar
  12. 12.
    Pawlak, Z.: Rough sets. International Journal of Computer and Information Science 11, 341–356 (1982)zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Wei, J.M.: Rough Set Based Approach to Selection of Node. International Journal of Computational Cognition 1, 25–40 (2003)Google Scholar
  14. 14.
    Wei, J.M., Wang, S.Q., et al.: Rough Set Based Approach for Inducing Decision Trees. Knowledge-Based Systems 20, 695–702 (2007)CrossRefGoogle Scholar
  15. 15.
    Yellasiri, R., Rao, C.R., Reddy, V.: Decision Tree Induction Using Rough Set Theory: Comparative Study. Journal of Theoretical and Applied Information Technology 3, 110–114 (2005)Google Scholar
  16. 16.
    Minz, S., Jain, R.: Rough Set Based Decision Tree Model for Classification. In: Kambayashi, Y., Mohania, M., Wöß, W. (eds.) DaWaK 2003. LNCS, vol. 2737, pp. 172–181. Springer, Heidelberg (2003)Google Scholar
  17. 17.
    Minz, S., Jain, R.: Refining Decision Tree Classifiers Using Rough Set Tools. International Journal of Hybrid Intelligent Systems 2, 133–148 (2005)zbMATHGoogle Scholar
  18. 18.
    Tan, A.C., Naiman, D.Q., Xu, L., Winclow, R.L., Geman, D.: Simple Decision Rules for Classifying Human Cancers from Gene Expression Profiles. Bioinformatics 21, 3896–3904 (2005)CrossRefGoogle Scholar
  19. 19.
    Khan, J., et al.: Classification and Diagnostic Prediction of Cancers Using Gene Expression Profiling and Artificial Neural Networks. Nat. Med. 7, 673–679 (2001)CrossRefGoogle Scholar
  20. 20.
    Ramaswamy, S., et al.: Multiclass Cancer Diagnosis Using Tumor Gene Expression Signatures. Proc. Natl. Acad. Sci. USA 98, 15149–15154 (2001)CrossRefGoogle Scholar
  21. 21.
    Su, A.I., et al.: Molecular Classification of Human Carcinomas by Use of Gene Expression Signatures. Cancer Res. 61, 7388–7393 (2001)Google Scholar
  22. 22.
    Perou, C.M., et al.: Molecular Portraits of Human Breast Tumours. Nature 406, 747–752 (2000)CrossRefGoogle Scholar
  23. 23.
    Wei, J.M., Wang, G.Y., Kong, X.M., Li, S.J., Wang, S.Q.: A New Method for Discretization of Continuous Attributes Based on VPRS. LNCS, pp. 183–190. Springer, Heidelberg (2006)Google Scholar
  24. 24.
    Wei, J.M., Wang, S.Q., Yu, G., Gu, L., Wang, G.Y., Yuan, X.J.: A Novel Method for Pruning Decision Trees. In: ICMLC 2009, pp. 339–343 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Jinmao Wei
    • 1
  • Shuqin Wang
    • 2
  • Guoying Wang
    • 1
  1. 1.College of Information Technical ScienceNankai UniversityTianjinChina
  2. 2.College of Computer and Information EngineeringTianjin Normal UniversityTianjinChina

Personalised recommendations