Advertisement

Low Training Strength High Capacity Classifiers for Accurate Ensembles Using Walsh Coefficients

  • Terry Windeatt
  • Cemre Zor
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7626)

Abstract

If a binary decision is taken for each classifier in an ensemble, training patterns may be represented as binary vectors. For a two-class supervised learning problem this leads to a partially specified Boolean function that may be analysed in terms of spectral coefficients. In this paper it is shown that a vote which is weighted by the coefficients enables a fast ensemble classifier that achieves performance close to Bayes rate. Experimental evidence shows that effective classifier performance may be achieved with one epoch of training of an MLP using Levenberg-Marquardt with 64 hidden nodes.

Keywords

Ensembles Multilayer Perceptrons Boolean Function Walsh Coefficients 

References

  1. 1.
    Smith, R.S., Windeatt, T.: A Bias-Variance Analysis of Bootstrapped Class-Separability Weighting for ECOC Ensembles. In: Proceedings of the 22nd International Conference on Pattern Recognition, Istanbul, Turkey (August 2010)Google Scholar
  2. 2.
    Windeatt, T.: Vote Counting Measures for Ensemble Classifiers. Pattern Recognition 36(12), 2743–2756 (2003)zbMATHCrossRefGoogle Scholar
  3. 3.
    Windeatt, T., Zor, C.: Minimising Added Classification Error using Walsh Coefficients. IEEE Trans. Neural Networks 22(8), 1334–1339 (2011)CrossRefGoogle Scholar
  4. 4.
    Windeatt, T.: Accuracy/ Diversity and ensemble classifier design. IEEE Trans. Neural Networks 17, 1194–1211 (2006)CrossRefGoogle Scholar
  5. 5.
    Hurst, L., Miller, D.M., Muzio, J.: Spectral Techniques in Digital Logic. Academic Press (1985)Google Scholar
  6. 6.
    Beauchamp, K.G.: Walsh Functions and their Applications. Academic Press (1975)Google Scholar
  7. 7.
    Tumer, K., Ghosh, J.: Error correlation and error reduction in ensemble classifiers. Connection Science 8(3), 385–404 (1996)CrossRefGoogle Scholar
  8. 8.
    Tikhonov, A.N., Arsenin, V.A.: Solutions of Ill-posed Problems. Winston & Sons, Washington (1977)zbMATHGoogle Scholar
  9. 9.
    Falkowski, B.J., Perkowski, M.A.: Effective Computer Methods for the Calculation of Rademacher-Walsh Spectrum for Completely and Incompletely Specified Boolean Functions. IEEE Trans. on Computer-Aided Design 11(10), 1207–1226 (1992)CrossRefGoogle Scholar
  10. 10.
    Kuncheva, L.I.: Combining Pattern Classifiers. Wiley (2004)Google Scholar
  11. 11.
    Prechelt, L.: Proben1: A set of neural network Benchmark Problems and Benchmarking Rules. Tech Report 21/94, Univ. Karlsruhe, Germany (September 1994)Google Scholar
  12. 12.
    Merz, C.J., Murphy, P.M.: UCI repository of ML databases, http://www.ics.uci.edu/~mlearn/MLRepository.html
  13. 13.
    Breiman, L.: Arcing Classifiers. The Annals of Statistics 26(3), 801–849 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Freund, Y., Shapire, R.E.: A Decision-Theoretic Generalization of On-line Learning and its Application to Boosting. In: Vitányi, P.M.B. (ed.) EuroCOLT 1995. LNCS, vol. 904, pp. 23–37. Springer, Heidelberg (1995)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Terry Windeatt
    • 1
  • Cemre Zor
    • 1
  1. 1.Univ SurreyGuildfordUK

Personalised recommendations