Abstract
If a binary decision is taken for each classifier in an ensemble, training patterns may be represented as binary vectors. For a two-class supervised learning problem this leads to a partially specified Boolean function that may be analysed in terms of spectral coefficients. In this paper it is shown that a vote which is weighted by the coefficients enables a fast ensemble classifier that achieves performance close to Bayes rate. Experimental evidence shows that effective classifier performance may be achieved with one epoch of training of an MLP using Levenberg-Marquardt with 64 hidden nodes.
Chapter PDF
Similar content being viewed by others
References
Smith, R.S., Windeatt, T.: A Bias-Variance Analysis of Bootstrapped Class-Separability Weighting for ECOC Ensembles. In: Proceedings of the 22nd International Conference on Pattern Recognition, Istanbul, Turkey (August 2010)
Windeatt, T.: Vote Counting Measures for Ensemble Classifiers. Pattern Recognition 36(12), 2743–2756 (2003)
Windeatt, T., Zor, C.: Minimising Added Classification Error using Walsh Coefficients. IEEE Trans. Neural Networks 22(8), 1334–1339 (2011)
Windeatt, T.: Accuracy/ Diversity and ensemble classifier design. IEEE Trans. Neural Networks 17, 1194–1211 (2006)
Hurst, L., Miller, D.M., Muzio, J.: Spectral Techniques in Digital Logic. Academic Press (1985)
Beauchamp, K.G.: Walsh Functions and their Applications. Academic Press (1975)
Tumer, K., Ghosh, J.: Error correlation and error reduction in ensemble classifiers. Connection Science 8(3), 385–404 (1996)
Tikhonov, A.N., Arsenin, V.A.: Solutions of Ill-posed Problems. Winston & Sons, Washington (1977)
Falkowski, B.J., Perkowski, M.A.: Effective Computer Methods for the Calculation of Rademacher-Walsh Spectrum for Completely and Incompletely Specified Boolean Functions. IEEE Trans. on Computer-Aided Design 11(10), 1207–1226 (1992)
Kuncheva, L.I.: Combining Pattern Classifiers. Wiley (2004)
Prechelt, L.: Proben1: A set of neural network Benchmark Problems and Benchmarking Rules. Tech Report 21/94, Univ. Karlsruhe, Germany (September 1994)
Merz, C.J., Murphy, P.M.: UCI repository of ML databases, http://www.ics.uci.edu/~mlearn/MLRepository.html
Breiman, L.: Arcing Classifiers. The Annals of Statistics 26(3), 801–849 (1998)
Freund, Y., Shapire, R.E.: A Decision-Theoretic Generalization of On-line Learning and its Application to Boosting. In: Vitányi, P.M.B. (ed.) EuroCOLT 1995. LNCS, vol. 904, pp. 23–37. Springer, Heidelberg (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Windeatt, T., Zor, C. (2012). Low Training Strength High Capacity Classifiers for Accurate Ensembles Using Walsh Coefficients. In: Gimel’farb, G., et al. Structural, Syntactic, and Statistical Pattern Recognition. SSPR /SPR 2012. Lecture Notes in Computer Science, vol 7626. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34166-3_77
Download citation
DOI: https://doi.org/10.1007/978-3-642-34166-3_77
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34165-6
Online ISBN: 978-3-642-34166-3
eBook Packages: Computer ScienceComputer Science (R0)