Abstract
We propose a fast fixed-point algorithm to improve the Relevant Component Analysis (RCA) in two-class cases. Using an objective function that maximizes the predictive information, our method is able to extract more than one discriminative component of data for two-class problems, which cannot be accomplished by classical Fisher’s discriminant analysis. After prewhitening the data, we apply Newton’s optimization method which automatically chooses the learning rate in the iterative training of each component. The convergence of the iterative learning is quadratic, i.e. much faster than the linear optimization by gradient methods. Empirical tests presented in the paper show that feature extraction using the new method resembles RCA for low-dimensional ionosphere data and significantly outperforms the latter in efficiency for high-dimensional facial image data.
Supported by the Academy of Finland in the projects Neural methods in information retrieval based on automatic content analysis and relevance feedback and Finnish Centre of Excellence in Adaptive Informatics Research.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Blake, C.L., Newman, D.J., Hettich, S., Merz, C.J.: UCI repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Duin, R.P.W., Loog, M.: Linear dimensionality reduction via a heteroscedastic extension of lda: the chernoff criterion. IEEE Transaction on Pattern Analysis and Machine Intelligence 26(6), 732–739 (2004)
Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annals of Eugenics 7 (1963)
Howland, P., Park, H.: Generalizing discriminant analysis using the generalized singular value decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(8), 995–1006 (2005)
Peltonen, J., Kaski, S.: Discriminative components of data. IEEETransactions on Neural Networks 16(1), 68–83 (2005)
Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face recognition algorithms. IEEE Trans. Pattern Analysis andMachine Intelligence 22, 1090–1104 (2000)
Xu, Y., Yang, J.Y., Jin, Z.: Theory analysis on FSLDA and ULDA. Pattern Recognition 36(12), 3031–3033 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yang, Z., Laaksonen, J. (2006). A Fast Fixed-Point Algorithm for Two-Class Discriminative Feature Extraction. In: Kollias, S., Stafylopatis, A., Duch, W., Oja, E. (eds) Artificial Neural Networks – ICANN 2006. ICANN 2006. Lecture Notes in Computer Science, vol 4132. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11840930_34
Download citation
DOI: https://doi.org/10.1007/11840930_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-38871-5
Online ISBN: 978-3-540-38873-9
eBook Packages: Computer ScienceComputer Science (R0)