Relaxation of Hard Classification Targets for LSE Minimization
In the spirit of stabilizing a solution to handle possible over-fitting of data which is especially common for high order models, we propose a relaxed target training method for regression models which are linear in parameters. This relaxation of training target from the conventional binary values to disjoint classification spaces provides good classification fidelity according to a threshold treatment during the decision process. A particular design to relax the training target is provided under practical consideration. Extension to multiple class problems is formulated before the method is applied to a plug-in full multivariate polynomial model and a reduced model on synthetic data sets to illustrate the idea. Additional experiments were performed using real-world data from the UCI data repository to derive certain empirical evidence.
KeywordsPattern Classification Parameter Estimation Pattern Recognition Multivariate Polynomials and Machine Learning
Unable to display preview. Download preview PDF.
- 1.Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases, University of California, Irvine, Dept. of Information and Computer Sciences (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
- 3.Schürmann, J.: Pattern Classification: A Unified View of Statistical and Neural Approaches. John Wiley & Sons, Inc., New York (1996)Google Scholar
- 5.Baram, Y.: Soft Nearest Neighbor Classification. In: International Conference on Neural Networks (ICNN), vol. 3, pp. 1469–1473 (1997)Google Scholar
- 7.Toh, K.-A., Yau, W.-Y., Jiang, X.: A Reduced Multivariate Polynomial Model For Multimodal Biometrics And Classifiers Fusion. IEEE Trans. Circuits and Systems for Video Technology (Special Issue on Image- and Video-Based Biometrics) 14(2), 224–233 (2004)Google Scholar
- 10.Tipping, M.E.: The Relevance Vector machine. In: Solla, S.A., Leen, T.K., Müller, K.-R. (eds.) Advances in Neural Information Processing Systems, vol. 12, pp. 652–658 (2000)Google Scholar
- 12.The MathWorks, Matlab And Simulink (2003), http://www.mathworks.com/
- 13.Ma, J., Zhao, Y., Ahalt, S.: OSU SVM Classifier Matlab Toolbox (ver 3.00), The Ohio State University (2002), http://eewww.eng.ohio-state.edu/~maj/osu_svm/
- 14.Tipping, M.: Sparse Bayesian Learning and the Relevance Vector Machine, Microsoft Research (2004), http://research.microsoft.com/mlp/RVM