Linear components of quadratic classifiers
We obtain a decomposition of any quadratic classifier in terms of products of hyperplanes. These hyperplanes can be viewed as relevant linear components of the quadratic rule (with respect to the underlying classification problem). As an application, we introduce the associated multidirectional classifier; a piecewise linear classification rule induced by the approximating products. Such a classifier is useful to determine linear combinations of the predictor variables with ability to discriminate. We also show that this classifier can be used as a tool to reduce the dimension of the data and helps identify the most important variables to classify new elements. Finally, we illustrate with a real data set the use of these linear components to construct oblique classification trees.
KeywordsSupervised classification Fisher linear discriminant analysis Quadratic discriminant analysis Reduction of the dimension Feature extraction Oblique classification trees
Mathematics Subject Classification62H30
The authors are grateful to the reviewers and the associate editor for their insightful comments which have improved the presentation of the paper. We would like to thank Jesús María Arregui (University of the Basque Country) and Jesús Gonzalo (Universidad Autónoma de Madrid) with whom we have shared illuminating conversations on quadratic forms.
- Bache K, Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml
- Golub GH, Van Loan CF (2013) Matrix computations, 4th edn. Johns Hopkins studies in the mathematical sciences. Johns Hopkins University Press, BaltimoreGoogle Scholar
- Park SH, Fürnkranz J (2007) Efficient pairwise classification. In: European conference on machine learning. Springer, pp 658–665Google Scholar
- R Core Team (2016) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. http://www.R-project.org/
- Ripley B (2014) Tree: classification and regression trees. R package version 1.0-35. http://CRAN.R-project.org/package=tree
- Truong A (2009) Fast growing and interpretable oblique trees via logistic regression models. Doctoral dissertation, University of OxfordGoogle Scholar