Abstract
Finding a hyperplane that separates two classes of data points with the minimum number of misclassifications is directly related to the following problem in linear programming: given an infeasible set of linear constraints, find the smallest number of constraints to remove such that the remaining constraints constitute a feasible set (the Maximum Feasible Subsystem problem). This relationship underlies an effective heuristic method for finding separating hyperplanes in classification problems [Chinneck 2001]. This paper shows how to tailor the maximum feasible subsystem hyperplane placement heuristic so that it can provide good values for metrics other than total accuracy. The concepts are demonstrated using accuracy-related metrics such as precision and recall, balancing the population accuracies, and balancing the accuracies on each side of the hyperplane, but the principles also apply to other metrics such as the Gini index, entropy, etc. Customizations such as these may prove useful in developing better decision trees.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amaldi E (1994) From Finding Maximum Feasible Subsystems Of Linear Systems To Feedforward Neural Network Design. Ph.D. thesis no. 1282, Département de Mathématiques, école Polytechnique Fédérale de Lausanne, Switzerland.
Amaldi E, Kann V (1995) The Complexity And Approximability Of Finding Maximum Feasible Subsystems Of Linear Relations, Theoretical Computer Science 147:181–210.
Bennett KP, Bredensteiner E (1997) A Parametric Optimization Method for Machine Learning, INFORMS J. on Computing 9:311–318.
Bennett KP, Mangasarian OL (1992) Neural Network Training via Linear Programming in Pardalos PM (ed.)Advances in Optimization and Parallel Computing, North Holland, Amsterdam, 56–67.
Blake CL, Merz CJ (1998) UCI Repository Of Machine Learning Databases. Department of Information and Computer Science, University of California, Irvine, CA. http://www.ics.uci.edu/?mlearn/MLRepository.html.
Brown G, Graves G (1975) Elastic Programming: A New Approach To Large-Scale Mixed Integer Optimisation, ORSA/TIMS conference, Las Vegas.
Chakravarti N (1994) Some Results Concerning Post-Infeasibility Analysis, European Journal of Operations Research 73:139–143.
Chinneck JW (1996) An Effective Polynomial-Time Heuristic for the Minimum-Cardinality IIS Set-Covering Problem, Annals of Mathematics and Artificial Intelligence 17:127–144.
Chinneck JW (2001) Fast Heuristics for the Maximum Feasible Subsystem Problem, INFORMS Journal on Computing 13:210–223.
Cristianini N, Shawe-Taylor J (2000) An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods, Cambridge University Press.
Flach PA (2003) The Geometry of ROC Space: Understanding Machine Learning Metrics through ROC Isometrics, Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003), Washington D.C.
Fürnkranz J, Flach PA (2003) An Analysis of Rule Evaluation Metrics, Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003), Washington D.C.
Glover F (1990) Improved Linear Programming Models for Discriminant Analysis, Decision Sciences 21:771–785.
Metz CE (1978) Basic Principles of ROC Analysis, Seminars in Nuclear Medicine 8:283–298.
Parker MR (1995) A set covering approach to infeasibility analysis of linear programming problems and related issues. Ph.D. thesis, Dept. of Mathematics, University of Colorado at Denver, Denver, Colorado.
Provost F, Fawcett T (2001) Robust Classification for Imprecise Environments, Machine Learning 42:203–231.
Rokach L, Maimon O (2008) Data Mining with Decision Trees: Theory and Applications, World Scientific, New Jersey.
Sankaran JK (1993) A Note On Resolving Infeasibility In Linear Programs By Constraint Relaxation, Operations Research Letters 13:19–20.
Xiao, B (1993) Necessary and Sufficient Conditions of Unacceptable Solutions in LP Discriminant Analysis, Decision Sciences 24:699–712.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer Science+Business Media, LLC
About this paper
Cite this paper
Chinneck, J.W. (2009). Tailoring Classifier Hyperplanes to General Metrics. In: Chinneck, J.W., Kristjansson, B., Saltzman, M.J. (eds) Operations Research and Cyber-Infrastructure. Operations Research/Computer Science Interfaces, vol 47. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-88843-9_19
Download citation
DOI: https://doi.org/10.1007/978-0-387-88843-9_19
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-88842-2
Online ISBN: 978-0-387-88843-9
eBook Packages: Computer ScienceComputer Science (R0)