Improving ANNs Performance on Unbalanced Data with an AUC-Based Learning Algorithm
This paper investigates the use of the Area Under the ROC Curve (AUC) as an alternative criteria for model selection in classification problems with unbalanced datasets. A novel algorithm, named here as AUCMLP, which incorporates AUC optimization into the Multi-layer Perceptron (MLPs) learning process is presented. The basic principle of AUCMLP is the solution of an optimization problem that aims at ranking quality as well as the separability of class distributions with respect to the threshold decision. Preliminary results achieved on real data, point out that our approach is promising, and can lead to better decision surfaces, specially under more severe unbalance conditions.
Keywordsunbalanced datasets classification Area Under the ROC Curve parameter estimation criteria
Unable to display preview. Download preview PDF.
- 1.Rumelhart, D.E., McClelland, J.L.: Parallel distributed processing: Explorations in the microstructure of cognition, vol. 1: Foundations. MIT Press (1986)Google Scholar
- 6.Cortes, C., Mohri, M.: AUC optimization vs. error rate minimization. In: Advances in Neural Information Processing Systems 16. MIT Press, Cambridge (2004)Google Scholar
- 7.Yan, L., Dodier, R.H., Mozer, M., Wolniewicz, R.H.: Optimizing classifier performance via an approximation to the wilcoxon-mann-whitney statistic. In: ICML 2003: Proceedings of the 20th Int. Conf. on Machine Learning, pp. 848–855 (2003)Google Scholar
- 8.Joachims, T.: A support vector method for multivariate performance measures. In: ICML 2005: Proc. of the 22nd Int. Conf. on Machine learning, pp. 377–384 (2005)Google Scholar
- 9.Herschtal, A., Raskutti, B., Campbell, P.K.: Area under ROC optimization using a ramp approximation. In: Proc. of 6th Int. Conf. on Data Mining, pp. 1–11 (2006)Google Scholar
- 12.Hanley, J.A., Mcneil, B.J.: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143, 29–36 (1982)Google Scholar
- 15.UCI machine learning repository, http://archive.ics.uci.edu/ml/