Assessments Metrics for Multi-class Imbalance Learning: A Preliminary Study

  • R. Alejo
  • J. A. Antonio
  • R. M. Valdovinos
  • J. H. Pacheco-Sánchez
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7914)


In this paper we study some of the most common global measures employed to measure the classifier performance on the multi-class imbalanced problems. The aim of this work consists of showing the relationship between global classifier performance (measure by global measures) and partial classifier performance, i.e., to determine if the results of global metrics match with the improved classifier performance over the minority classes. We have used five strategies to deal with the class imbalance problem over five real multi-class datasets on neural networks context.


Multi-class imbalance global measures accuracy by class 


  1. 1.
    Tsoumakas, G., Katakis, I.: Multi-label classification: An overview. Int. J. Data Warehousing and Mining, 1–13 (2007)Google Scholar
  2. 2.
    Ou, G., Murphey, Y.L.: Multi-class pattern classification using neural networks. Pattern Recognition 40(1), 4–18 (2007)zbMATHCrossRefGoogle Scholar
  3. 3.
    Wang, S., Yao, X.: Multi-class imbalance problems: Analysis and potential solutions. IEEE Transactions on IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics (99), 1–12 (2012)Google Scholar
  4. 4.
    Zhou, Z.H., Liu, X.Y.: Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Transactions on Knowledge and Data Engineering 18, 63–77 (2006)CrossRefGoogle Scholar
  5. 5.
    Pérez-Godoy, M.D., Fernández, A., Rivera, A.J., del Jesus, M.J.: Analysis of an evolutionary rbfn design algorithm, co2rbfn, for imbalanced data sets. Pattern Recogn. Lett. 31(15), 2375–2388 (2010)CrossRefGoogle Scholar
  6. 6.
    He, H., Garcia, E.: Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering 21(9), 1263–1284 (2009)CrossRefGoogle Scholar
  7. 7.
    García, V., Mollineda, R.A., Sánchez, J.S.: Theoretical analysis of a performance measure for imbalanced data. In: ICPR, pp. 617–620 (2010)Google Scholar
  8. 8.
    Ferri, C., Hernández-Orallo, J., Modroiu, R.: An experimental comparison of performance measures for classification. Pattern Recognition Letter 30(1), 27–38 (2009)CrossRefGoogle Scholar
  9. 9.
    Fawcett, T.: An introduction to roc analysis. Pattern Recogn. Lett. 27, 861–874 (2006)CrossRefGoogle Scholar
  10. 10.
    Bruzzone, L., Serpico, S.: Classification of imbalanced remote-sensing data by neural networks. Pattern Recognition Letters 18, 1323–1328 (1997)CrossRefGoogle Scholar
  11. 11.
    A. Asuncion, D.N.: UCI machine learning repository (2007)Google Scholar
  12. 12.
    Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artif. Intell. 97(1-2), 273–324 (1997)zbMATHCrossRefGoogle Scholar
  13. 13.
    Weiss, G.M., Provost, F.J.: Learning when training data are costly: The effect of class distribution on tree induction. J. Artif. Intell. Res. (JAIR) 19, 315–354 (2003)Google Scholar
  14. 14.
    Wilson, D.: Asymptotic properties of nearest neighbor rules using edited data. IEEE Transactions on Systems, Man and Cybernetics 2(4), 408–420 (1972)zbMATHCrossRefGoogle Scholar
  15. 15.
    Sánchez, J.S., Pla, F., Ferri, F.J.: Prototype selection for the nearest neighbour rule through proximity graphs. Pattern Recognition Letters 18(6), 507–513 (1997)CrossRefGoogle Scholar
  16. 16.
    Alejo, R., Valdovinos, R., García, V., Pacheco-Sanchez, J.: A hybrid method to face class overlap and class imbalance on neural networks and multi-class scenarios. Pattern Recognition Letters 34(4), 380–388 (2012)CrossRefGoogle Scholar
  17. 17.
    García, S., Herrera, F.: Evolutionary undersampling for classification with imbalanced datasets: Proposals and taxonomy. Evolutionary Computation 17, 275–306 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • R. Alejo
    • 1
  • J. A. Antonio
    • 1
  • R. M. Valdovinos
    • 2
  • J. H. Pacheco-Sánchez
    • 3
  1. 1.Tecnológico de Estudios Superiores de JocotitlánJocotitlánMéxico
  2. 2.Centro Universitario UAEM Valle de ChalcoUniversidad Autónoma del Estado de MéxicoValle de ChalcoMéxico
  3. 3.Instituto Tecnológico de TolucaMetepecMéxico

Personalised recommendations