Hybrid Associative Memories for Imbalanced Data Classification: An Experimental Study

  • L. Cleofas-Sánchez
  • V. García
  • R. Martín-Félez
  • R. M. Valdovinos
  • J. S. Sánchez
  • O. Camacho-Nieto
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7914)

Abstract

Hybrid associative memories are based on the combination of two well-known associative networks, the lernmatrix and the linear associator, with the aim of taking advantage of their merits and overcoming their limitations. While these models have extensively been applied to information retrieval problems, they have not been properly studied in the framework of classification and even less with imbalanced data. Accordingly, this work intends to give a comprehensive response to some issues regarding imbalanced data classification: (i) Are the hybrid associative models suitable for dealing with this sort of data? and, (ii) Does the degree of imbalance affect the performance of these neural classifiers? Experiments on real-world data sets demonstrate that independently of the imbalance ratio, the hybrid associative memories perform poorly in terms of area under the ROC curve, but the hybrid associative classifier with translation appears to be the best solution when assessing the true positive rate.

Keywords

Class Imbalance Associative Memory Neural Network 

References

  1. 1.
    Anderson, J.A.: A simple neural network generating an interactive memory. Mathematical Biosciences 14(3-4), 197–220 (1972)MATHCrossRefGoogle Scholar
  2. 2.
    Deng, T., Zhao, L., Feng, L., Xue, W.: Information re-finding by context: A brain memory inspired approach. In: Proc. 20th ACM International Conference on Information and Knowledge Management, Glasgow, Scotland, UK, pp. 1553–1558 (2011)Google Scholar
  3. 3.
    Pantazi, S.V.: Methodological review: Unsupervised grammar induction and similarity retrieval in medical language processing using the Deterministic Dynamic Associative Memory (DDAM) model. Journal of Biomedical Informatics 43(5), 844–857 (2010)CrossRefGoogle Scholar
  4. 4.
    Yaakobi, E., Bruck, J.: On the uncertainty of information retrieval in associative memories. In: Proc. IEEE International Symposium on Information Theory, Cambridge, MA, pp. 106–110 (2012)Google Scholar
  5. 5.
    Steinbuch, K.: Die Lernmatrix. Kybernetik 1, 36–45 (1961)MATHCrossRefGoogle Scholar
  6. 6.
    Anderson, J.A., Rosenfeld, E. (eds.): Neurocomputing: Foundations of Research. MIT Press, Cambridge (1988)Google Scholar
  7. 7.
    Kohonen, T.: Self-Organization and Associative Memory. Springer, New York (1989)CrossRefGoogle Scholar
  8. 8.
    Santiago-Montero, R.: Clasificador Híbrido de Patrones basado en la Lernmatrix de Steinbuch y el Linear Associator de Anderson-Kohonen. Master’s thesis, Centro de Investigación en Computación, Instituto Politécnico Nacional, México D.F. (2003)Google Scholar
  9. 9.
    Japkowicz, N., Shaju, S.: The class imbalance problem: A systematic study. Intelligent Data Analysis 6(5), 429–449 (2002)MATHGoogle Scholar
  10. 10.
    Galar, M., Fernández, A., Barrenechea, E., Bustince, H., Herrera, F.: A review on ensembles for the class imbalance problems: Bagging, boosting, and hybrid-based approaches. IEEE Trans. on Systems, Man, and Cybernetics – Part C: Applications and Reviews 42(4), 463–484 (2012)CrossRefGoogle Scholar
  11. 11.
    García, V., Sánchez, J.S., Mollineda, R.A.: On the effectiveness of preprocessing methods when dealing with different levels of class imbalance. Knowledge-Based Systems 25(1), 13–21 (2012)CrossRefGoogle Scholar
  12. 12.
    Akbani, R., Kwek, S., Japkowicz, N.: Applying support vector machines to imbalanced datasets. In: Proc. 15th European Conference on Machine Learning, Pisa, Italy, pp. 39–50 (2004)Google Scholar
  13. 13.
    Cieslak, D.A., Chawla, N.V.: Learning decision trees for unbalanced data. In: Proc. 19th European Conference on Machine Learning, Antwerp, Belgium, pp. 241–256 (2008)Google Scholar
  14. 14.
    Khoshgoftaar, T.M., Van Hulse, J., Napolitano, A.: Supervised neural network modeling: An empirical investigation into learning from imbalanced data with labeling errors. IEEE Trans. on Neural Networks 21(5), 813–830 (2010)CrossRefGoogle Scholar
  15. 15.
    Hoens, T.R., Qian, Q., Chawla, N.V., Zhou, Z.-H.: Building decision trees for the multi-class imbalance problem. In: Tan, P.-N., Chawla, S., Ho, C.K., Bailey, J. (eds.) PAKDD 2012, Part I. LNCS, vol. 7301, pp. 122–134. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  16. 16.
    Cleofas-Sánchez, L., Guzmán Escobedo, M., Valdovinos, R.M., Yáñez Márquez, C., Camacho-Nieto, O.: Using hybrid associative classifier with traslation (HACT) for studying imbalance data sets. Engineering and Research 32(1), 53–57 (2012)Google Scholar
  17. 17.
    Hasson, M.H.: Associative Neural Memories: Theory and Implementation. Oxford University Press (1993)Google Scholar
  18. 18.
    Yáñez Márquez, C., Díaz de León, J.L.: Introducción a las Memorias Asociativas. Instituto Politécnico Nacional, México D.F. (2003)Google Scholar
  19. 19.
    Alcalá-Fernández, J., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F.: KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. Journal of Multiple-Value Logic and Soft Computing 17(2-3), 255–287 (2011)Google Scholar
  20. 20.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explorations Newsletter 11(1), 10–18 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • L. Cleofas-Sánchez
    • 1
  • V. García
    • 2
  • R. Martín-Félez
    • 2
  • R. M. Valdovinos
    • 3
  • J. S. Sánchez
    • 2
  • O. Camacho-Nieto
    • 4
  1. 1.Instituto Politécnico NacionalCentro de Investigación en ComputaciónMéxico D.F.México
  2. 2.Institute of New Imaging Technologies, Department of Computer Languages and SystemsUniversitat Jaume ICastellón de la PlanaSpain
  3. 3.Centro Universitario Valle de ChalcoUniversidad Autónoma del Estado de MéxicoValle de ChalcoMéxico
  4. 4.Instituto Politécnico NacionalCentro de Innovación y Desarrollo Tecnológico en CómputoMéxico D.F.México

Personalised recommendations