Advertisement

Bio-inspired Metaheuristics for Hyper-parameter Tuning of Support Vector Machine Classifiers

  • Adán Godínez-Bautista
  • Luis Carlos Padierna
  • Alfonso Rojas-DomínguezEmail author
  • Héctor Puga
  • Martín Carpio
Chapter
Part of the Studies in Computational Intelligence book series (SCI, volume 749)

Abstract

Support Vector Machines (SVMs) are machine learning models with many diverse applications. The performance of these models depends on a set of assigned hyper-parameters. The task of hyper-parameter tuning has been performed by metaheuristics methods and recent studies have shown that the effectiveness of these methods is statistically equivalent. In this work we compare four bio-inspired metaheuristics (Bat Algorithm, Firefly Algorithm, Particle Swarm Optimization Algorithm and Social Emotional Optimization Algorithm) to test the hypothesis that the efficiency among these differs while the effectiveness remains. Experimental results on several classification problems indicate that there exist bio-inspired algorithms with higher efficiency, in terms of the required number of SVM evaluations to find optimal hyper-parameters. Based on these results the Bat Algorithm is recommended for SVM hyper-parameter tuning.

Keywords

Bio-inspired metaheuristics Hyper-parameter optimization Support vector machines Pattern classification 

Notes

Acknowledgements

This work was partially supported by the National Council of Science and Technology (CONACYT) of Mexico [grant numbers: 375524 (Luis C. Padierna), CATEDRAS-2598 (A. Rojas), 416761 (Adán Godínez)].

References

  1. 1.
    V.N. Vapnik, Statistical Learning Theory (New York, 1998)Google Scholar
  2. 2.
    M. Heydari, M. Teimouri, Z. Heshmati, S.M. Alavinia, Comparison of various classification algorithms in the diagnosis of type 2 diabetes in Iran. Int. J. Diab. Dev. Countries (2015)Google Scholar
  3. 3.
    R. Langone, C. Alzate, B. De Ketelaere, J. Vlasselaer, W. Meert, J.A.K. Suykens, Engineering applications of artificial intelligence LS-SVM based spectral clustering and regression for predicting maintenance of industrial machines. Eng. Appl. Artif. Intell. 37, 268–278 (2015)CrossRefGoogle Scholar
  4. 4.
    V.N. Vapnik, An overview of statistical learning theory. IEEE Trans. Neural Netw. 10(5), 988–999 (1999)Google Scholar
  5. 5.
    L.C. Padierna, A. Rojas, Hyper-parameter tuning for support vector machines by estimation of distribution algorithms, pp. 787–800 (2017)Google Scholar
  6. 6.
    C.L. Huang, C.J. Wang, A GA-based feature selection and parameters optimization for support vector machines. Expert Syst. Appl. 31(2), 231–240 (2006)CrossRefGoogle Scholar
  7. 7.
    X. Yang, A new metaheuristic bat-inspired algorithm, pp. 1–10 (2010)Google Scholar
  8. 8.
    X. Yang, L. Press, Nature-Inspired Metaheuristic Algorithms Second Edition Google Scholar
  9. 9.
    J. Kennedy, R. Eberhart, Particle swarm optimization, pp. 1942–1948 (1995)Google Scholar
  10. 10.
    Z. Cui, Y. Xu, J. Zeng, Social emotional optimization algorithm with random emotional selection strategy. Theory New Appl. Swarm Intell (2012)Google Scholar
  11. 11.
    C.Z. Naiyang Deng, Y. Tian, Support Vector Machines (CRC Press, 2013)Google Scholar
  12. 12.
    J. Shawe-Taylor, N. Cristianini, Kernel Methods for Pattern Analysis (Cambridge, 2004)Google Scholar
  13. 13.
    R. Poli, J. Kennedy, T. Blackwell, Particle swarm optimization an overview (2007)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Adán Godínez-Bautista
    • 1
  • Luis Carlos Padierna
    • 1
  • Alfonso Rojas-Domínguez
    • 1
    Email author
  • Héctor Puga
    • 1
  • Martín Carpio
    • 1
  1. 1.Tecnológico Nacional de México - Instituto Tecnológico de LeónLeónMexico

Personalised recommendations