A review of genetic-based evolutionary algorithms in SVM parameters optimization

Abstract

Parameters optimization is a research hotspot of SVM and has gained increasing interest from various research fields. Compared with other optimization algorithms, genetic-based evolutionary algorithms that have achieved optimization according to the laws of separation and free combination in genetics are gradually attracted much attention. Also, due to the characteristics of self-organization and self-adaptation, these algorithms often enable SVM to obtain appropriate parameters, so that the model can be applied to more applications. Additionally, many improvements have been proposed in the past two decades in order to allow the optimized SVM model to obtain better performance. This work focuses on reviewing the current state of genetic-based evolutionary algorithms used to optimize parameters of SVM and its variants. First, we introduce the principles of SVM and provide a survey on optimization methods of its parameters. Then we propose a taxonomy of improving genetic-based evolutionary algorithms according to code mechanism, parameters control, population structure, evolutionary strategy, operation mechanism, operators, and many other hybrid approaches. Furthermore, this paper analyzes and compares the advantages and disadvantages of the above algorithms explicitly, and provides their applicable scenarios as well. Finally, we highlight the existing problems of genetic-based evolutionary algorithms used for parameters optimization of SVM and prospect development trends of this field in the future.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. 1.

    Vapnik V, Izmailov R (2017) Knowledge transfer in SVM and neural networks. Ann Math Artif Intell 81(1):3–19. https://doi.org/10.1007/s10472-017-9538-x

    MathSciNet  MATH  Article  Google Scholar 

  2. 2.

    Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297. https://doi.org/10.1023/A:1022627411411

    MATH  Article  Google Scholar 

  3. 3.

    Fayed HA, Atiya AF (2019) Speed up grid-search for parameter selection of support vector machines. Appl Soft Comput J 80:202–210. https://doi.org/10.1016/j.asoc.2019.03.037

    Article  Google Scholar 

  4. 4.

    Yang M, Zhang B, Song YL (2018) Application of support vector machine based on optimized kernel function in people detection. Laser Optoelectron Prog 55(04):107–114

    Google Scholar 

  5. 5.

    Kari T, Gao WS, Zhang ZW, Mo WX, Wang HB, Cui YP (2018) Power transformer fault diagnosis based on a support vector machine and a genetic algorithm. J Tsinghua Univ (Sci Technol) 58(07):623–629

    Google Scholar 

  6. 6.

    Liao ZY, Wang YT, Xie XL, Liu JM (2017) Face recognition by support vector machine based on particle swarm optimization. Comput Eng 43(12):248–254

    Google Scholar 

  7. 7.

    Peng Z, Jiang Y, Yang X, Zhao Z, Zhang L, Wang Y (2018) Bus arrival time prediction based on pca-ga-svm. Neural Netw World 28(1):87–104. https://doi.org/10.14311/NNW.2018.28.005

    Article  Google Scholar 

  8. 8.

    Li K, Wang L, Wu J, Zhang Q, Liao G, Su L (2018) Using ga-svm for defect inspection of flip chips based on vibration signals. Microelectron Reliab 81:159–166. https://doi.org/10.1016/j.microrel.2017.12.032

    Article  Google Scholar 

  9. 9.

    Yang B (2019) Dynamic risk identification safety model based on fuzzy support vector machine and immune optimization algorithm. Saf Sci 118:205–211. https://doi.org/10.1016/j.ssci.2019.05.022

    Article  Google Scholar 

  10. 10.

    Zhang Y, Yu J, Xia C, Yang K, Cao H, Wu Q (2019) Research on ga-svm based head-motion classification via mechanomyography feature analysis. Sensors (Basel, Switzerland) 19(9):1986. https://doi.org/10.3390/s19091986

    Article  Google Scholar 

  11. 11.

    Wang M, Wan Y, Ye Z, Lai X (2017) Remote sensing image classification based on the optimal support vector machine and modified binary coded ant colony optimization algorithm. Inform Sci 402:50–68. https://doi.org/10.1016/j.ins.2017.03.027

    Article  Google Scholar 

  12. 12.

    Yan X, Jia M (2018) A novel optimized svm classification algorithm with multi-domain feature and its application to fault diagnosis of rolling bearing. Neurocomputing 313:47–64. https://doi.org/10.1016/j.neucom.2018.05.002

    Article  Google Scholar 

  13. 13.

    Zhang Z, He X, Sun X, Guo L, Wang J, Wang F (2015) Image recognition of maize leaf disease based on ga-svm. Chem Eng Trans 46:199–204. https://doi.org/10.3303/CET1546034

    Article  Google Scholar 

  14. 14.

    Tang X, Hong H, Shu Y, Tang H, Li J, Liu W (2019) Urban waterlogging susceptibility assessment based on a pso-svm method using a novel repeatedly random sampling idea to select negative samples. J Hydrol 576:583–595. https://doi.org/10.1016/j.jhydrol.2019.06.058

    Article  Google Scholar 

  15. 15.

    Ye F (2018) Evolving the svm model based on a hybrid method using swarm optimization techniques in combination with a genetic algorithm for medical diagnosis. Multimed Tools Appl 77(3):3889–3918. https://doi.org/10.1007/s11042-016-4233-1

    Article  Google Scholar 

  16. 16.

    Li S, Yuan ZG, Wang C, Chen TE, Guo ZC (2018) Optimization of support vector machine parameters based on group intelligence algorithm[J]. CAAI Trans Intell Syst 13(01):70–80

    Google Scholar 

  17. 17.

    Dong H, Jian G (2015) Parameter selection of a support vector machine, based on a chaotic particle swarm optimization algorithm. Cybern Inf Technol 15(3):140–149. https://doi.org/10.1515/cait-2015-0047

    Article  Google Scholar 

  18. 18.

    Dorigo M, Gambardella LM (1997) Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Trans Evolut Comput 1(1):53–66. https://doi.org/10.1109/4235.585892

    Article  Google Scholar 

  19. 19.

    Li XL, Shao ZJ, Qian JX (2002) An optimizing method based on autonomous animats: fish-swarm algorithm. Syst Eng Theory Pract 11:32–38

    Google Scholar 

  20. 20.

    Li XL, Qian JX (2003) Studies on artificial fish swarm optimization algorithm based on decomposition and coordination techniques. J Circuits Syst 01:1–6

    Google Scholar 

  21. 21.

    Karaboga D (2005) An idea based on honey bee swarm for numerical optimization. Technical Report-TR06. Erciyes University

  22. 22.

    Yang X, Deb S, Fong S, He X, Zhao Y (2016) From swarm intelligence to metaheuristics: nature-inspired optimization algorithms. Computer 49(9):52–59. https://doi.org/10.1109/MC.2016.292

    Article  Google Scholar 

  23. 23.

    Ye F, Lou XY, Sun LF (2017) An improved chaotic fruit fly optimization based on a mutation strategy for simultaneous feature selection and parameter optimization for svm and its applications. PLoS ONE 12(4):e173516. https://doi.org/10.1371/journal.pone.0173516

    Article  Google Scholar 

  24. 24.

    Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61. https://doi.org/10.1016/j.advengsoft.2013.12.007

    Article  Google Scholar 

  25. 25.

    Jiang M, Luo J, Jiang D, Xiong J, Song H, Shen J (2016) A cuckoo search-support vector machine model for predicting dynamic measurement errors of sensors. IEEE Access 4:5030–5037. https://doi.org/10.1109/ACCESS.2016.2605041

    Article  Google Scholar 

  26. 26.

    Dantas Dias ML, Rocha Neto AR (2017) Training soft margin support vector machines by simulated annealing: a dual approach. Expert Syst Appl 87:157–169. https://doi.org/10.1016/j.eswa.2017.06.016

    Article  Google Scholar 

  27. 27.

    Sartakhti JS, Afrabandpey H, Saraee M (2017) Simulated annealing least squares twin support vector machine (sa-lstsvm) for pattern classification. Soft Comput 21(15):4361–4373. https://doi.org/10.1007/s00500-016-2067-4

    Article  Google Scholar 

  28. 28.

    Seifollahi S, Bagirov A, Zare Borzeshi E, Piccardi M (2019) A simulated annealing-based maximum-margin clustering algorithm. Comput Intell-Us 35(1):23–41. https://doi.org/10.1111/coin.12187

    MathSciNet  Article  Google Scholar 

  29. 29.

    Rajathi GI, Jiji GW (2019) Chronic liver disease classification using hybrid whale optimization with simulated annealing and ensemble classifier. Symmetry 11(1):33. https://doi.org/10.3390/sym11010033

    Article  Google Scholar 

  30. 30.

    Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor

    Google Scholar 

  31. 31.

    Gozali AA, Fujimura S (2019) Dm-limga: dual migration localized island model genetic algorithm—a better diversity preserver island model. Evolut Intell 12(4):527–539. https://doi.org/10.1007/s12065-019-00253-2

    Article  Google Scholar 

  32. 32.

    Fernandez M, Caballero J, Fernandez L, Sarai A (2011) Genetic algorithm optimization in drug design qsar: bayesian-regularized genetic neural networks (brgnn) and genetic algorithm-optimized support vectors machines (ga-svm). Mol Divers 15(1):269–289. https://doi.org/10.1007/s11030-010-9234-9

    Article  Google Scholar 

  33. 33.

    Martins M, Costa L, Frizera A, Ceres R, Santos C (2013) Hybridization between multi-objective genetic algorithm and support vector machine for feature selection in walker-assisted gait. Comput Methods Prog Biol 113(3):736–748. https://doi.org/10.1016/j.cmpb.2013.12.005

    Article  Google Scholar 

  34. 34.

    Tao Z, Huiling L, Wenwen W, Xia Y (2019) Ga-svm based feature selection and parameter optimization in hospitalization expense modeling. Appl Soft Comput J 75:323–332. https://doi.org/10.1016/j.asoc.2018.11.001

    Article  Google Scholar 

  35. 35.

    Xalf L, Xian C (2002) Choosing multiple parameters for svm based on genetic algorithm. In: 2002 6th International conference on signal processing proceedings, Beijing, China

  36. 36.

    Huang C, Wang C (2006) A ga-based feature selection and parameters optimization for support vector machines. Expert Syst Appl 31(2):231–240. https://doi.org/10.1016/j.eswa.2005.09.024

    Article  Google Scholar 

  37. 37.

    Subhashini KR, Chinta P (2019) An augmented animal migration optimization algorithm using worst solution elimination approach in the backdrop of differential evolution. Evolut Intell 12(2):273–303. https://doi.org/10.1007/s12065-019-00223-8

    Article  Google Scholar 

  38. 38.

    Zhong Y, Cao Q, Zhao J, Ma A, Zhao B, Zhang L (2017) Optimal decision fusion for urban land-use/land-cover classification based on adaptive differential evolution using hyperspectral and lidar data. Remote Sens Basel. https://doi.org/10.3390/rs9080868

    Article  Google Scholar 

  39. 39.

    Wang L, Pan Q, Suganthan PN, Wang W, Wang Y (2010) A novel hybrid discrete differential evolution algorithm for blocking flow shop scheduling problems. Comput Oper Res 37(3):509–520. https://doi.org/10.1016/j.cor.2008.12.004

    MathSciNet  MATH  Article  Google Scholar 

  40. 40.

    Annepu V, Rajesh A (2019) Implementation of self adaptive mutation factor and cross-over probability based differential evolution algorithm for node localization in wireless sensor networks. Evolut Intell 12(3):469–478. https://doi.org/10.1007/s12065-019-00239-0

    Article  Google Scholar 

  41. 41.

    Sarkar S, Das S, Chaudhuri SS (2016) Hyper-spectral image segmentation using rényi entropy based multi-level thresholding aided with differential evolution. Expert Syst Appl 50:120–129. https://doi.org/10.1016/j.eswa.2015.11.016

    Article  Google Scholar 

  42. 42.

    Aburomman AA, Ibne Reaz MB (2017) A novel weighted support vector machines multiclass classifier based on differential evolution for intrusion detection systems. Inform Sci 414:225–246. https://doi.org/10.1016/j.ins.2017.06.007

    Article  Google Scholar 

  43. 43.

    Lin LL, Jiang SD, Liu XD (2008) Simultaneous selection of parameters and features for SVM based on the differential evolution algorithm. J Jilin Univ (Eng Technol Ed) 38(S2):255–259

    Google Scholar 

  44. 44.

    Lin LL, Jiang SD, Liu XD (2009) Parameter selection for an SVM based on a differential evolution algorithm. J Harbin Eng Univ 30(02):138–141

    MATH  Google Scholar 

  45. 45.

    Zhao H, Shen L, Yang JG, Yang LG, Xu HM (2010) The model for calculating ultimate analysis of coal by its proximate analysis based on DE-SVM. J China Coal Soc 35(10):1721–1724

    Google Scholar 

  46. 46.

    Guo Y, Song AG, Bao JT, Cui JW, Zhang HT (2011) Mobile robot traversability prediction based on differential evolution support vector machine. Robot 33(03):257–264

    Google Scholar 

  47. 47.

    Zhang J, Niu Q, Li K, Irwin GW (2011) Model selection in svms using differential evolution. IFAC Proc Vol 44(1):14717–14722. https://doi.org/10.3182/20110828-6-IT-1002.00584

    Article  Google Scholar 

  48. 48.

    Li YH, Zhong YH, Yuan CQ (2013) Application of DE-SVM fusion algorithm in intrusion detection. Comput Eng Appl 49(12):70–73

    Google Scholar 

  49. 49.

    Yang JW, Xu J, Wu XY, Lu YX, Wei JQ (2016) Evaluation method for operational effectiveness based on support vector machine with differential evolution. J Gun Launch Control 37(01):16–20

    Google Scholar 

  50. 50.

    Shen SH (2017) Diesel engine fault diagnosis based on support vector machine optimized by differential evolution. Smart Fact 05:85–88

    Google Scholar 

  51. 51.

    Wang L, Zhou DF, Bai RG (2018) Fault diagnosis of tolerance analog circuits based on differential evolution invasive weed algorithm. Appl Res Comput 35(09):2621–2623

    Google Scholar 

  52. 52.

    Lv PL, Weng XX, Peng XJ (2019) Public traffic passenger recognition based on differential evolution algorithm SVM. J Guangxi Normal Univ (Nat Sci Edn) 37(01):106–114

    Google Scholar 

  53. 53.

    Lin BH, Gu XS (2008) Soft sensor modeling based on DE-LSSVM. J Chem Ind Eng (China) 07:1681–1685

    Google Scholar 

  54. 54.

    Xu SJ, Long W (2012) Parameters selection for LSSVM based on differential evolution to mid-long term runoff prediction. Sci Technol Eng 12(27):6955–6959

    Google Scholar 

  55. 55.

    García-Nieto PJ, García-Gonzalo E, Fernández JRA, Muñiz CD (2019) Modeling of the algal atypical increase in la barca reservoir using the de optimized least square support vector machine approach with feature selection. Math Comput Simulat 166:461–480. https://doi.org/10.1016/j.matcom.2019.07.011

    MathSciNet  Article  Google Scholar 

  56. 56.

    Cheng M, Hoang N, Wu Y (2013) Hybrid intelligence approach based on ls-svm and differential evolution for construction cost index estimation: a taiwan case study. Automat Constr 35:306–313. https://doi.org/10.1016/j.autcon.2013.05.018

    Article  Google Scholar 

  57. 57.

    Yue XF, Shao HH (2015) Fault diagnosis method of rolling bearing based on DE-LSSVM. Comput Meas Control 23(12):3933–3935

    Google Scholar 

  58. 58.

    Jun-hong ZYL (2017) Application of complete ensemble intrinsic time-scale decomposition and least-square svm optimized using hybrid de and pso to fault diagnosis of diesel engines. Front Inf Technol Electron Eng 18(2):272–286

    Google Scholar 

  59. 59.

    Bao ZY, Yu JZ, Yang S (2018) Intelligent optimization algorithm and its MATLAB example, 2nd edn. Publishing House of Electronics Industry, Beijing

    Google Scholar 

  60. 60.

    Oliveira DC, Chavarette FR, Lopes MLM (2019) Damage diagnosis in an isotropic structure using an artificial immune system algorithm. J Braz Soc Mech Sci 41(11):1–11. https://doi.org/10.1007/s40430-019-1971-9

    Article  Google Scholar 

  61. 61.

    Li JW, Ren LH, Ding YS, Chen L (2018) Adaptive integrated classification method based on immune optimization for EEG. J Mech Electr Eng 35(08):873–879

    Google Scholar 

  62. 62.

    Wu H, Hou Z (2004) A short-term load forecasting approach based on immune support vector machines. Power Syst Technol 28(23):47–51

    Google Scholar 

  63. 63.

    Li Y, Wu ZS, Li YF, Zhu YJ (2018) Defects classification method of welding joints based on artificial immune and support vector machine. J Sichuan Univ (Eng Sci Edn) 50(04):221–227

    Google Scholar 

  64. 64.

    Cao YM, Jing DQ, Liu CG (2018) The prediction of the displacement of the arch dam based on the twin support vector machine optimized by immune algorithm. J Yangtze River Sci Res Inst 2018:1–6

    Google Scholar 

  65. 65.

    Wang C, Ma G, Li J, Dai Z, Liu J (2019) Prediction of corrosion rate of submarine oil and gas pipelines based on ia-svm model. IOP Conf Ser Earth Environ Sci 242:22023. https://doi.org/10.1088/1755-1315/242/2/022023

    Article  Google Scholar 

  66. 66.

    Gupta P, Mehlawat MK, Mittal G (2012) Asset portfolio optimization using support vector machines and real-coded genetic algorithm. J Global Optim 53(2):297–315. https://doi.org/10.1007/s10898-011-9692-3

    MathSciNet  MATH  Article  Google Scholar 

  67. 67.

    Meng T, Zhou XZ, Lei YJ (2016) A parameters optimization method for an SVM based on adaptive genetic algorithm. Comput Meas Control 24(09):215–217

    Google Scholar 

  68. 68.

    Fu H, Li L (2011) Simulation research of SVM parameters optimization based on immune algorithm of vector distance. Comput Simul 28(05):201–204

    Google Scholar 

  69. 69.

    de Sampaio WB, Silva AC, de Paiva AC, Gattass M (2015) Detection of masses in mammograms with adaption to breast density using genetic algorithm, phylogenetic trees, lbp and svm. Expert Syst Appl 42(22):8911–8928. https://doi.org/10.1016/j.eswa.2015.07.046

    Article  Google Scholar 

  70. 70.

    Chen P, Yuan L, He Y, Luo S (2016) An improved SVM classifier based on double chains quantum genetic algorithm and its application in analogue circuit diagnosis. Neurocomputing 211:202–211. https://doi.org/10.1016/j.neucom.2015.12.131

    Article  Google Scholar 

  71. 71.

    Yan XT, Wu MQ (2009) Adaptive differential evolution algorithm based on least square SVM. J Syst Simul 21(07):1921–1925

    Google Scholar 

  72. 72.

    Yu X (2017) Disaster prediction model based on support vector machine for regression and improved differential evolution. Nat Hazards 85(2):959–976. https://doi.org/10.1007/s11069-016-2613-5

    Article  Google Scholar 

  73. 73.

    Tian Y, Ju X, Qi Z (2014) Efficient sparse nonparallel support vector machines for classification. Neural Comput Appl 24(5):1089–1099. https://doi.org/10.1007/s00521-012-1331-5

    Article  Google Scholar 

  74. 74.

    Devos O, Downey G, Duponchel L (2014) Simultaneous data pre-processing and svm classification model selection based on a parallel genetic algorithm applied to spectroscopic data of olive oils. Food Chem 148:124–130. https://doi.org/10.1016/j.foodchem.2013.10.020

    Article  Google Scholar 

  75. 75.

    Song XR, Zeng J, Gao S, Chen CB (2018) Target recognition based on differential evolution algorithm of least squares support vector machine. Sci Technol Eng 18(16):68–73

    Google Scholar 

  76. 76.

    Jiaqiang E, Qian C, Zhu H, Peng Q, Zuo W, Liu G (2017) Parameter-identification investigations on the hysteretic preisach model improved by the fuzzy least square support vector machine based on adaptive variable chaos immune algorithm. J Low Freq Noise Vib Active Control 36(3):227–242. https://doi.org/10.1177/0263092317719634

    Article  Google Scholar 

  77. 77.

    Adankon MM, Cheriet M (2010) Genetic algorithm-based training for semi-supervised svm. Neural Comput Appl 19(8):1197–1206. https://doi.org/10.1007/s00521-010-0358-8

    Article  Google Scholar 

  78. 78.

    Zhang J, Li Y, Cao Y, Zhang L (2017) Immune SVM used in wear fault diagnosis of aircraft engine Beijing Hangkong Hangtian Daxue Xuebao. J Beijing Univ Aeronaut Astronaut 43(7):1419–1425. https://doi.org/10.13700/j.bh.1001-5965.2016.0553

    Article  Google Scholar 

  79. 79.

    Corus D, Oliveto PS (2017) Standard steady state genetic algorithms can Hillclimb faster than mutation-only evolutionary algorithms

  80. 80.

    Zhang D, Liu W, Xu X, Deng Q (2010)A novel interpolation method based on differential evolution-simplex algorithm optimized parameters for support vector regression, vol 6382. Springer, Berlin, pp 64–75. https://doi.org/10.1007/978-3-642-16493-4_7

  81. 81.

    Fu H, Feng SC, Liu J, Tang B (2016) The modeling and simulation of gas concentration prediction based on De-EDA-SVM. Chin J Sens Actuators 29(02):285–289

    Google Scholar 

  82. 82.

    Yang L, Xu Z (2019) Feature extraction by PCA and diagnosis of breast tumors using svm with de-based parameter tuning. Int J Mach Learn Cybern 10(3):591–601. https://doi.org/10.1007/s13042-017-0741-1

    Article  Google Scholar 

  83. 83.

    Sun W, Liu MH (2015) Short-term load forecasting based on improved least squares-support vector machine. Electric Power Sci Eng 31(12):16–21

    Google Scholar 

  84. 84.

    Zhai S, Jiang T (2015) A new sense-through-foliage target recognition method based on hybrid differential evolution and self-adaptive particle swarm optimization-based support vector machine. Neurocomputing 149:573–584. https://doi.org/10.1016/j.neucom.2014.08.017

    Article  Google Scholar 

  85. 85.

    Dai S, Niu D, Han Y (2018) Forecasting of power grid investment in china based on support vector machine optimized by differential evolution algorithm and grey wolf optimization algorithm. Appl Sci 8(4):636. https://doi.org/10.3390/app8040636

    Article  Google Scholar 

  86. 86.

    Wang Z, Zhang Z, Wang W (2019) Emotion recognition based on framework of badeba-svm. Math Probl Eng 2019:1–9. https://doi.org/10.1155/2019/9875250

    Article  Google Scholar 

  87. 87.

    Leung CSK, Lau HYK (2016) A hybrid multi-objective immune algorithm for numerical optimization, Porto, Portugal, 2016. In: IJCCI 2016 proceedings of the 8th international joint conference on computational intelligence, SciTePress, pp 105–114

  88. 88.

    Zhou C, Pan P, Yang P, Huang L (2018) Cloud service selection based on chaos quantum immune algorithm, IEEE, pp 1–6. https://doi.org/10.1109/icmic.2018.8529860

Download references

Funding

The funding was provided by National Natural Science Foundation of China (Grand Nos. 41361077, 41561085) and Natural Science Foundation of Jiangxi Province (Grand No. 20161BAB203091).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Deer Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ji, W., Liu, D., Meng, Y. et al. A review of genetic-based evolutionary algorithms in SVM parameters optimization. Evol. Intel. (2020). https://doi.org/10.1007/s12065-020-00439-z

Download citation

Keywords

  • Support vector machine
  • Parameter optimization
  • Evolutionary algorithms
  • Genetic algorithm
  • Differential evolution algorithm
  • Immune algorithm
  • Global optimization