Advertisement

Quantum-Behaved Particle Swarm Optimization for Parameter Optimization of Support Vector Machine

  • Alaa TharwatEmail author
  • Aboul Ella Hassanien
Article
  • 40 Downloads

Abstract

Support vector machine (SVM) parameters such as penalty parameter and kernel parameters have a great influence on the complexity and accuracy of SVM model. In this paper, quantum-behaved particle swarm optimization (QPSO) has been employed to optimize the parameters of SVM, so that the classification error can be reduced. To evaluate the proposed model (QPSO-SVM), the experiment adopted seven standard classification datasets which are obtained from UCI machine learning data repository. For verification, the results of the QPSO-SVM algorithm are compared with the standard PSO, and genetic algorithm (GA) which is one of the well-known optimization algorithms. Moreover, the results of QPSO are compared with the grid search, which is a conventional method of searching parameter values. The experimental results demonstrated that the proposed model is capable to find the optimal values of the SVM parameters. The results also showed lower classification error rates compared with standard PSO and GA algorithms.

Keywords

Quantum particle swarm optimization (QPSO) Optimization algorithms Support vector machine (SVM) Classification Parameter optimization 

Notes

References

  1. Ali, S., & Smith, K. (2003). Automatic parameter selection for polynomial kernel. In Proceedings of IEEE International Conference on Information Reuse and Integration, (IRI 2003), Lens, France, October, (Vol. 27-29 pp. 243–249).Google Scholar
  2. Bashir, Z., & El-Hawary, M. (2009). Applying wavelets to short-term load forecasting using PSO-based neural networks. IEEE Transactions on Power Systems, 24(1), 20–27.Google Scholar
  3. Ben-Hur, A., Ong, C.S., Sonnenburg, S., Schölkopf, B., Rätsch, G. (2008). Support vector machines and kernels for computational biology. PLoS Comput Biol, 4(10), e1000173.Google Scholar
  4. Blake, C., & Merz, C.J. (1998). {U C I} repository of machine learning databases repository of machine learning databases.Google Scholar
  5. Byvatov, E., & Schneider, G. (2002). Support vector machine applications in bioinformatics. Applied Bioinformatics, 2(2), 67–77.Google Scholar
  6. Cai, Y., Sun, J., Wang, J., Ding, Y., Tian, N., Liao, X., et al. (2008). Optimizing the codon usage of synthetic gene with QPSO algorithm. Journal of Theoretical Biology, 254(1), 123–127.MathSciNetzbMATHGoogle Scholar
  7. Chander, A., Chatterjee, A., Siarry, P. (2011). A new social and momentum component adaptive PSO algorithm for image segmentation. Expert Systems with Applications, 38(5), 4998–5004.Google Scholar
  8. Chapelle, O., Vapnik, V., Bousquet, O. (2002). Choosing multiple parameters for support vector machines. Machine Learning, 46(1-3), 131–159.zbMATHGoogle Scholar
  9. Clerc, M., & Kennedy, J. (2002). The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation, 6(1), 58–73.Google Scholar
  10. Dudani, S.A. (1976). The distance-weighted k-nearest-neighbor rule. IEEE Transactions on Systems, Man, and Cybernetics, SMC-6(4), 325–327.Google Scholar
  11. Eberhart, R., & Kennedy, J. (1995). A new optimizer using particle swarm theory. In Proceedings of the Sixth International Symposium on Micro Machine and Human Science, 1995. MHS’95. pp. 39–43.Google Scholar
  12. Friedrichs, F., & Igel, C. (2005). Evolutionary tuning of multiple SVM parameters. Neurocomputing, 64, 107–117.Google Scholar
  13. Hassan, R., Cohanim, B., De, Weck, Venter, O. (2005). G A comparison of particle swarm optimization and the genetic algorithm. In Proceedings of the 1st AIAA multidisciplinary design optimization specialist conference, Honolulu, Hawaii, April 23-26 (pp. 1–13).Google Scholar
  14. He, H., & Garcia, E.A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284.Google Scholar
  15. Huang, C.L., & Wang, C.J. (2006). A GA-based feature selection and parameters optimizationfor support vector machines. Expert Systems with Applications, 31(2), 231–240.Google Scholar
  16. Kecman, V. (2001). Learning and soft computing: support vector machines, neural networks, and fuzzy logic models. Cambridge: MIT Press.zbMATHGoogle Scholar
  17. Keerthi, S.S., & Lin, C.J. (2003). Asymptotic behaviors of support vector machines with Gaussian kernel. Neural Computation, 15(7), 1667–1689.zbMATHGoogle Scholar
  18. Kennedy, J. (2004). Probability and dynamics in the particle swarm. In Congress on Evolutionary Computation.Google Scholar
  19. Kennedy, J. (2005). Dynamic-probabilistic particle swarms. In Proceedings of the 7th Annual Conference On Genetic And Evolutionary Computation, pp. 201–207.Google Scholar
  20. Kennedy, J. (2010). Particle swarm optimization. In Encyclopedia of Machine Learning. Springer, pp. 760–766.Google Scholar
  21. Krohling, R.A. (2004). Gaussian swarm: a novel particle swarm optimization algorithm. In Proceedings of IEEE Conference on Cybernetics and Intelligent Systems, (Vol. 1 pp. 372–376).Google Scholar
  22. Krohling, R.A., & Santos Coelho, L. dos. (2006). PSO-E: particle swarm with exponential distribution. In 2006 IEEE International Conference on Evolutionary Computation, pp 1428–1433.Google Scholar
  23. Li, S., Wang, R., Hu, W., Sun, J. (2007). A new QPSO based BP neural network for face detection. In Fuzzy Information and Engineerin. Springer, pp. 355–363.Google Scholar
  24. Liang, J.J., Qin, A.K., Suganthan, P.N. (2006). Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Transactions on Evolutionary Computation, 10(3), 281–295.Google Scholar
  25. Lin, S.W., Ying, K.C., Chen, S.C., Lee, Z.J. (2008). Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications, 35(4), 1817–1824.Google Scholar
  26. Liu, B., Wang, L., Jin, Y.H., Tang, F. (2005). Huang D X. Chaos, Solitons & Fractals, 25(5), 1261–1271.Google Scholar
  27. Liu, J., Xu, W., Sun, J. (2005). Quantum-behaved particle swarm optimization with mutation operator. In Proceedings of the 17th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’05).Google Scholar
  28. Maitra, M., & Chatterjee, A. (2008). A hybrid cooperative–comprehensive learning based PSO algorithm for image segmentation using multilevel thresholding. Expert Systems with Applications, 34(2), 1341– 1350.Google Scholar
  29. Merwe, D., Van der, Engelbrecht, A.P. (2003). Data clustering using particle swarm optimization. In The 2003 Congress on Evolutionary Computation(CEC’03), (Vol. 1 pp. 215–220).Google Scholar
  30. Mikki, S.M., & Kishk, A.A. (2006). Quantum particle swarm optimization for electromagnetics. IEEE Transactions on Antennas and Propagation, 54(10), 2764–2775.Google Scholar
  31. Miyatake, M., Veerachary, M., Toriumi, F., Fujii, N., Ko, H. (2011). Maximum power point tracking of multiple photovoltaic arrays: a PSO approach. IEEE Transactions on Aerospace and Electronic Systems, 47(1), 367–380.Google Scholar
  32. Omkar, S., Khandelwal, R., Ananth, T., Naik, G.N. (2009). Quantum behaved Particle Swarm Optimization (QPSO) for multi-objective design optimization of composite structures. Expert Systems with Applications, 36(8), 11312–11322.Google Scholar
  33. Pal, S.K., & Mitra, S. (1992). Multilayer perceptron, fuzzy sets, and classification. IEEE Transactions on Neural Networks, 3(5), 683–697.Google Scholar
  34. Panda, S., & Padhy, N.P. (2008). Optimal location and controller design of STATCOM for power system stability improvement using PSO. Journal of the Franklin Institute, 345(2), 166–181.zbMATHGoogle Scholar
  35. Reynolds, C.W. (1987). Flocks, herds and schools: a distributed behavioral model. ACM Siggraph Computer Graphics, 21(4), 25–34.Google Scholar
  36. Richer, T.J., & Blackwell, T.M. (2006). The Lévy particle swarmvy particle swarm. In 2006 IEEE International Conference on Evolutionary Computation, pp. 808–815.Google Scholar
  37. Santos Coelho, L. dos. (2010). Gaussian quantum-behaved particle swarm optimization approaches for constrained engineering design problems. Expert Systems with Applications, 37(2), 1676–1683.Google Scholar
  38. Santos Coelho, L. dos, & Krohling, R.A. (2005). Predictive controller tuning using modified particle swarm optimization based on Cauchy and Gaussian distributions. In Soft Computing: Methodologies and Applications. Springer, pp. 287–298.Google Scholar
  39. Scholköpf, B., & Smola, A.J. (2001). Learning with kernels: support vector machines, regularization, optimization, and beyond. Cambridge: MIT Press.Google Scholar
  40. Subasi, A. (2013). Classification of EMG signals using PSO optimized SVM for diagnosis of neuromuscular disorders. Computers in Biology and Medicine, 43(5), 576–586.Google Scholar
  41. Sun, J., Fang, W., Palade, V., Wu, X. (2011). Quantum-behaved particle swarm optimization with Gaussian distributed local attractor point. Applied Mathematics and Computation, 218(7), 3763–3775.zbMATHGoogle Scholar
  42. Sun, J., Fang, W., Wu, X., Palade, V. (2012). Quantum-behaved particle swarm optimization: analysis of individual particle behavior and parameter selection. Evolutionary Computation, 20(3), 349–393.Google Scholar
  43. Sun, J., & Feng, B. (2004). Particle swarm optimization with particles having quantum behavior. In Congress on Evolutionary Computation.Google Scholar
  44. Sun, J., Xu, W., Feng, B. (2004). A global search strategy of quantum-behaved particle swarm optimization. In Proceedings of IEEE Conference on Cybernetics and Intelligent Systems, (Vol. 1 pp. 111–116).Google Scholar
  45. Sun, J., Xu, W., Feng, B. (2005). Adaptive parameter control for quantum-behaved particle swarm optimization on individual level. In Proceedings of IEEE International Conference on Systems, Man and Cybernetics, (Vol. 4 pp. 3049–3054).Google Scholar
  46. Tharwat, A. (2016). Linear vs. quadratic discriminant analysis classifier: a tutorial. International Journal of Applied Pattern Recognition, 3(2), 145–180.Google Scholar
  47. Tharwat, A. (2016). Principal component analysis-a tutorial. International Journal of Applied Pattern Recognition, 3(3), 197–240.Google Scholar
  48. Tharwat, A., Gabel, T., Hassanien, A.E. (2017). Parameter optimization of support vector machine using dragonfly algorithm. In International Conference on Advanced Intelligent Systems and Informatics, pp. 309–319.Google Scholar
  49. Tharwat, A., Gaber, T., Ibrahim, A. (2017). Linear discriminant analysis: a detailed tutorial. AI Communications, 30(2), 169–190.MathSciNetGoogle Scholar
  50. Tharwat, A., & Hassanien, A.E. (2018). Chaotic antlion algorithm for parameter optimization of support vector machine. Applied Intelligence, 48(3), 670–686.Google Scholar
  51. Tharwat, A., Hassanien, A.E., Elnaghi, B.E. (2016). A BA-based algorithm for parameter optimization of support vector machine. Pattern Recognition Letters.Google Scholar
  52. Tharwat, A., Hemedan, A.A., Hassanien, A.E., Thomas G. (2018). A biometric-based model for fish species classification. Fisheries Research, 204, 324–336.Google Scholar
  53. Tharwat, A., & Moemen, Y.S. (2017). Classification of toxicity effects of biotransformed hepatic drugs using whale optimized support vector machines. Journal of Biomedical Informatics, 68, 132–149.Google Scholar
  54. Wang, G., & Guo, L. (2013). A novel hybrid bat algorithm with harmony search for global numerical optimization. Journal of Applied Mathematics.Google Scholar
  55. Wang, L. (2005). Support vector machines: theory and applications, vol. 177. Berlin: Springer.Google Scholar
  56. Wu, C.H., Tzeng, G.H., Lin, R.H. (2009). A novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression. Expert Systems with Applications, 36(3), 4725–4735.Google Scholar
  57. Xi, M., & Sun, J. (2008). An improved quantum-behaved particle swarm optimization algorithm with weighted mean best position. Applied Mathematics and Computation, 205(2), 751–759.zbMATHGoogle Scholar
  58. Xinchao, Z. (2010). A perturbed particle swarm algorithm for numerical optimization. Applied Soft Computing, 10(1), 119–124.Google Scholar
  59. Yang, X.S. (2014). Nature-inspired optimization algorithms, 1st edn. Amsterdam: Elsevier.zbMATHGoogle Scholar
  60. Zhang, X., & Chen, X. (2010). An ACO-based algorithm for parameter optimization of support vector machines. Expert Systems with Applications, 37(9), 6618–6628.Google Scholar
  61. Zhang, Y., & Zhang, P. (2015). Machine training and parameter settings with social emotional optimization algorithm for support vector machine. Pattern Recognition Letters, 54, 36–42.Google Scholar
  62. Zhao, M., Fu, C., Ji, L., Tang, K. (2011). Feature selection and parameter optimization for support vector machines: a new approach based on genetic algorithm with feature chromosomes. Expert Systems with Applications, 38(5), 5197–5204.Google Scholar

Copyright information

© Classification Society of North America 2019

Authors and Affiliations

  1. 1.Faculty of Computer Science and EngineeringFrankfurt University of Applied SciencesFrankfurt am MainGermany
  2. 2.Faculty of Computers and InformationCairo UniversityGizaEgypt
  3. 3.Scientific Research Group in Egypt (SRGE)GizaEgypt

Personalised recommendations