Exploring Optimal Model for Machine Learning by Differential Evolution

  • Yi-Chuan ChiuEmail author
  • Yung-Tsan Jou
  • Hsing-Hung Lin
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1074)


In the big data era, the data scientist or business analyst own business data can apply machine learning algorithm to train inference models easily since the application of big data attracts more and more attention and the technique of training models can be obtained easier and cheaper than ever. The focus of big data applications has gradually shifted from model training to the prediction and inference. For enterprise application scenarios, selecting the most precise model among many trained models has become a significant topic of research. Though ensemble methods have been proposed to discover best model by multiple training phase, studies of exploring best combination within multiple modes are still few. Finding the appropriate parameters to configure different machine learning models is an NP-hard problem that needs metaheuristic algorithm to solve. This study proposes a differential evolution algorithm to integrate multiple trained machine learning models into a hybrid model. For experiment, the regression model is taken as an example and the differential evolution algorithm is compared with the ant colony optimization algorithm in this paper. Three benchmark datasets are employed to examine, and the results discovered that the differential evolution algorithm outperforms ant colony optimization.


Machine learning Differential evolution Ant colony optimization 


  1. 1.
    Nilsson, N.J.: Learning Machines: Foundations of Trainable Pattern-Classifying Systems. McgrawHill, New York (1965)zbMATHGoogle Scholar
  2. 2.
    Sagi, O., Rokach, L.: Ensemble learning: a survey. WIREs Data Min. Knowl. Discov. 8(4), e1249 (2018)Google Scholar
  3. 3.
    McCullagh, P., Nelder, J.: Generalized Linear Models. Chapman and Hall, London (1989)CrossRefGoogle Scholar
  4. 4.
    Storn, R., Price, K.: Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim. 11(4), 341–359 (1997)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Mayera, D.G., Kinghornb, B.P., Archer, A.A.: Differential evolution – an easy and efficient evolutionary algorithm for model optimization. Agric. Syst. 83(3), 315–328 (2005)CrossRefGoogle Scholar
  6. 6.
    Oparaa, K.R., Arabasb, J.: Differential Evolution: a survey of theoretical analyses. Swarm Evol. Comput. 44, 546–558 (2018). in pressCrossRefGoogle Scholar
  7. 7.
    Gong, W., Cai, Z.: Differential evolution made faster and more robust. In: Proceedings of 2006 IEEE International Conference on Industrial Technology (2006)Google Scholar
  8. 8.
    Arafa, M., Elsayed, A.S., Fahmy, M.M.: An enhanced differential evolution optimization algorithm. In: Proceedings of 2014 Fourth International Conference on Digital Information and Communication Technology and its Applications (2014)Google Scholar
  9. 9.
    Elsayed, S., Sarker, R.: An adaptive configuration of differential evolution algorithms for big data. In: Proceedings of IEEE Congress on Evolutionary Computation, pp. 695–702 (2015)Google Scholar
  10. 10.
    Piotrowski, A.P.: Review of differential evolution population size. Swarm Evol. Comput. 32, 1–24 (2017)CrossRefGoogle Scholar
  11. 11.
    Das, S., Suganthan, P.N.: Differential evolution: a survey of the state-of-the-art. IEEE Trans. Evol. Comput. 15(1), 4–31 (2011)CrossRefGoogle Scholar
  12. 12.
    Dorigo, M.: Optimization, learning and natural algorithms, Ph.D. Thesis, Politecnico di Milano, Italia (1992)Google Scholar
  13. 13.
    Dorigo, M., Blum, C.: Ant colony optimization theory: a survey. Theor. Comput. Sci. 344(2–3), 243–278 (2005)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Niu, D., Wang, Y., Wu, D.D.: Power load forecasting using support vector machine and ant colony optimization. Exp. Syst. Appl. 37(3), 2531–2539 (2010)CrossRefGoogle Scholar
  15. 15.
    Alwan, H.B., Ku-Mahamud, K.R.: Feature selection and model selection algorithm using incremental mixed variable ant colony optimization for support vector machine classifier. Int. J. Math. Comput. Simul. 7(5), 406–414 (2013)Google Scholar
  16. 16.
  17. 17.
    UCI Machine Learning Repository Homepage.

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Industrial and System EngineeringChung Yuan Christian UniversityTaoyuanTaiwan, R.O.C.
  2. 2.Department of Industrial EngineeringChung Hua UniversityHsinchuTaiwan, R.O.C.

Personalised recommendations