Convergence Rate Evaluation of Derivative-Free Optimization Techniques

  • Thomas LuxEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10122)


This paper presents a convergence rate comparison of five different derivative-free numerical optimization techniques across a set of 50 benchmark objective functions. Results suggest that Adaptive Memory Programming for constrained Global Optimization, and a variant of Simulated Annealing are two of the fastest-converging numerical optimization techniques in this set. Lastly, there is a mechanism for expanding the set of optimization algorithms provided.


Optimization Convergence Metaheuristic Derivative-free 



This research project was funded by the Roanoke College Mathematics Computer Science and Physics Department. The python code for AMPGO and the benchmarking library were a result of the freely available work done by Andrea Gavana at [9].


  1. 1.
    Kirkpatrick, S.: Optimization by simulated annealing: quantitative studies. J. Stat. Phys. 34(5–6), 975–986 (1984)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Lasdon, L., Duarte, A., Glover, F., Laguna, M., Martí, R.: Adaptive memory programming for constrained global optimization. Comput. Oper. Res. 37(8), 1500–1509 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Yang, X.S., Deb, S.: Cuckoo search via lévy flights. In: World Congress on Nature and Biologically Inspired Computing, NaBIC 2009, pp. 210–214. IEEE (2009)Google Scholar
  4. 4.
    Civicioglu, P.: Backtracking search optimization algorithm for numerical optimization problems. Appl. Math. Comput. 219(15), 8121–8144 (2013)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Karaboga, D., Gorkemli, B.: A quick artificial bee colony (qABC) algorithm and its performance on optimization problems. Appl. Soft Comput. 23, 227–238 (2014)CrossRefGoogle Scholar
  6. 6.
    Civicioglu, P., Besdok, E.: A conceptual comparison of the cuckoo-search, particle swarm optimization, differential evolution and artificial bee colony algorithms. Artif. Intell. Rev. 39(4), 315–346 (2013)CrossRefGoogle Scholar
  7. 7.
    Moré, J.J., Wild, S.M.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20(1), 172–191 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Floudas, C.A., Pardalos, P.M.: Encyclopedia of Optimization. Springer Science and Business Media, Heidelberg (2009)Google Scholar
  9. 9.
    Gavana, A.: Global optimization benchmarks and AMPGO. Accessed Apr 2016 (2014)Google Scholar
  10. 10.
    Karaboga, D., Basturk, B.: A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J. Glob. Optim. 39(3), 459–471 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Rios, L.M., Sahinidis, N.V.: Derivative-free optimization: a review of algorithms and comparison of software implementations. J. Glob. Optim. 56(3), 1247–1293 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Alizamir, S., Pardalos, P.M., Rebennack, S.: Improving the neighborhood selection strategy in simulated annealing using the optimal stopping problem. INTECH Open Access Publisher (2008)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Roanoke CollegeSalemUSA

Personalised recommendations