Advertisement

Comparison of Several Stochastic and Deterministic Derivative-Free Global Optimization Algorithms

  • Vladislav SovrasovEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11548)

Abstract

In this paper popular open-source solvers are compared against Globalizer solver, which is developed at the Lobachevsky State University. The Globalizer is designed to solve problems with black-box objective function satisfying the Lipschitz condition and shows competitive performance with other similar solvers. The comparison is done on several sets of challenging multi-extremal benchmark functions. Also this work considers a method of heuristic hyperparameters control for the Globalizer allowing to reduce amount of initial tuning before optimization. The proposed scheme allows substantially increase convergence speed of the Globalizer by switching between “local” and “global” search phases in runtime.

Keywords

Deterministic global optimization Stochastic global optimization Algorithms comparison Derivative-free algorithms Black-box optimization Multi-extremal problems 

References

  1. 1.
    Ali, M.M., Khompatraporn, C., Zabinsky, Z.B.: A numerical evaluation of several stochastic algorithms on selected continuous global optimization test problems. J. Glob. Optim. 31(4), 635–672 (2005)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Beiranvand, V., Hare, W., Lucet, Y.: Best practices for comparing optimization algorithms. Optim. Eng. 18(4), 815–848 (2017)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Evtushenko, Y., Posypkin, M.: A deterministic approach to global box-constrained optimization. Optim. Lett. 7, 819–829 (2013)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Gablonsky, J.M., Kelley, C.T.: A locally-biased form of the direct algorithm. J. Glob. Optim. 21(1), 27–37 (2001)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Gaviano, M., Kvasov, D.E., Lera, D., Sergeev, Ya.D.: Software for generation of classes of test functions with known local and global minima for global optimization. ACM Trans. Math. Softw. 29(4), 469–480 (2003)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Gergel, V.P., Barkalov, K.A., Sysoyev, A.V.: A novel supercomputer software system for solving time-consuming global optimization problems. Numer. Algebr. Control. Optim. 8(1), 47–62 (2018)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Grishagin, V.: Operating characteristics of some global search algorithms. Probl. Stoch. Search 7, 198–206 (1978). (In Russian)zbMATHGoogle Scholar
  8. 8.
    Johnson, S.G.: The NLOpt nonlinear-optimization package. http://ab-initio.mit.edu/nlopt. Accessed 24 Dec 2018
  9. 9.
    Jones, E., Oliphant, T., Peterson, P., et al.: SciPy: open source scientific tools for Python (2001–). http://www.scipy.org/. Accessed 24 Dec 2018
  10. 10.
    Jones, D.R.: The direct global optimization algorithm. In: Floudas, C., Pardalos, P. (eds.) The Encyclopedia of Optimization, pp. 725–735. Springer, Boston (2009).  https://doi.org/10.1007/978-0-387-74759-0_128CrossRefGoogle Scholar
  11. 11.
    Kan, A.H.G.R., Timmer, G.T.: Stochastic global optimization methods Part II: multi level methods. Math. Program. 39, 57–78 (1987)CrossRefGoogle Scholar
  12. 12.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN 1995 - International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)Google Scholar
  13. 13.
    Kvasov, D.E., Mukhametzhanov, M.S.: Metaheuristic vs. deterministic global optimization algorithms: the univariate case. Appl. Math. Comput. 318, 245–259 (2018)MathSciNetGoogle Scholar
  14. 14.
    Liberti, L., Kucherenko, S.: Comparison of deterministic and stochastic approaches to global optimization. Int. Trans. Oper. Res. 12, 263–285 (2005)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Madsen, K., Zertchaninov, S.: A new branch-and-bound method for global optimization (1998)Google Scholar
  16. 16.
    Mullen, K.: Continuous global optimization in R. J. Stat. Softw. 60(6), 1–45 (2014)CrossRefGoogle Scholar
  17. 17.
    Paulavivcius, R., Zilinskas, J., Grothey, A.: Parallel branch and bound for global optimization with combination of Lipschitz bounds. Optim. Methods Softw. 26(3), 487–498 (1997)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Pošík, P., Huyer, W., Pál, L.: A comparison of global search algorithms for continuous black box optimization. Evol. Comput. 20(4), 509–541 (2012)CrossRefGoogle Scholar
  19. 19.
    Price, W.L.: Global optimization by controlled random search. J. Optim. Theory Appl. 40(3), 333–348 (1983)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Schluter, M., Egea, J.A., Banga, J.R.: Extended ant colony optimization for non-convex mixed integer nonlinear programming. Comput. Oper. Res. 36(7), 2217–2229 (2009)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Sergeyev, Y., Kvasov, D.: Global search based on efficient diagonal partitions and a set of Lipschitz constants. SIAM J. Optim. 16(3), 910–937 (2006)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Sergeyev, Y.D., Strongin, R.G., Lera, D.: Introduction to Global Optimization Exploiting Space-Filling Curves. Springer Briefs in Optimization. Springer, New York (2013).  https://doi.org/10.1007/978-1-4614-8042-6CrossRefzbMATHGoogle Scholar
  23. 23.
    Storn, R., Price, K.: Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim. 11(4), 341–359 (1997)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Strongin, R.: Numerical Methods in Multiextremal Problems (Information-Statistical Algorithms). Nauka, Moscow (1978). (In Russian)Google Scholar
  25. 25.
    Strongin R.G., Sergeyev Ya.D.: Global Optimization with Non-convex Constraints: Sequential and Parallel Algorithms. Kluwer Academic Publishers, Dordrecht (2000)CrossRefGoogle Scholar
  26. 26.
    Torczon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 9(1), 1–25 (1997)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Xiang, Y., Sun, D., Fan, W., Gong, X.: Generalized simulated annealing algorithm and its application to the thomson model. Phys. Lett. A 233(3), 216–220 (1997)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Lobachevsky State University of Nizhni NovgorodNizhni NovgorodRussia

Personalised recommendations