Optimization and Engineering

, Volume 17, Issue 1, pp 205–228 | Cite as

A batch, derivative-free algorithm for finding multiple local minima

  • Jeffrey Larson
  • Stefan M. Wild


We propose a derivative-free algorithm for finding high-quality local minima for functions that require significant computational resources to evaluate. Our algorithm efficiently utilizes the computational resources allocated to it and also has strong theoretical results, almost surely starting a finite number of local optimization runs and identifying all local minima. We propose metrics for measuring how efficiently an algorithm finds local minima, and we benchmark our algorithm on synthetic problems (with known local minima) and two real-world applications.


Derivative-free optimization Multistart Parallel algorithms Global optimization 

Mathematics Subject Classification

90C56 90C30 90C26 



We are grateful to Maria Rudnaya and Aswin Kannan for coding the microscopy problem and to Christine Shoemaker for valuable discussions on multistart methods.


  1. Audet C, Dennis JE Jr, Le Digabel S (2008) Parallel space decomposition of the mesh adaptive direct search algorithm. SIAM J Optim 19(3):1150–1170. doi: 10.1137/070707518 CrossRefMathSciNetMATHGoogle Scholar
  2. Csendes T, Pál L, Sendín JOH, Banga JR (2007) The GLOBAL optimization method revisited. Optim Lett 2(4):445–454. doi: 10.1007/s11590-007-0072-3 CrossRefGoogle Scholar
  3. Custódio AL, Madeira JFA (2014) GLODS: global and local optimization using direct search. J Glob Optim 62(1):1–19. doi: 10.1007/s10898-014-0224-9 CrossRefGoogle Scholar
  4. Easterling DR, Watson LT, Madigan ML, Castle BS, Trosset MW (2014) Parallel deterministic and stochastic global minimization of functions with very many minima. Comput Optimi Appl 57(2):469–492. doi: 10.1007/s10589-013-9592-1 CrossRefMathSciNetMATHGoogle Scholar
  5. Gaviano M, Kvasov DE, Lera D, Sergeyev YD (2003) Algorithm 829: software for generation of classes of test functions with known local and global minima for global optimization. ACM Trans Math Softw 29(4):469–480. doi: 10.1145/962437.962444 CrossRefMathSciNetMATHGoogle Scholar
  6. Gheribi AE, Robelin C, Le Digabel S, Audet C, Pelton AD (2011) Calculating all local minima on liquidus surfaces using the factsage software and databases and the mesh adaptive direct search algorithm. J Chem Thermodyn 43(9):1323–1330. doi: 10.1016/j.jct.2011.03.021 CrossRefGoogle Scholar
  7. He J, Verstak A, Watson LT, Sosonkina M (2007) Design and implementation of a massively parallel version of DIRECT. Comput Optim Appl 40(2):217–245. doi: 10.1007/s10589-007-9092-2 CrossRefMathSciNetGoogle Scholar
  8. He J, Watson LT, Sosonkina M (2009) Algorithm 897: VTDIRECT95: serial and parallel codes for the global optimization algorithm DIRECT. ACM Trans Mathl Softw 36(3):1–24. doi: 10.1145/1527286.1527291 CrossRefMathSciNetGoogle Scholar
  9. He J, Verstak A, Sosonkina M, Watson L (2009) Performance modeling and analysis of a massively parallel DIRECT-Part 2. Int J High Perform Comput Appl 23(1):29–41. doi: 10.1177/1094342008098463 CrossRefGoogle Scholar
  10. He J, Verstak A, Watson L, Sosonkina M (2009) Performance modeling and analysis of a massively parallel DIRECT-Part 1. Int J High Perform Comput Appl 23(1):14–28. doi: 10.1177/1094342008098462 CrossRefGoogle Scholar
  11. Hough PD, Kolda TG, Torczon VJ (2001) Asynchronous parallel pattern search for nonlinear optimization. SIAM J Sci Comput 23(1):134–156. doi: 10.1137/S1064827599365823 CrossRefMathSciNetMATHGoogle Scholar
  12. Huyer W, Neumaier A (1999) Global optimization by multilevel coordinate search. J Glob Optim 14(1995):1–28. doi: 10.1023/A:1008382309369 MathSciNetGoogle Scholar
  13. Jones DR, Perttunen CD, Stuckman BE (1993) Lipschitzian optimization without the Lipschitz constant. J Optim Theor Appl 79(1):157–181. doi: 10.1007/BF00941892 CrossRefMathSciNetMATHGoogle Scholar
  14. Moré JJ, Wild SM (2009) Benchmarking derivative-free optimization algorithms. SIAM J Optim 20(1):172–191. doi: 10.1137/080724083 CrossRefMathSciNetMATHGoogle Scholar
  15. Rinnooy Kan AHG, Timmer GT (1987) Stochastic global optimization methods, Part I: clustering methods. Math Program 39(1):27–56. doi: 10.1007/BF02592070 CrossRefMathSciNetMATHGoogle Scholar
  16. Rinnooy Kan AHG, Timmer GT (1987) Stochastic global optimization methods, Part II: multi level methods. Math Program 39(1):57–78. doi: 10.1007/BF02592071 CrossRefMathSciNetMATHGoogle Scholar
  17. Rudnaya ME, Kho SC, Mattheij RMM, Maubach JML (2010) Derivative-free optimization for autofocus and astigmatism correction in electron microscopy. In: Proceedings of the 2nd International Conference on Engineering Optimization, 1–10. Lisbon, Portugal.
  18. Rudnaya ME, Van den Broek W, Doornbos RMP, Mattheij RMM, Maubach JML (2011) Defocus and twofold astigmatism correction in HAADF-STEM. Ultramicroscopy 111(8):1043–1054. doi: 10.1016/j.ultramic.2011.01.034 CrossRefGoogle Scholar
  19. Törn A, Zilinskas A (1989) Global optimization. Springer, New YorkCrossRefMATHGoogle Scholar
  20. Wild SM (2009) Derivative-free optimization algorithms for computationally expensive functions. Ph.D. thesis, Cornell University. URL: Scholar
  21. Wild SM, Regis RG, Shoemaker CA (2008) ORBIT: optimization by radial basis function interpolation in trust-regions. SIAM J Sci Comput 30(6):3197–3219. doi: 10.1137/070691814 CrossRefMathSciNetMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York(outside the USA) 2015

Authors and Affiliations

  1. 1.Argonne National LaboratoryArgonneUSA

Personalised recommendations