Efficient Parameter Optimization Based on Combination of Direct Global and Local Search Methods

Abstract

In this paper we focus on direct optimization methods which require nothing but goal function values for orientation (blind search). On the one hand this property ensures robustness and universal applicability. On the other hand direct optimization usually requires a lot of computational effort (goal function evaluations) to ensure optimization success (convergence towards a globally-optimal region of the search space) and an acceptable quality of the optimization result (small approximation error). These fundamental drawbacks of direct optimization are due to the fact that no auxiliary information like derivatives or other problem specific knowledge is exploited to accelerate the optimization process. In order to substantially improve the performance of direct optimization we propose a combination of probabilistic global and deterministic local optimization methods. The resulting combined 2-phase optimization strategy has been proven to be both powerful and efficient. The excellent heuristic properties of this hybrid method allows to use it as the basic component of a multiple-stage optimization strategy. The goal of multiple-stage optimization is to perform a systematic analysis of the most important extreme points of a given optimization problem. This paper presents the structure of our developed optimization algorithms. Beyond that, results of an extensive optimization experiment are presented showing the potentialities but also the limits of our developed methods.