Sensitivity of Parameter Control Mechanisms with Respect to Their Initialization

Abstract

The parameter setting problem constitutes one of the major challenges in evolutionary computation, and is subject to considerable research efforts. Since the optimal parameter values can change during the optimization process, efficient parameter control techniques that automatically identify and track reasonable parameter values are sought.

A potential drawback of dynamic parameter selection is that state-of-the-art control mechanisms introduces themselves new sets of hyper-parameters, which need to be tuned for the problem at hand. The general hope is that the performance of an algorithm is much less sensitive with respect to these hyper-parameters than with respect to its original parameters. This belief is backed up by a number of empirical and theoretical results. What is less understood in discrete black-box optimization, however, is the influence of the initial parameter value. We contribute with this work an empirical sensitivity analysis for three selected algorithms with self-adjusting parameter choices: the (1 + 1) EA$_{\alpha }$, the 2-rate $(1+\lambda )$ EA$_{2r,r/2}$, and the $(1+(\lambda ,\lambda ))$ GA. In all three cases we observe fast convergence of the parameters towards their optimal choices. The performance loss of a sub-optimal initialization is shown to be almost negligible for the former two algorithms. For the $(1+(\lambda ,\lambda ))$ GA, in contrast, the choice of $\lambda$ is more critical; our results suggest to initialize it by a small value.