Bayesian Heuristic Approach (BHA) and Applications to Discrete Optimization

Abstract

Industrial size discrete optimization problems are most often solved using “heuristics” (expert opinions defining how to solve a family of problems). The paper is concerned about ways to speed up the search in a discrete optimization problem by combining several heuristics involving randomization. Using expert knowledge an a priori distribution on a set of heuristic decision rules is defined and is continuously updated while solving a particular problem. This approach (BHA or Bayesian Heuristic Approach) is different from the traditional Bayesian Approach (BA) where the a priori distribution is defined on a set of functions to be minimized. The paper focuses on the main objective of BHA that is improving any given heuristic by “mixing” it with other decision rules. In addition to providing almost sure convergence such mixed decision rules often outperform (in terms of speed) even the best heuristics as judged by the three considered examples. However, the final results of BHA depend on the quality of the specific heuristic. That means the BHA should be regarded as a tool for enhancing the best heuristics but not for replacing them.

This way the formal Bayesian Approach (BA) is extended to a semiformal Bayesian Heuristic Approach (BHA) where heuristics may be included more flexibly. The paper is concluded by an example of Dynamic Visualization Approach (DVA). The goal of DVA is to exploit heuristics directly, bypassing any formal mathematical framework.

The formal description of BHA and its application is published in a number of books and papers. In this paper the BHA is introduced and explained using three examples as illustrations, namely knapsack, flow-shop, and batch scheduling. This helps to show when and why BHA works more efficiently as compared with the traditional optimization methods. The global optimization software is described in short as a tool needed implementing BHA.

The purpose of the paper is to inform the authors inventing and applying various heuristics about the possibilities and limitations of BHA hoping that they will improve their heuristics using this powerful tool.