An Empirical Evaluation of Some Multiple Criteria Methods for Discrete Alternatives

Abstract

A number of techniques have been developed for evaluating preferences in decision problems involving discrete alternatives and multiple criteria. There have been relatively few studies attempting to compare different approaches. In this paper we present some preliminary results of a study of some algorithms. Six approaches are considered: one equal-weights approach and five linear programming (LP) based techniques. The LP based techniques include the Zionts approach and four techniques of Srinivasan and Shocker available in their LINMAP program package.

These six methods were used to solve an initial set of 23 problems of predetermined size. Three of these problems are examples from the literature. The next set of 20 problems consisted of 30 alternatives and 5 criteria each; the latter were generated by using a uniform distribution of random numbers. An underlying “true” linear utility function is assumed and is the basis for providing the input or decision maker responses needed by the algorithms.

The equal-weights approach was then tested further, using an additional 240 problems (generated with all non-dominated alternatives using a uniform random number distribution) with 4 variations in problem sizes and under 3 different assumptions of the type of the underlying “true” utility function.

The performance of the algorithms, on a number of prespecified criteria, is discussed. Briefly, the Srinivasan and Shocker approaches generally did best, but required the most input on the part of the user; the Zionts approach did quite well and required only modest inputs from the user, and equal weights did worst. However, equal weights did much better than might be expected by chance.

This study is currently being further expanded to include more algorithms, and the effects of the presence of decision maker response errors, and nonlinear utility functions.