Discrete Event Dynamic Systems

, Volume 19, Issue 3, pp 317–345 | Cite as

Ordinal Optimization and Quantification of Heuristic Designs

  • Zhen Shen
  • Yu-Chi Ho
  • Qian-Chuan Zhao


This paper focuses on the performance evaluation of complex man-made systems, such as assembly lines, electric power grid, traffic systems, and various paper processing bureaucracies, etc. For such problems, applying the traditional optimization tool of mathematical programming and gradient descent procedures of continuous variables optimization are often inappropriate or infeasible, as the design variables are usually discrete and the accurate evaluation of the system performance via a simulation model can take too much calculation. General search type and heuristic methods are the only two methods to tackle the problems. However, the “goodness” of heuristic methods is generally difficult to quantify while search methods often involve extensive evaluation of systems at many design choices in a large search space using a simulation model resulting in an infeasible computation burden. The purpose of this paper is to address these difficulties simultaneously by extending the recently developed methodology of Ordinal Optimization (OO). Uniform samples are taken out from the whole search space and evaluated with a crude but computationally easy model when applying OO. And, we argue, after ordering via the crude performance estimates, that the lined-up uniform samples can be seen as an approximate ruler. By comparing the heuristic design with such a ruler, we can quantify the heuristic design, just as we measure the length of an object with a ruler. In a previous paper we showed how to quantify a heuristic design for a special case but we did not have the OO ruler idea at that time. In this paper we propose the OO ruler idea and extend the quantifying method to the general case and the multiple independent results case. Experimental results of applying the ruler are also given to illustrate the utility of this approach.


Ordinal optimization Order statistics Discrete event dynamic systems Hypothesis testing 

Key notations




The whole search space


An element of the whole search space


A heuristic design


The true performance of a design


Observed performance of a design

Ni(i = 1,2,...,u)

A segment (set) of uniform samples, also the length of the segment when there is no ambiguity

N[r](r = 1,2,...,u)

r-th order statistics of the segments, i.e., r-th smallest


Bounding level for Type II error probability

n%, n0%

Denote the ordinal position of a design in \(\mathit{\Theta}\), n% and n 0% are within [0, 1]


A number defined as N 0: = 113.9/n 0%, called “standard length” in this paper



The authors thank Prof. Christos G. Cassandras of Boston University for insightful comments on the research in this paper, and thank him for helpful discussions in applying the methods in this paper to quantify the heuristic design for the routing problem of the two queues in Section 4.


  1. Armold DV (2002) Noisy optimization with evolution strategies. Springer, New YorkGoogle Scholar
  2. Balakrishnan N, Rao CR (1998) Handbook of statistics 17, order statistics: applications, edited. Elsevier, AmsterdamGoogle Scholar
  3. Cassandras CG, Lafortune S (1999) Introduction to discrete event systems. Kluwer Academic, DordrechtMATHGoogle Scholar
  4. Deng M, Ho YC (1999) An ordinal optimization approach to optimal control problems. Automatica 35:331–338MATHCrossRefMathSciNetGoogle Scholar
  5. Ho YC (1989) Introduction to special issue on dynamics of dynamics of discrete event systems. Proc IEEE 77(1):3–6CrossRefGoogle Scholar
  6. Ho YC (1999) An explanation of ordinal optimization: soft computing for hard problems. Inf Sci 113:169–192MATHCrossRefGoogle Scholar
  7. Ho YC (2005) On centralized optimal control. IEEE Trans Automat Contr 50(4):537–538CrossRefGoogle Scholar
  8. Ho YC, Sreenivas R, Vakili P (1992) Ordinal optimization of discrete event dynamic systems. J DEDS 2(2):61–88MATHGoogle Scholar
  9. Ho YC, Zhao QC, Jia QS (2007) Ordinal optimization: soft computing for hard problems. Springer, New YorkMATHGoogle Scholar
  10. Hopfield JJ, Tank DW (1985) “Neural” computation of decisions in optimization problems. Biol Cybern 52:141–152MATHMathSciNetGoogle Scholar
  11. Knapp AW (2005) Basic real analysis. Birkhäuser, Boston, p. 271Google Scholar
  12. Pinedo M (2002) Scheduling theory, algorithms, and systems (2nd ed). Prentice-Hall, New YorkMATHGoogle Scholar
  13. Shen Z, Bai H-X, Zhao YJ (2005) Ordinal optimization references list, updated May 2007.
  14. Shen Z, Zhao QC, Jia QS (2009) Quantifying heuristics in the ordinal optimization framework. J DEDS (under review)Google Scholar
  15. Wilson GV, Pawley GS (1988) On the stability of the traveling salesman problem algorithm of Hopfield and Tank. Biol Cybern 58:63–70MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Center for Intelligent and Networked Systems (CFINS), Department of Automation, TNLISTTsinghua UniversityBeijingPeople’s Republic of China
  2. 2.Department of Manufacturing Engineering and Center for Information and Systems EngineeringBoston UniversityBrooklineUSA
  3. 3.School of Engineering and Applied SciencesHarvard UniversityCambridgeUSA

Personalised recommendations