Analyzing the Performance of Local Search Algorithms Using Generalized Hill Climbing Algorithms

  • Sheldon H. Jacobson
Part of the Operations Research/Computer Science Interfaces Series book series (ORCS, volume 15)

Abstract

Generalized hill climbing algorithms provide a well-defined framework to model how local search algorithms address intractable discrete optimization problems. Monte Carlo search, simulated annealing, threshold accepting, and tabu search, among others, can all be formulated as particular generalized hill climbing algorithms. Moreover, generalized hill climbing algorithms provide a structure for classifying and studying a large body of stochastic and deterministic local search algorithms. In particular, the generalized hill climbing algorithm framework can be used to develop a general Markov chain model for the application of local search algorithms to intractable discrete optimization problems. Moreover, the generalized hill climbing algorithm framework has been used to evaluate the finite-time performance of local search algorithms. This analysis is of particular interest to practitioners for whom traditional convergence results are often of limited interest and value. This paper presents a survey of recent results on how the generalized hill climbing algorithm framework has been used to model and gain insight into the performance of local search algorithms. Many of these results provide tools for comparing and evaluating different types of local search algorithms using common performance measures.

Keywords

Entropy Boiling Hull Ceria Bors 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    E.H.L. Aarts, J.H.M. Korst, and P.J.M. van Laarhoven. Simulated Annealing. In: Local Search in Combinatorial Optimization E.H.L. Aarts and J.K. Lenstra, editors, pages 91–120, Wiley, 1997.Google Scholar
  2. [2]
    E.H.L. Aarts and J.K. Lenstra. Local Search in Combinatorial Optimization, Wiley, 1997.MATHGoogle Scholar
  3. [3]
    I. Althofer and K.U. Koschnick. On the Convergence of Thresholding Accepting. Applied Mathematics and Optimization, 24:183–195, 1991.MathSciNetCrossRefGoogle Scholar
  4. [4]
    E. Balas and A. Vazacopoulos. Guided Local Search with Shifting Bottleneck for Job Shop Scheduling. Management Science, 44:262–275, 1998.MATHCrossRefGoogle Scholar
  5. [5]
    O. Catoni. Metropolis, Simulated Annealing, and Iterated Energy Transformation Algorithms: Theory and Experiments. Journal of Complexity, 12:595–623, 1996.MathSciNetMATHCrossRefGoogle Scholar
  6. [6]
    S. Ceria, P. Nobili, and A. Sassano. A Lagrangian-Based Heuristic for Large-Scale Set Covering Problems. Mathematical Programming, 81:215–228, 1998.MathSciNetMATHGoogle Scholar
  7. [7]
    I. Charon and O. Hudry. The Noising Method: A New Method for Combinatorial Optimization. Operations Research Letters, 14:133–137, 1993.MathSciNetMATHCrossRefGoogle Scholar
  8. [8]
    T.S. Chiang and Y.Y. Chow. A Limit-Theorem for a Class of Inhomoge-neous Markov-Processes. Annals of Probability, 17:1483–1502, 1989.MathSciNetMATHCrossRefGoogle Scholar
  9. [9]
    H. Cohn and M. Fielding. Simulated Annealing: Searching for an Optimal Temperature Schedule. SIAM Journal on Optimization, 9:779–802, 1999.MathSciNetMATHCrossRefGoogle Scholar
  10. [10]
    S.A. Cook. The Complexity of Theorem-Proving Procedures. In: Proceedings of the Third Annual ACM Symposium on Theory of Computing, Association for Computing Machinery, New York, pages 151–158, 1971.CrossRefGoogle Scholar
  11. [11]
    C.R. Coullard, A.B. Gamble, and P.C. Jones. Matching Problems in Selective Assembly Operations. Annals of Operations Research, 76:95–107, 1998.MATHCrossRefGoogle Scholar
  12. [12]
    M.P. Desai. Some Results Characterizing the Finite Time Behaviour of the Simulated Annealing Algorithm. Sadhana-Academy Proceedings in Engineering Sciences, 24:317–337, 1999.MathSciNetMATHGoogle Scholar
  13. [13]
    G. Dueck and T. Scheuer. Thresholding Accepting: A General Purpose Optimization Algorithm Appearing Superior to Simulated Annealing. European Journal of Operational Research, 46:271–281, 1990.MathSciNetCrossRefGoogle Scholar
  14. [14]
    R.W. Eglese. Simulated Annealing: A Tool for Operational Research. European Journal of Operational Research, 46:271–281, 1990.MathSciNetMATHCrossRefGoogle Scholar
  15. [15]
    U. Faigle and W. Kern. Some Convergence Results for Probabilistic Tabu Search. ORSA Journal on Computing, 4:32–37, 1992.MATHCrossRefGoogle Scholar
  16. [16]
    A.G. Ferreira and J. Zerovnik. Bounding the Probability of Success of Stochastic Methods for Global Optimization. Computers and Mathematics Applications, 25:1–8, 1993.MathSciNetMATHCrossRefGoogle Scholar
  17. [17]
    M.A. Fleischer. Simulated Annealing: Past, Present, and Future. In: Proceedings of the 1995 Winter Simulation Conference, C. Alexopoulus, K. Kang, W.R. Lilegdon, and D. Goldsman, editors, pages 155–161, Institute of Electrical and Electronics Engineers Press, 1995.CrossRefGoogle Scholar
  18. [18]
    M.A. Fleischer and S.H. Jacobson. Information Theory and the Finite-Time Behavior of the Simulated Annealing Algorithm: Experimental Results. INFORMS Journal on Computing, 11:35–43, 1999.MathSciNetMATHCrossRefGoogle Scholar
  19. [19]
    B.L. Fox. Integrating and Accelerating Tabu Search, Simulated Annealing, and Genetic Algorithms. Annals of Operations Research, 41:47–67, 1993.MATHCrossRefGoogle Scholar
  20. [20]
    B.L. Fox. Random Restarting versus Simulated Annealing. Computers and Mathematics Applications, 27:33–35, 1994.MATHCrossRefGoogle Scholar
  21. [21]
    B.L. Fox. Faster Simulated Annealing. SIAM Journal of Optimization, 5:488–505, 1995.MATHCrossRefGoogle Scholar
  22. [22]
    M.R. Garey and D.S. Johnson. Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman and Company, 1979.MATHGoogle Scholar
  23. [23]
    F. Glover and M. Laguna. Tabu Search. Kluwer, 1997.MATHCrossRefGoogle Scholar
  24. [24]
    P. Hansen and B. Jaumard. Algorithms for the Maximum Satisfiability Problem. Computing, 44:279–303, 1990.MathSciNetMATHCrossRefGoogle Scholar
  25. [25]
    B. Hajek. Cooling Schedules for Optimal Annealing. Mathematics of Operations Research, 13:311–329, 1988.MathSciNetMATHCrossRefGoogle Scholar
  26. [26]
    Y.-C. Ho, R. Sreenivas, and P. Vakili. Ordinal Optimization of Discrete Event Dynamic Systems. Discrete Event Dynamical Systems, 2:61–88, 1992.MATHCrossRefGoogle Scholar
  27. [27]
    D.S. Hochbaum and A. Pathria. Generalized p-Center Problems: Complexity Results and Approximation Algorithms. European Journal of Operational Research, 100:594–607, 1997.MATHCrossRefGoogle Scholar
  28. [28]
    S.H. Jacobson, A.W. Johnson, K.A. Sullivan, M.A. Fleischer, and A. Kumar. Metaheuristics for a Flexible Assembly System Design Problem. Journal of Heuristics, 3:139–159, 1997.MATHCrossRefGoogle Scholar
  29. [29]
    S.H. Jacobson, K.A. Sullivan, and A.W. Johnson. Generalized Hill Climbing Algorithms for Discrete Manufacturing Process Design Problems using Computer Simulation Models. In: Proceedings of the 11 th European Simulation Multiconference, pages 473–478, 1997.Google Scholar
  30. [30]
    S.H. Jacobson, K.A. Sullivan, and A.W. Johnson. Discrete Manufacturing Process Design Optimization Using Computer Simulation and Generalized Hill Climbing Algorithms. Engineering Optimization, 31:247–260, 1998.CrossRefGoogle Scholar
  31. [31]
    S.H. Jacobson and E. Yücesan. On the Complexity of Verifying Structural Properties of Discrete Event Simulation Models. Operations Research, 47:476–481, 1999.MATHCrossRefGoogle Scholar
  32. [32]
    S.H. Jacobson and E. Yücesan. On the Effectiveness of Generalized Hill Climbing Algorithms. Technical Report, University of Illinois, 2001.Google Scholar
  33. [33]
    S.H. Jacobson and E. Yücesan. Assessing the Performance of Generalized Hill Climbing Algorithms. Technical Report, University of Illinois, 2001.Google Scholar
  34. [34]
    A.W. Johnson and S.H. Jacobson. A Convergence Result for a Class of Generalized Hill Climbing Algorithms. To appear in: Applied Mathematics and Computation. Google Scholar
  35. [35]
    A.W. Johnson and S.H. Jacobson. On the Convergence of Generalized Hill Climbing Algorithms. To appear in: Discrete Applied Mathematics. Google Scholar
  36. [36]
    E.L. Johnson and G.L. Nemhauser. Recent Developments and Future-Directions in Mathematical Programming. IBM Systems Journal, 31:79–93, 1992.MATHCrossRefGoogle Scholar
  37. [37]
    A. Kumar, S.H. Jacobson, and E.C. Sewell. Computational Analysis of a Flexible Assembly System Design Problem. European Journal Operational Research, 123:17–36, 2000.CrossRefGoogle Scholar
  38. [38]
    A.M. Law and W.D. Kelton. Simulation Modeling and Analysis, McGraw Hill, 1991.Google Scholar
  39. [39]
    G.E. Liepins and M.R. Hilliard. Genetic Algorithms: Foundations and Applications. Annals of Operations Research, 21:31–58, 1989.MATHCrossRefGoogle Scholar
  40. [40]
    C. Mazza. Parallel Simulated Annealing. Random Structures and Algorithms, 3:139–148, 1992.MathSciNetMATHCrossRefGoogle Scholar
  41. [41]
    D. Mitra, F. Romeo, and A. Sangiovanni-Vincentelli. Convergence and Finite-Time Behavior of Simulated Annealing. Advances in Applied Probability, 18:747–771, 1986.MathSciNetMATHCrossRefGoogle Scholar
  42. [42]
    H. Mühlenbein. Genetic Algorithms. In: Local Search in Combinatorial Optimization, E.H.L. Aarts and J.K. Lenstra, editors, pages 137–172, Wiley 1997.Google Scholar
  43. [43]
    G.L. Nemhauser. The Age of Optimization — Solving Large-Scale Real-World Problems. Operations Research, 42:5–13, 1994.MATHCrossRefGoogle Scholar
  44. [44]
    S.M. Ross. A First Course in Probability, Macmillan Publishing Company, 1988.Google Scholar
  45. [45]
    P.C. Schuur. Classification of Acceptance Criteria for the Simulated Annealing Algorithm. Mathematics of Operations Research, 22:266–275,1997.MathSciNetMATHCrossRefGoogle Scholar
  46. [46]
    L. Shi and S. Olafsson. Nested Partitions Method for Global Optimization. Operations Research, 48:390–407, 2000.MathSciNetMATHCrossRefGoogle Scholar
  47. [47]
    R. Srichander. Efficient Schedules for Simulated Annealing. Engineering Optimization, 24:161–176, 1995.CrossRefGoogle Scholar
  48. [48]
    K.A. Sullivan and S.H. Jacobson. Ordinal Hill Climbing Algorithms for Discrete Manufacturing Process Design Optimization Problems. Discrete Event Dynamical Systems, 10:307–324, 2000.MathSciNetMATHCrossRefGoogle Scholar
  49. [49]
    K.A. Sullivan and S.H. Jacobson. A Convergence Analysis of Generalized Hill Climbing Algorithms. To appear in: IEEE Transactions on Automatic Control Google Scholar
  50. [50]
    C.A. Tovey. Local Improvement on Discrete Structures. In: Local Search in Combinatorial Optimization, E.H.L. Aarts and J.K. Lenstra, editors, pages 57–89, Wiley, 1997.Google Scholar
  51. [51]
    M. Yannakakis. Computational Complexity. In: Local Search in Combinatorial Optimization, E.H.L. Aarts and J.K. Lenstra, editors, pages 19–55, Wiley, 1997.Google Scholar
  52. [52]
    E. Yücesan and S.H. Jacobson. Computational Issues for ACCESSIBILITY in Discrete Event Simulation. ACM Transactions on Modeling and Computer Simulation, 6:53–75, 1996.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2002

Authors and Affiliations

  • Sheldon H. Jacobson
    • 1
  1. 1.Department of Mechanical and Industrial EngineeringUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations