Neural Networks Approaches for Combinatorial Optimization Problems

  • Theodore B. Trafalis
  • Suat Kasap

Abstract

Most of the engineering design problems and applications can be formulated as a nonlinear programming problem in which the objective function is nonlinear and has many local optima in its feasible region. It is desirable to find a local optimum that corresponds to the global optimum. The problem of finding the global optimum is known as the global optimization problem. Most such global optimization problems are difficult to solve. The main difficulties in finding the global optimum are that there are no operationally useful optimality conditions for identifying whether a point is indeed a global optimum, except in cases of special structured problems [33] and so it is computationally intensive to obtain the global optimum. Therefore, it is desirable and sometimes necessary to find a near global optimum in a reasonable time rather than obtaining the global optimum.

Keywords

Energy Function Travel Salesman Problem Travel Salesman Problem Knapsack Problem Combinatorial Optimization Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    S. Abe, J. Kawakami, and K. Hirasawa, Solving inequality constrained combinatorial optimization problems by the Hopfield neural networks, Neural Networks Vol. 5 (1992) pp. 663–670.CrossRefGoogle Scholar
  2. [2]
    D.H. Ackley, G.E.. Hinton, and T.J. Sejnowski, A learning algorithm for Boltzman machines, Cognitive Science Vol. 9 (1985) pp. 147–169.CrossRefGoogle Scholar
  3. [3]
    N. Ansari, E.S.H. Hou, and Y. Yu, A new method to optimize the satellite broadcasting schedules using the mean field annealing of a Hopfield neural network, IEEE Transactions on Neural Networks Vol. 6 No. 2 (1995) pp. 470–482.CrossRefGoogle Scholar
  4. [4]
    M. Bengtsson and P. Roivainen, Using the Potts glass for solving the clustering problem, International Journal of Neural Systems Vol. 6 No. 2 (1995) pp. 119–132.CrossRefGoogle Scholar
  5. [5]
    G. Bilbro, R. Mann, T.K. Miller, W.E. Snyder, D.E. Van Den Bout, and M. White, Optimization by mean field annealing, in D.S. Touretzky (ed.) Proceedings of the Annual Conferences on Advances in Neural Information Processing Systems, Volume 1, (Morgan Kaufmann Publishers, 1988 ) pp. 91–98.Google Scholar
  6. [6]
    A. Bovier and P. Picco, Mathematical Aspects of Spin Glasses and Neural Networks, (Birkhäuser, 1998 ).Google Scholar
  7. [7]
    N.E.G. Buchler, E.R.P. Zuiderweg, H. Wang, and R.A. Goldstein, Protein heteronuclear NMR assignments using mean-field simulated annealing, Journal of Magnetic Resonance Vol. 125 (1997) pp. 34–42.CrossRefGoogle Scholar
  8. [8]
    B. Cabon, G. Verfaillie, D. Martinez, and P. Bourret, Using mean field methods for boosting backtrack search in constraint satisfaction problems, in W. Wahlster (ed.) 12th European Conference on Artificial Intelligence, ( John Wiley and Sons, Ltd., 1996 ).Google Scholar
  9. [9]
    D. Chowdhury, Spin Glasses and Other Frustrated Systems, (Princeton University Press, 1986 ).Google Scholar
  10. [10]
    D. Chowdhury, Spin Glasses and Other Frustrated Systems, (Princeton University Press, 1986 ).Google Scholar
  11. [11]
    A. Cichocki and R. Unbehauen, Neural Networks for Optimization and Signal Processing, (John Wiley and Sons, 1993 ).Google Scholar
  12. [12]
    R. Courant and D. Hilbert, Methods of Mathematical Physics, (Inter-science Publishers Inc., 1953 ).Google Scholar
  13. [13]
    K.A. Dowsland, Simulated annealing, in C.R. Reeves (ed.) Modern Heuristic Techniques for Combinatorial Problems, (John Wiley and Sons, 1993 ) pp. 20–69.Google Scholar
  14. [14]
    R. Durbin, R. Szeliski, and A.L. Yuille, An analysis of the elastic net approach to the travelling salesman problem, Neural Computation Vold (1989) pp. 348–358.Google Scholar
  15. [15]
    I.M. Elfadel, Convex potential and their conjugates in analog mean-field optimization, Neural Computation Vol. 7 (1995) pp. 1079–1104.CrossRefGoogle Scholar
  16. [16]
    S. Elmohamed, P. Coddington, and G. Fox, A comparison of annealing techniques for academic course scheduling, Northeast Parallel Architectures Center (NPAC) technical report SCCS-777, January 25, 1997.Google Scholar
  17. [17]
    L. Fang and T. Li, Design of competition-based neural networks for combinatorial optimization, International Journal of Neural Systems Vold No. 3 (1990) pp. 221–235.Google Scholar
  18. [18]
    L. Fang, W.H. Wilson, and T. Li, Mean field annealing neural net for quadratic assignment, in International Neural Network Conference, July 9–13, Paris, France,(Kluwer Academic Publishers, 1990) pp. 282286.Google Scholar
  19. [19]
    L. Fausett, Fundamentals of Neural Networks: Architectures, Algorithms, and Applications, (Prentice-Hall, 1994 ).Google Scholar
  20. [20]
    L. Faybusovich, Interior point methods and entropy, IEEE Conference on Decision and Control, (1991) pp. 2094–2095.Google Scholar
  21. [21]
    K.H. Fischer and J.A. Hertz, Spin Glasses, (Cambridge University Press, 1991 ).Google Scholar
  22. [22]
    C.A. Floudas and P.M. Pardalos, Recent Advances in Global Optimization, (Princeton University Press, 1994 ).Google Scholar
  23. [23]
    D. Geiger and F. Girosi, Coupled Markov random fields and mean field theory, in D.S. Touretzky (ed.) Proceedings of the Annual Conferences on Advances in Neural Information Processing Systems, Volume 2, (Morgan Kaufmann Publishers, 1989 ) pp. 660–667.Google Scholar
  24. [24]
    L. Gislen, C. Peterson, and B. Soderberg, Teachers and classes with neural networks, International Journal of Neural Systems Vol.1 No. 1 (1989) pp. 167–183.CrossRefGoogle Scholar
  25. [25]
    L. Gislen, C. Peterson, and B. Soderberg, Complex scheduling with Potts neural networks, Neural Computation Vol. 4 (1992) pp. 805–831.CrossRefGoogle Scholar
  26. [26]
    R.J. Glauber, Time-dependent statistics of the Ising model, Journal of Mathematical Physics Vol. 4 No. 2 (1963) pp. 294–307.MathSciNetMATHCrossRefGoogle Scholar
  27. [27]
    S. Haykin, Neural Networks: A Comprehensive Foundation, (Macmillan College Publishing Company, 1994 ).Google Scholar
  28. [28]
    L. Herault and J.-J. Niez, Neural networks and combinatorial optimization: a study of NP-complete graph problems, in E. Gelenbe (ed.) Neural Networks: Advances and Applications, ( Elsevier Science Publishers B. V., 1991 ) pp. 165–213.Google Scholar
  29. [29]
    T. Hofmann and J.M. Buhmann, Pairwise data clustering by deterministic annealing, IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 19 No. 1 (1997) pp. 1–14.CrossRefGoogle Scholar
  30. [30]
    J.J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, in Proceedings of the National Academy of Sciences of the U.S.A., Biophysics, (1982) pp. 2554–2558.Google Scholar
  31. [31]
    J.J. Hopfield, Neurons with graded response have collective computational properties like those of two-state neurons, in Proceedings of the National Academy of Sciences of the U.S.A., Biophysics, (1984) pp. 3088–3092.Google Scholar
  32. [32]
    J.J. Hopfield and D.W. Tank, “Neural” computation of decisions in optimization problems, Biological Cybernetics Vol.52 (1985) pp. 141–152.Google Scholar
  33. [33]
    R. Horst, P.M. Pardalos, and N.V. Thoai, Introduction to Global Optimization, (Kluwer Academic Publishers, 1995 ).Google Scholar
  34. [34]
    H. Igarashi, A solution to combinatorial optimization problems using a two-layer random field model: Mean-field approximation, in World Congress on Neural Networks, July 11–15, Portland, Oregon, Volume 1, ( Lawrance Erlbaum Associates Inc. Publishers, 1993 ) pp. 283–286.Google Scholar
  35. [35]
    H. Igarashi, A solution for combinatorial optimization problems using a two-layer random field model: Mean-field approximation, Systems and Computers in Japan Vol. 25 No. 8 (1994) pp. 61–71.CrossRefGoogle Scholar
  36. [36]
    S. Ishii and M.-A. Sato, Chaotic Potts spin model for combinatorial optimization problems, Neural Networks Vol.10 No. 5 (1997) pp. 941963.Google Scholar
  37. [37]
    I. Kanter and H. Sompolinsky, Graph optimisation problems and the Potts glass, Journal of Physics A: Math. Gen. Vol. 20 (1987) pp. L673 - L679.MathSciNetCrossRefGoogle Scholar
  38. [38]
    S. Kirkpatrick, C.D. Gelatt, and M.P. Vecchi, Optimization by simulated annealing, Science Vol. 220 No. 4598 (1983) pp. 671–680.MathSciNetMATHCrossRefGoogle Scholar
  39. [39]
    J.J. Kosowsky and A.L. Yuille, The invisible hand algorithm: Solving the assignment problem with statistical physics, Neural Networks Vol. 7 No. 3 (1994) pp. 477–490.MATHCrossRefGoogle Scholar
  40. [40]
    N. Kurita and K.-I. Funahashi, On the Hopfield neural networks and mean field theory, Neural Networks Vol. 9 No. 9 (1996) pp. 1531–1540.MATHCrossRefGoogle Scholar
  41. [41]
    T. Kurokawa and S. Kozuka, Use of neural networks for the optimum frequency assignment problem, Electronics and Communications in Japan, Part 1 Vol. 77 No. 11 (1994) pp. 106–116.Google Scholar
  42. [42]
    M. Lagerholm, C. Peterson, and B. Soderberg, Airline crew scheduling with Potts neurons, Neural Computation Vol. 9 (1997) pp. 1589–1599.CrossRefGoogle Scholar
  43. [43]
    K.-C. Lee and Y. Takefuji, Maximum clique problems: Part 1, in Y. Takefuji and J. Wang (eds.) Neural Computing for Optimization and Combinatorics, (World Scientific, 1996 ) pp. 31–61.CrossRefGoogle Scholar
  44. [44]
    K.-C. Lee and Y. Takefuji, Maximum clique problems: Part 2, in Y. Takefuji and J. Wang (eds.) Neural Computing for Optimization and Combinatorics, (World Scientific, 1996 ) pp. 63–77.Google Scholar
  45. [45]
    B.C. Levy and M.B. Adams, Global optimization with stochastic neural networks, in Neural Networks for Optimization and Signal Processing, Proceedings of the First International Conference on Neural Networks, San Diego,(1987) pp. 681–690.Google Scholar
  46. [46]
    W.A. Little, The existence of persistent states in the brain, Mathematical Biosciences Vol. 19 (1974) pp. 101–120.MATHCrossRefGoogle Scholar
  47. [47]
    C.-K. Looi, Neural network methods in combinatorial optimization, Computers in Operations Research Vol.19 No. 3 /4 (1992) pp. 191–208.Google Scholar
  48. [48]
    S. Matsuda, Set-theoretic comparison of mapping of combinatorial optimization problems to Hopfield neural networks, Systems and Computers in Japan Vol. 27 No. 6 (1996) pp. 45–59.CrossRefGoogle Scholar
  49. [49]
    W.S. McCullough and W. Pitts, A logical calculus of the ideas immanent in nervous activity, Bulletin of Mathematical Biophysics Vol. 5 (1943) pp. 115–133.CrossRefGoogle Scholar
  50. [50]
    I.I. Melamed, Neural networks and combinatorial optimization, Automation and Remote Control Vol. 55 No. 11 (1994) pp. 1553–1584.MathSciNetMATHGoogle Scholar
  51. [51]
    M. Mezard, G. Parisi, and M.A. Virasoro, Spin Glass Theory and Beyond, (World Scientific, 1987 ).Google Scholar
  52. [52]
    B. Muller and J. Reinhardt, Neural Networks: An Introduction, (Springer-Verlag, 1990 ).Google Scholar
  53. [53]
    M. Ohlsson, C. Peterson, and B. Soderberg, Neural networks for optimization problems with inequality constraints: The knapsack problem, Neural Computation Vol. 5 (1993) pp. 331–339.CrossRefGoogle Scholar
  54. [54]
    M. Ohlsson and H. Pi, A study of the mean field approach to knapsack problems, Neural Networks Vol. 10 No. 2 (1997) pp. 263–271.CrossRefGoogle Scholar
  55. [55]
    C.H. Papadimitriou and K. Steiglitz, Combinatorial Optimization: Algorithms and Complexity, (Prentice-Hall, 1982 ).Google Scholar
  56. [56]
    C. Peterson, Parallel distributed approaches to combinatorial optimization: Benchmark studies on traveling salesman problem, Neural Computation Vol. 2 (1990) pp. 261–269.CrossRefGoogle Scholar
  57. [57]
    C. Peterson, Mean field theory neural networks for feature recognition, content addressable memory and optimization, Connection Science Vol. 3 No. 1 (1991) pp. 3–33.CrossRefGoogle Scholar
  58. [58]
    C. Peterson, Solving optimization problems with mean field methods, Physica A Vol. 200 (1993) pp. 570–580.CrossRefGoogle Scholar
  59. [59]
    C. Peterson and J.R. Anderson, A mean field theory learning algorithm for neural networks, Complex Systems Vol.1 (1987) pp. 995–1019.Google Scholar
  60. [60]
    C. Peterson and J.R. Anderson, Neural networks and NP-complete optimization problems; a performance study on the graph bisection problem, Complex Systems Vol. 2 (1988) pp. 59–89.MathSciNetMATHGoogle Scholar
  61. [61]
    C. Peterson and B. Soderberg, A new method for mapping optimization problems onto neural networks, International Journal of Neural systems Vol. 1 No. 1 (1989) pp. 3–22.CrossRefGoogle Scholar
  62. [62]
    C. Peterson and B. Soderberg, Artificial neural networks, in C.R. Reeves (ed.) Modern Heuristic Techniques for Combinatorial Problems, (John Wiley and Sons, 1993 ) pp. 197–242.Google Scholar
  63. [63]
    C. Peterson and B. Soderberg, Artificial neural networks, in E. Aarts and J.K. Lenstra (eds.) Local Search in Combinatorial Optimization, (John Wiley and Sons, 1997 ) pp. 177–213.Google Scholar
  64. [64]
    F. Qian and H. Hirata, A parallel computation based on mean field theory for combinatorial optimization and Boltzman machines, Systems and Computers in Japan Vol. 25 No. 12 (1994) pp. 86–97.Google Scholar
  65. [65]
    J. Ramanujam and P. Sadayappan, Mapping combinatorial optimization problems onto neural networks, Information Sciences Vol. 82 (1995) pp. 239–255.MathSciNetCrossRefGoogle Scholar
  66. [66]
    C.R. Reeves, Modern Heuristic Techniques for Combinatorial Problems, (John Wiley and Sons, 1993 ).Google Scholar
  67. [67]
    L.E. Reichl, A Modern Course in Statistical Physics, (University of Texas Press, 1980 ).Google Scholar
  68. [68]
    D.E. Rumelhart, G.E. Hinton, and R.J. Williams, Learning representations by back-propagating errors, Nature (London) Vol. 323 (1986) pp. 533–536.CrossRefGoogle Scholar
  69. [69]
    M.-A. Sato and S. Ishii, Bifurcations in mean field theory, Physical Review E Vol. 53 No. 5 (1996) pp. 5153–5168.CrossRefGoogle Scholar
  70. [70]
    G.M. Shim, D. Kim, and M.Y. Choi, Potts-glass model of layered feed-forward neural networks, Physical Review A Vol. 45 No. 2 (1992) pp. 1238–1248.MathSciNetCrossRefGoogle Scholar
  71. [71]
    P. Stolorz, Merging constrained optimisation with deterministic annealing to “solve” combinatorially hard problems, in J.E. Moody, S.J. Hanson, and R.P. Lippmann (eds.) Advances in Neural Information Processing Systems, ( Chapman and Hall, 1991 ) pp. 1025–1032.Google Scholar
  72. [72]
    Y. Uesaka, Mathematical basis of neural networks for combinatorial optirnization problems, Optoelectronics–Devices and Technologies Vol. 8 No. 1 (1993) pp. 1–9.Google Scholar
  73. [73]
    K. Urahama, Mathematical programming formulation for neural combinatorial optimization algorithms, Electronics and Communications in Japan Vol. 78 No. 9 (1995) pp. 67–75.CrossRefGoogle Scholar
  74. [74]
    K. Urahama and S.-I. Ueno, A gradient system solution to Potts mean field equations and its electronic implementation, International Journal of Neural Systems Vol. 4 No. 1 (1993) pp. 27–34.CrossRefGoogle Scholar
  75. [75]
    K. Urahama and T. Yamada, Constrained Potts mean field systems and their electronic implementation, International Journal of Neural Systems Vol. 5 No. 3 (1994) pp. 229–239.CrossRefGoogle Scholar
  76. [76]
    D.E. Van Den Bout and T.K. Miller, Graph partitioning using annealed networks, IEEE Transactions on Neural Networks Vol.]. No. 2 (1990) pp. 192–203.CrossRefGoogle Scholar
  77. [77]
    P.J.M. Van Laarhoven, Theoretical and Computational Aspects of Simulated Annealing, (Stichting Mathematisch Centrum, 1988 ).Google Scholar
  78. [78]
    J. Wang, Deterministic neural networks for combinatorial optimization, in O.M. Omidvar (ed.) Progress in Neural Networks, (Ablex, 1994 ) pp. 319–340.Google Scholar
  79. [79]
    F.Y. Wu, The Potts model, Reviews of Modern Physics Vol. 54 No. 1 (1982) pp. 235–268.MathSciNetCrossRefGoogle Scholar
  80. [80]
    L. Xu and A.L. Yuille, Robust principal component analysis by self-organizing rules based on statistical physics approach, IEEE Transactions on Neural Networks Vol. 6 No. 1 (1995) pp. 131–143.CrossRefGoogle Scholar
  81. [81]
    A.L. Yuille, Generalized Deformable models, statistical physics, and matching problems, Neural Computation Vol. 2 (1990) pp. 1–24.CrossRefGoogle Scholar
  82. [82]
    A.L. Yuille and J.J. Kosowsky, Statistical physics algorithms that converge, in R.J. Mammone (ed.) Artificial Neural Networksfor Speech and Vision,(Chapman and Hall, 1993 ) pp. 19–36.Google Scholar
  83. [83]
    A.L. Yuille and J.J. Kosowsky, Statistical physics algorithms that converge, Neural Computation Vol. 6 (1994) pp. 341–356.CrossRefGoogle Scholar
  84. [84]
    A.L. Yuille, P. Stolorz, and J. Utans, Statistical physics, mixtures of distributions, and the EM algorithm, Neural Computation Vol. 6 (1994) pp. 334–340.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 1999

Authors and Affiliations

  • Theodore B. Trafalis
    • 1
  • Suat Kasap
    • 1
  1. 1.School of Industrial EngineeringUniversity of OklahomaNormanUSA

Personalised recommendations