Abstract
The problem of binary optimization of a quadratic functional is discussed. By analyzing the generalized Hopfield model we obtain expressions describing the relationship between the depth of a local minimum and the size of the basin of attraction. Based on this, we present the probability of finding a local minimum as a function of the depth of the minimum. Such a relation can be used in optimization applications: it allows one, basing on a series of already found minima, to estimate the probability of finding a deeper minimum, and to decide in favor of or against further running the program. The iterative algorithm that allows us to represent any symmetric N ×N matrix as a weighted Hebbian series of bipolar vectors with a given accuracy is proposed. It so proves that all conclusions about neural networks and optimization algorithms that are based on Hebbian matrices are true for any other type of matrix. The theory is in a good agreement with experimental results.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Hopfield, J.J.: Neural Networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci.USA. 79, 2554–2558 (1982)
Hopfield, J.J., Tank, D.W.: Neural computation of decisions in optimization problems. Biological Cybernetics 52, 141–152 (1985)
Fu, Y., Anderson, P.W.: Application of statistical mechanics to NP-complete problems in combinatorial optimization. Journal of Physics A 19, 1605–1620 (1986)
Poggio, T., Girosi, F.: Regularization algorithms for learning that are equivalent to multilayer networks. Science 247, 978–982 (1990)
Smith, K.A.: Neural Networks for Combinatorial Optimization: A Review of More Than a Decade of Research. INFORMS Journal on Computing 11(1), 15–34 (1999)
Joya, G., Atencia, M., Sandoval, F.: Hopfield Neural Networks for Optimization: Study of the Different Dynamics. Neurocomputing 43, 219–237 (2002)
Hartmann, A.K., Rieger, H.: New Optimization Algorithms in Physics. Wiley-VCH, Berlin (2004)
Tang, H., Tan, K.C., Yi, Z.: A columnar competitive model for solving combinatorial optimization problems. IEEE Trans. Neural Networks 15, 1568–1574 (2004)
Kwok, T., Smith, K.A.: A noisy self-organizing neural network with bifurcation dynamics for combinatorial optimization. IEEE Trans. Neural Networks 15, 84–98 (2004)
Salcedo-Sanz, S., Santiago-Mozos, R., Bousono-Calzon, C.: A hybrid Hopfield network- simulated annealing approach for frequency assignment in satellite communications systems. IEEE Trans. Systems, Man and Cybernetics 34, 1108–1116 (2004)
Wang, L.P., Li, S., Tian, F.Y., Fu, X.J.: A noisy chaotic neural network for solving combinatorial optimization problems: Stochastic chaotic simulated annealing. IEEE Trans. System, Man, Cybern, Part B - Cybernetics 34, 2119–2125 (2004)
Kryzhanovsky, B., Magomedov, B.: Application of Domain Neural Network to Optimization Tasks. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 397–403. Springer, Heidelberg (2005)
Kryzhanovsky, B.V., Magomedov, B.M., Mikaelyan, A.L.: A Relation Between the Depth of a Local Minimum and the Probability of Its Detection in the Generalized Hopfield Model. Doklady Mathematics 72(3), 986–990 (2005)
Kryzhanovsky, B., Magomedov, B., Fonarev, A.: On the Probability of Finding Local Minima in Optimization Problems. In: Proc. of International Joint Conf. on Neural Networks IJCNN-2006 Vancouver, pp. 5882–5887 (2006)
Wang, L.P., Shi, H.: A gradual noisy chaotic neural network for solving the broadcast scheduling problem in packet radio networks. IEEE Trans. Neural Networks 17, 989–1000 (2006)
Litinskii, L.B., Magomedov, B.M.: Global Minimization of a Quadratic Functional: Neural Networks Approach. Pattern Recognition and Image Analysis 15(1), 80–82 (2005)
Perez-Vincente, C.J.: Finite capacity of sparce-coding model. Europhys. Lett. 10, 627–631 (1989)
Amit, D.J., Gutfreund, H., Sompolinsky, H.: Spin-glass models of neural networks. Physical Review A 32, 1007–1018 (1985)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kryzhanovsky, B., Kryzhanovsky, V. (2008). Binary Optimization: On the Probability of a Local Minimum Detection in Random Search. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing – ICAISC 2008. ICAISC 2008. Lecture Notes in Computer Science(), vol 5097. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69731-2_10
Download citation
DOI: https://doi.org/10.1007/978-3-540-69731-2_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69572-1
Online ISBN: 978-3-540-69731-2
eBook Packages: Computer ScienceComputer Science (R0)