Abstract
The problem of binary optimization is discussed. By analyzing the generalized Hopfield model we obtain expressions describing the relationship between the depth of a local minimum and the size of the basin of attraction. The shape of local minima landscape is described. Based on this, we present the probability of finding a local minimum as a function of the depth of the minimum. Such a relation can be used in optimization applications: it allows one, basing on a series of already found minima, to estimate the probability of finding a deeper minimum, and to decide in favor of or against further running the program. It is shown, that the deepest minimum is defined with the gratest probability in random search. The theory is in a good agreement with experimental results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hopfield, J.J.: Neural Networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci.USA. v.79, pp.2554-2558 (1982)
Hopfield, J.J., Tank, D.W.: Neural computation of decisions in optimization problems. Biological Cybernetics, v.52, pp.141-152 (1985)
Fu, Y., Anderson, P.W.: Application of statistical mechanics to NP-complete problems in combinatorial optimization. Journal of Physics A., v.19, pp.1605-1620 (1986)
Poggio, T., Girosi, F.: Regularization algorithms for learning that are equivalent to multilayer networks. Science 247, pp.978-982 (1990)
Smith, K.A.: Neural Networks for Combinatorial Optimization: A Review of More Than a Decade of Research. INFORMS Journal on Computing v.11 (1), pp.15-34 (1999)
Hartmann, A.K., Rieger, H.: New Optimization Algorithms in Physics, Wiley-VCH, Berlin (2004)
Huajin Tang; Tan, K.C.; Zhang Yi: A Columnar Competitive Model for Solving Combinatorial optimization problems. IEEE Trans. Neural Networks v.15, pp.1568–1574 (2004)
Kwok, T., Smith, K.A.: A noisy self-organizing neural network with bifurcation dynamics for combinatorial optimization. IEEE Trans. Neural Networks v.15, pp.84–98 (2004)
Salcedo-Sanz, S.; Santiago-Mozos, R.; Bousono-Calzon, C.: A hybrid Hopfield network-simulated annealing approach for frequency assignment in satellite communications systems. IEEE Trans. Systems, Man and Cybernetics, v. 34, 1108–1116 (2004)
Wang, L.P., Li, S., Tian F.Y, Fu, X.J.: A noisy chaotic neural network for solving combinatorial optimization problems: Stochastic chaotic simulated annealing. IEEE Trans. System, Man, Cybern, Part B - Cybernetics v.34, pp.2119-2125 (2004)
Wang, L.P., Shi, H.: A gradual noisy chaotic neural network for solving the broadcast scheduling problem in packet radio networks. IEEE Trans. Neural Networks, vol.17, pp.989–1000 (2006)
Joya, G., Atencia, M., Sandoval, F.: Hopfield Neural Networks for Optimization: Study of the Different Dynamics. Neurocomputing, v.43, pp. 219-237 (2002)
Kryzhanovsky, B., Magomedov, B.: Application of domain neural network to optimization tasks. Proc. of ICANN’2005. Warsaw. LNCS 3697, Part II, pp.397-403 (2005)
Kryzhanovsky, B., Magomedov, B., Fonarev, A.: On the Probability of Finding Local Minima in Optimization Problems. Proc. of International Joint Conf. on Neural Networks IJCNN-2006 Vancouver, pp.5882-5887 (2006)
Kryzhanovsky, B.V.: Expansion of a matrix in terms of external products of configuration vectors. Optical Memory & Neural Networks, v. 17, No.1, pp.17-26 (2008)
Perez-Vincente, C.J.: Finite capacity of sparce-coding model. Europhys. Lett, v.10, pp.627-631 (1989)
Amit, D.J., Gutfreund, H., Sompolinsky, H.: Spin-glass models of neural networks. Physical Review A, v.32, pp.1007-1018 (1985)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Kryzhanovsky, B., Kryzhanovsky, V., Mikaelian, A. (2009). The Shape of a Local Minimum and the Probability of its Detection in Random Search. In: Filipe, J., Cetto, J.A., Ferrier, JL. (eds) Informatics in Control, Automation and Robotics. Lecture Notes in Electrical Engineering, vol 24. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85640-5_4
Download citation
DOI: https://doi.org/10.1007/978-3-540-85640-5_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-85639-9
Online ISBN: 978-3-540-85640-5
eBook Packages: EngineeringEngineering (R0)