Abstract
Unlearning, a reverse process to learning according to Hebb's rule, is a local and unsupervised procedure that gives rise to a substantial improvement of the retrieval properties of an associative neural network: (i) an enhancement of both the storage capacity and the domains of attraction, (ii) the possibility to store correlated patterns, and (iii) the capability to distinguish between patterns and non-retrieval states. Three different versions of this type of algorithm are reviewed and the basic properties of these algorithms are investigated. We demonstrate that the same microscopic mechanism underlies all three of them. Furthermore, unlearning is applied succesfully to the storage of temporal sequences of correlated patterns which have been learned in a purely Hebbian way.
This is a preview of subscription content, log in via an institution.
Preview
Unable to display preview. Download preview PDF.
References
D.O. Hebb, The organization of behavior. Wiley, New York (1949)
D. Amit, H. Gutfreund, H. Sompolinsky, Statistical mechanics of neural networks near saturation. Ann. Phys. N.Y. 173, 30–67 (1987)
E. Gardner, The space of interactions in neural network models. J. Phys. A: Math. Gen. 21, 257–270 (1988)
J.J. Hopfield, D.I. Feinstein, R.G. Palmer, Unlearning has stabilizing effects in collective memories, Nature 304,158–9 (1983)
F. Crick, G. Mitchinson The function of dream sleep, Nature 304, 111–114 (1983)
A. Plakhov, S. Semenov, Unlearning-type procedures for reaching a perfect storage in neural networks. Moscow, preprint (1992)
J.L. van Hemmen, L.B. Ioffe, R. Kühn, M. Vaas, Increasing the efficiency of a neural network through unlearning. Physica A 163, 386–392 (1990)
J.L. van Hemmen, N. Klemmer, Unlearning and its relevance to REM sleep: Decorrelating correlated data. In: J.G. Taylor, E.R. Caianiello, R.M.J. Cotterill and J.W. Clark (Eds.) Neural network dynamics. London:Springer, 30–43 (1992)
S. Wimbauer, N. Klemmer, J.L. van Hemmen, Universality of unlearning, Neural Networks (to appear)
J.J. Hopfield, Neural Networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982)
W.A. Little, The existence of persistent states in the brain. Math. Biosci. 19, 101–120 (1974)
A.V.M. Herz, Z. Li, J.L. van Hemmen, Statistical mechanics of temporal association in neural networks with transmisson delays. Phys. Rev. Lett. 66, 1370–73 (1991)
I. Kanter, H. Sompolinsky, Associative recall of memory without errors. Phys. Rev. A 35, 380–392 (1987)
W. Kinzel, M. Opper, Dynamics of learning, in: E. Domany, J.L. van Hemmen, K. Schulten (Eds.) Models of neural networks. Springer, Berlin, 149–172 (1991)
A.V.M. Herz, B. Sulzer, R. Kühn, J. L. van Hemmen, The Hebb rule: Storing static and dynamic objects in an associative neural network. Europhys. Lett. 7, 663–669 (1988)
A.V.M. Herz, B. Sulzer, R. Kühn, J. L. van Hemmen, Hebbian learning reconsidered: Representation of static and dynamic objects in associative neural nets. Biol. Cybern. 60, 457–467 (1989)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1995 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wimbauer, S., van Hemmen, J.L. (1995). Hebbian unlearning. In: Andersson, S.I. (eds) Analysis of Dynamical and Cognitive Systems. Lecture Notes in Computer Science, vol 888. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-58843-4_16
Download citation
DOI: https://doi.org/10.1007/3-540-58843-4_16
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-58843-6
Online ISBN: 978-3-540-49113-2
eBook Packages: Springer Book Archive