Improved Retrieval for Challenging Scenarios in Clique-Based Neural Networks
This paper describes new retrieval algorithms based on heuristic approach in clique-based neural networks introduced by Gripon and Berrou. This associative memory model resembles the well-known Willshaw model with specificity of clustered structure. Several retrieval algorithms exist, for instance, Winners-Take-All and Losers-Kicked-Out. These methods work generally well when the input message suffers reasonable distortions, but the performance drops dramatically in some challenging scenarios because of severe interference. By means of simulations, we show that the proposed heuristic retrieval algorithms are able to significantly mitigate this issue while maintaining biological plausibility to some extent.
KeywordsAssociative memory Retrieval algorithms Sparse coding Recurrent neural networks
Unable to display preview. Download preview PDF.
- 3.Willshaw, D.J.: Models of distributed associative memory, ph.D dissertation, Edinburgh University (1971)Google Scholar
- 6.Gripon, V., Berrou, C.: Sparse neural networks with large learning diversity. IEEE Transactions on Neural Networks 22(7), July 2011Google Scholar
- 7.Aliabadi, B.K., Berrou, C., Gripon, V., Jiang, X.: Storing sparse messages in networks of neural cliques. IEEE Transactions on Neural Networks and Learning Systems, May 2014Google Scholar
- 8.Jiang, X., Gripon, V., Berrou, C., Rabbat, M.: Storing sequences in binary tournament-based neural networks. arXiv: 1409.0334 IEEE Transactions on Neural Networks and Learning Systems (under revision)
- 9.Jiang, X., Gripon, V., Berrou, C.: Learning long sequences in binary neural networks. In: International Conference on Advanced Cognitive Technologies and Applications, Nice, France, pp. 165–170 (2012)Google Scholar
- 10.Aboudib, A., Gripon, V., Jiang, X.: A study of retrieval algorithms of sparse messages in networks of neural cliques. In: International Conference on Advanced Cognitive Technologies and Applications, Venice, Italy, pp. 142–146 (2014)Google Scholar
- 13.Palm, G., Schwenker, F., Sommer, F.T., Strey, A.: Neural associative memories. Biological Cybernetics 36 (1993)Google Scholar
- 14.Gripon, V., Berrou, C.: Nearly-optimal associative memories based on distributed constant weight codes. In: Information Theory and Applications Workshop (ITA), February (2012)Google Scholar
- 15.Gripon, V., Jiang, X.: Mémoires associatives pour observations floues. In: Proceedings of XXIV-th Gretsi seminar, September (2013)Google Scholar
- 16.Boguslawski, B., Gripon, V., Seguin, F., Heitzmann, F.: Twin Neurons for Efficient Real-World Data Distribution in Networks of Neural Cliques. Applications in Power Management in Electronic circuits. IEEE Transactions on Neural Networks and Learning Systems, submitted toGoogle Scholar