Advertisement

Improved Retrieval for Challenging Scenarios in Clique-Based Neural Networks

  • Xiaoran JiangEmail author
  • Max Raphael Sobroza Marques
  • Pierre-Julien Kirsch
  • Claude Berrou
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9094)

Abstract

This paper describes new retrieval algorithms based on heuristic approach in clique-based neural networks introduced by Gripon and Berrou. This associative memory model resembles the well-known Willshaw model with specificity of clustered structure. Several retrieval algorithms exist, for instance, Winners-Take-All and Losers-Kicked-Out. These methods work generally well when the input message suffers reasonable distortions, but the performance drops dramatically in some challenging scenarios because of severe interference. By means of simulations, we show that the proposed heuristic retrieval algorithms are able to significantly mitigate this issue while maintaining biological plausibility to some extent.

Keywords

Associative memory Retrieval algorithms Sparse coding Recurrent neural networks 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Hopfield, J.: Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America 79(8), 2554 (1982)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Hopfield, J.: Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the National Academy of Sciences 81(10), 3088–3092 (1984)CrossRefGoogle Scholar
  3. 3.
    Willshaw, D.J.: Models of distributed associative memory, ph.D dissertation, Edinburgh University (1971)Google Scholar
  4. 4.
    Sommer, F.T., Palm, G.: Improved bidirectional retrieval of sparse patterns stored by Hebbian learning. Neural Networks 12, 281–297 (1999)CrossRefGoogle Scholar
  5. 5.
    Palm, G.: Neural associative memories and sparse coding. Neural Networks 37, 165–171 (2013)CrossRefGoogle Scholar
  6. 6.
    Gripon, V., Berrou, C.: Sparse neural networks with large learning diversity. IEEE Transactions on Neural Networks 22(7), July 2011Google Scholar
  7. 7.
    Aliabadi, B.K., Berrou, C., Gripon, V., Jiang, X.: Storing sparse messages in networks of neural cliques. IEEE Transactions on Neural Networks and Learning Systems, May 2014Google Scholar
  8. 8.
    Jiang, X., Gripon, V., Berrou, C., Rabbat, M.: Storing sequences in binary tournament-based neural networks. arXiv: 1409.0334 IEEE Transactions on Neural Networks and Learning Systems (under revision)
  9. 9.
    Jiang, X., Gripon, V., Berrou, C.: Learning long sequences in binary neural networks. In: International Conference on Advanced Cognitive Technologies and Applications, Nice, France, pp. 165–170 (2012)Google Scholar
  10. 10.
    Aboudib, A., Gripon, V., Jiang, X.: A study of retrieval algorithms of sparse messages in networks of neural cliques. In: International Conference on Advanced Cognitive Technologies and Applications, Venice, Italy, pp. 142–146 (2014)Google Scholar
  11. 11.
    Ackley, D., Hinton, G.E.: A learning algorithm for Boltzmann machines. Cognitive Science 9(1), 147–169 (1985)CrossRefGoogle Scholar
  12. 12.
    McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics 5, 115–133 (1943)zbMATHMathSciNetCrossRefGoogle Scholar
  13. 13.
    Palm, G., Schwenker, F., Sommer, F.T., Strey, A.: Neural associative memories. Biological Cybernetics 36 (1993)Google Scholar
  14. 14.
    Gripon, V., Berrou, C.: Nearly-optimal associative memories based on distributed constant weight codes. In: Information Theory and Applications Workshop (ITA), February (2012)Google Scholar
  15. 15.
    Gripon, V., Jiang, X.: Mémoires associatives pour observations floues. In: Proceedings of XXIV-th Gretsi seminar, September (2013)Google Scholar
  16. 16.
    Boguslawski, B., Gripon, V., Seguin, F., Heitzmann, F.: Twin Neurons for Efficient Real-World Data Distribution in Networks of Neural Cliques. Applications in Power Management in Electronic circuits. IEEE Transactions on Neural Networks and Learning Systems, submitted toGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Xiaoran Jiang
    • 1
    Email author
  • Max Raphael Sobroza Marques
    • 1
  • Pierre-Julien Kirsch
    • 1
  • Claude Berrou
    • 1
  1. 1.Télécom Bretagne, Electronics department, UMR CNRS LabsticcTechnopôle Brest Iroise-CS 83818Brest CedexFrance

Personalised recommendations