Journal of Computational Neuroscience

, Volume 40, Issue 2, pp 157–175 | Cite as

Storing structured sparse memories in a multi-modular cortical network model

  • Alexis M. Dubreuil
  • Nicolas Brunel


We study the memory performance of a class of modular attractor neural networks, where modules are potentially fully-connected networks connected to each other via diluted long-range connections. On this anatomical architecture we store memory patterns of activity using a Willshaw-type learning rule. P patterns are split in categories, such that patterns of the same category activate the same set of modules. We first compute the maximal storage capacity of these networks. We then investigate their error-correction properties through an exhaustive exploration of parameter space, and identify regions where the networks behave as an associative memory device. The crucial parameters that control the retrieval abilities of the network are (1) the ratio between the number of synaptic contacts of long- and short-range origins (2) the number of categories in which a module is activated and (3) the amount of local inhibition. We discuss the relationship between our model and networks of cortical patches that have been observed in different cortical areas.


Modular network Attractor network Memory Cortex 


Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Amit, D.J. (1989). Modeling brain function: Cambridge University Press.Google Scholar
  2. Amit, D.J., & Fusi, S. (1994). Dynamic learning in neural networks with material sysnapses. Neural Computation, 6, 957–982.CrossRefGoogle Scholar
  3. Bosking, W., Zhang, Y., Schofield, B., & Fitzpatrick, D. (1997). Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. J Neurosci., 17, 2112–2127.PubMedGoogle Scholar
  4. Braitenberg, V., & Schutz̈, A. (1991). Anatomy of the cortex: Springer-Verlag.Google Scholar
  5. Brunel, N. (2003). Network models of memory. In Methods and Models in Neurophysics, Volume Session LXXX: Lecture Notes of the Les Houches Summer School, pages 407–476.Google Scholar
  6. Brunel, N. (2005). Network models of memory. In Chow, C., Gutkin, B., Hansel, D., Meunier, C., & Dalibard, J. (Eds.) Methods and Models in Neurophysics, Volume Session LXXX: Lecture Notes of the Les Houches Summer School. Elsevier.Google Scholar
  7. Buzas, P., Eysel, U.T., Adorjan, P., & Kisvarday, Z.F. (2001). Axonal topography of cortical basket cells in relation to orientation, direction, and ocular dominance maps. J Comp Neurol, 437, 259–285.CrossRefPubMedGoogle Scholar
  8. DeFelipe, J., Conley, M., & Jones, E.G. (1986). Long-range focal collateralization of axons arising from corticocortical cells in monkey sensory-motor cortex. J Neurosci., 6, 3749–3766.PubMedGoogle Scholar
  9. Dubreuil, A.M. (2014). Memory and cortical connectivity. Université Paris Descartes: PhD thesis.Google Scholar
  10. Dubreuil, A.M., Amit, Y., & Brunel, N. (2014). Memory capacity of networks with stochastic binary synapses. PLoS computational biology, e1003727(8).Google Scholar
  11. Enoki, R., Hu, Y.L., Hamilton, D., & Fine, A. (2009). Expression of long-term plasticity at individual synapses in hippocampus is graded, bidirectional, and mainly presynaptic: optical quantal analysis. Neuron, 62(2), 242–253.CrossRefPubMedGoogle Scholar
  12. Funahashi, S., Bruce, C.J., & Goldman-Rakic, P.S. (1989). Mnemonic coding of visual space in the monkey’s dorsolateral prefrontal cortex. J. Neurophysiol., 61, 331–349.PubMedGoogle Scholar
  13. Fuster, J.M. (1995). Memory in the cerebral cortex: MIT Press.Google Scholar
  14. Fuster, J.M., & Alexander, G. (1971). Neuron activity related to short-term memory. Science, 173, 652–654.CrossRefPubMedGoogle Scholar
  15. Gilbert, C.D., & Wiesel, T. (1989). Columnar specificity of intrinsic horizontal and corticocortical connections in cat visual cortex. J Neurosci., 9, 2432–2442.PubMedGoogle Scholar
  16. Hellwig, B. (2000). A quantitative analysis of the local connectivity between pyramidal neurons in layers 2/3 of the rat visual cortex. Biological cybernetics, 82(2), 111–121.CrossRefPubMedGoogle Scholar
  17. Holmgren, C., Harkany, T., Svennenfors, B., & Zilberter, Y. (2003). Pyramidal cell communication within local networks in layer 2/3 of rat neocortex. J. Physiol., 551, 139–153.CrossRefPubMedPubMedCentralGoogle Scholar
  18. Hopfield, J.J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. U.S.A., 79, 2554–2558.CrossRefPubMedPubMedCentralGoogle Scholar
  19. Huth, A.G., Nishimoto, S., Vu, A.T., & Gallant, J.L. (2012). A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron, 76(6), 1210–1224.CrossRefPubMedPubMedCentralGoogle Scholar
  20. Johansson, C., & Lansner, A. (2007). Imposing biological constraints onto an abstract neocortical attractor network model. Neural Comput., 19(7), 1871–1896.CrossRefPubMedGoogle Scholar
  21. Kalisman, N., Silberberg, G., & Markram, H. (2005). The neocortical microcircuit as a tabula rasa. Proc Natl Acad Sci U S A, 102(3), 880–885.CrossRefPubMedPubMedCentralGoogle Scholar
  22. Knoblauch, A., Palm, G., & Sommer, F.T. (2010). Memory capacities for synaptic and structural plasticity. Neural Computation, 22(2), 289–341.CrossRefPubMedGoogle Scholar
  23. Kropff, E., & Treves, A. (2005). The storage capacity of Potts models for semantic memory retrieval. J. Stat. Mech., 8, P08010.Google Scholar
  24. Loewenstein, Y., Kuras, A., & Rumpel, S. (2011). Multiplicative dynamics underlie the emergence of the log-normal distribution of spine sizes in the neocortex in vivo. J. Neurosci., 31(26), 9481–9488.CrossRefPubMedGoogle Scholar
  25. Mari, C.F. (2004). Extremely dilute modular neuronal networks: Neocortical memory retrieval dynamics. Journal of Computational Neuroscience, 17, 57–79.CrossRefPubMedGoogle Scholar
  26. Mari, C.F., & Treves, A. (1998). Modeling neocortical areas with a modular neural network. Biosystems, 48 (1), 47–55.CrossRefGoogle Scholar
  27. Markram, H., Lubke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, 275, 213–215.CrossRefPubMedGoogle Scholar
  28. Meli, C., & Lansner, A. (2013). A modular attractor associative memory with patchy connectivity and weight pruning. Network, 24, 129–150.PubMedGoogle Scholar
  29. Miller, E.K., Erickson, C.A., & Desimone, R. (1996). Neural mechanisms of visual working memory in prefrontal cortex of the macaque. J. Neurosci., 16, 5154–5167.PubMedGoogle Scholar
  30. Miyashita, Y. (1988). Neuronal correlate of visual associative long-term memory in the primate temporal cortex. Nature, 335, 817–820.CrossRefPubMedGoogle Scholar
  31. Montgomery, J.M., & Madison, D.V. (2004). Discrete synaptic states define a major mechanism of synapse plasticity. Trends Neurosci., 27(12), 744–750.CrossRefPubMedGoogle Scholar
  32. Nadal, J.-P. (1991). Associative memory: on the (puzzling) sparse coding limit. J. Phys. A: Math. Gen., 24, 1093–1101.CrossRefGoogle Scholar
  33. O’Connor, D.H., Wittenberg, G.M., & Wang, S. S.-H. (2005). Graded bidirectional synaptic plasticity is composed of switch-like unitary events. Proc Natl Acad Sci U S A, 102, 9679– 9684.CrossRefPubMedPubMedCentralGoogle Scholar
  34. O’Kane, D. & Treves, A. (1992). Short-and long-range connections in autoassociative memory. Journal of Physics A: Mathematical and General, 25, 5055.CrossRefGoogle Scholar
  35. Perin, R., Berger, T.K., & Markram, H. (2011). A synaptic organizing principle for cortical neuronal groups. Proc. Natl. Acad. Sci. U.S.A., 108, 5419–5424.CrossRefPubMedPubMedCentralGoogle Scholar
  36. Petersen, C.C., Malenka, R.C., Nicoll, R.A., & Hopfield, J.J. (1998). All-or-none potentiation at CA3-CA1 synapses. Proc.Natl.Acad.Sci.USA, 95, 4732–4737.CrossRefPubMedPubMedCentralGoogle Scholar
  37. Pucak, M.L., Levitt, J.B., Lund, J.S., & Lewis, D.A. (1996). Patterns of intrinsic and associational circuitry in monkey prefrontal cortex. J. Comp. Neurol., 338, 360–376.Google Scholar
  38. Romo, R., Brody, C.D., Hernández, A., & Lemus, L. (1999). Neuronal correlates of parametric working memory in the prefrontal cortex. Nature, 399, 470–474.CrossRefPubMedGoogle Scholar
  39. Roudi, Y., & Treves, A. (2004). An associative network with spatially organized connectivity. Journal of Statistical Mechanics: Theory and Experiment, 2004(07), P07010.CrossRefGoogle Scholar
  40. Roudi, Y., & Treves, A. (2006). Localized activity profiles and storage capacity of rate-based autoassociative networks. Physical Review E, 73(6), 061904.Google Scholar
  41. Sjöström, P.J., Turrigiano, G.G., & Nelson, S. (2001). Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron, 32, 1149–1164.CrossRefPubMedGoogle Scholar
  42. Stepanyants, A., Martinez, L.M., Ferecsko, A.S., & Kisvarday, Z.F. (2009). The fractions of short-and long-range connections in the visual cortex. Proceedings of the National Academy of Sciences, 106(9), 3555–3560.CrossRefGoogle Scholar
  43. Tsao, D.Y., Freiwald, W.A., Knutsen, T.A., Mandeville, J.B., & Tootell, R.B.H. (2003). Faces and objects in macaque cerebral cortex. Nature Neuroscience, 6(9), 989–995.CrossRefPubMedGoogle Scholar
  44. van Vreeswijk, C., & Sompolinsky, H. (2003). Irregular activity in large networks of neurons. In Methods and Models in Neurophysics, Volume Session LXXX: Lecture Notes of the Les Houches Summer School, pages 341–402.Google Scholar
  45. Wang, X.-J. (2001). Synaptic reverberation underlying mnemonic persistent activity. Trends Neurosci., 24, 455–463.CrossRefPubMedGoogle Scholar
  46. Willshaw, D., Buneman, O.P., & Longuet-Higgins, H. (1969). Non-holographic associative memory. Nature, 222, 960–962.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Laboratoire de Physique ThéoriqueEcole Normale SupérieureParisFrance
  2. 2.Laboratoire Jean Perrin, UPMCParisFrance
  3. 3.Departments of Statistics and NeurobiologyUniversity of ChicagoChicagoUSA

Personalised recommendations