# Storing structured sparse memories in a multi-modular cortical network model

- 346 Downloads
- 3 Citations

## Abstract

We study the memory performance of a class of modular attractor neural networks, where modules are potentially fully-connected networks connected to each other via diluted long-range connections. On this anatomical architecture we store memory patterns of activity using a Willshaw-type learning rule. P patterns are split in categories, such that patterns of the same category activate the same set of modules. We first compute the maximal storage capacity of these networks. We then investigate their error-correction properties through an exhaustive exploration of parameter space, and identify regions where the networks behave as an associative memory device. The crucial parameters that control the retrieval abilities of the network are (1) the ratio between the number of synaptic contacts of long- and short-range origins (2) the number of categories in which a module is activated and (3) the amount of local inhibition. We discuss the relationship between our model and networks of cortical patches that have been observed in different cortical areas.

## Keywords

Modular network Attractor network Memory Cortex## Notes

### Compliance with Ethical Standards

### Conflict of interest

The authors declare that they have no conflict of interest.

## References

- Amit, D.J. (1989).
*Modeling brain function*: Cambridge University Press.Google Scholar - Amit, D.J., & Fusi, S. (1994). Dynamic learning in neural networks with material sysnapses.
*Neural Computation*,*6*, 957–982.CrossRefGoogle Scholar - Bosking, W., Zhang, Y., Schofield, B., & Fitzpatrick, D. (1997). Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex.
*J Neurosci.*,*17*, 2112–2127.PubMedGoogle Scholar - Braitenberg, V., & Schutz̈, A. (1991).
*Anatomy of the cortex*: Springer-Verlag.Google Scholar - Brunel, N. (2003). Network models of memory. In
*Methods and Models in Neurophysics, Volume Session LXXX: Lecture Notes of the Les Houches Summer School, pages 407–476*.Google Scholar - Brunel, N. (2005). Network models of memory. In Chow, C., Gutkin, B., Hansel, D., Meunier, C., & Dalibard, J. (Eds.)
*Methods and Models in Neurophysics, Volume Session LXXX: Lecture Notes of the Les Houches Summer School*. Elsevier.Google Scholar - Buzas, P., Eysel, U.T., Adorjan, P., & Kisvarday, Z.F. (2001). Axonal topography of cortical basket cells in relation to orientation, direction, and ocular dominance maps.
*J Comp Neurol*,*437*, 259–285.CrossRefPubMedGoogle Scholar - DeFelipe, J., Conley, M., & Jones, E.G. (1986). Long-range focal collateralization of axons arising from corticocortical cells in monkey sensory-motor cortex.
*J Neurosci.*,*6*, 3749–3766.PubMedGoogle Scholar - Dubreuil, A.M. (2014).
*Memory and cortical connectivity*. Université Paris Descartes: PhD thesis.Google Scholar - Dubreuil, A.M., Amit, Y., & Brunel, N. (2014). Memory capacity of networks with stochastic binary synapses.
*PLoS computational biology*,*e1003727*(8).Google Scholar - Enoki, R., Hu, Y.L., Hamilton, D., & Fine, A. (2009). Expression of long-term plasticity at individual synapses in hippocampus is graded, bidirectional, and mainly presynaptic: optical quantal analysis.
*Neuron*,*62*(2), 242–253.CrossRefPubMedGoogle Scholar - Funahashi, S., Bruce, C.J., & Goldman-Rakic, P.S. (1989). Mnemonic coding of visual space in the monkey’s dorsolateral prefrontal cortex.
*J. Neurophysiol.*,*61*, 331–349.PubMedGoogle Scholar - Fuster, J.M. (1995).
*Memory in the cerebral cortex*: MIT Press.Google Scholar - Fuster, J.M., & Alexander, G. (1971). Neuron activity related to short-term memory.
*Science*,*173*, 652–654.CrossRefPubMedGoogle Scholar - Gilbert, C.D., & Wiesel, T. (1989). Columnar specificity of intrinsic horizontal and corticocortical connections in cat visual cortex.
*J Neurosci.*,*9*, 2432–2442.PubMedGoogle Scholar - Hellwig, B. (2000). A quantitative analysis of the local connectivity between pyramidal neurons in layers 2/3 of the rat visual cortex.
*Biological cybernetics*,*82*(2), 111–121.CrossRefPubMedGoogle Scholar - Holmgren, C., Harkany, T., Svennenfors, B., & Zilberter, Y. (2003). Pyramidal cell communication within local networks in layer 2/3 of rat neocortex.
*J. Physiol.*,*551*, 139–153.CrossRefPubMedPubMedCentralGoogle Scholar - Hopfield, J.J. (1982). Neural networks and physical systems with emergent collective computational abilities.
*Proc. Natl. Acad. Sci. U.S.A.*,*79*, 2554–2558.CrossRefPubMedPubMedCentralGoogle Scholar - Huth, A.G., Nishimoto, S., Vu, A.T., & Gallant, J.L. (2012). A continuous semantic space describes the representation of thousands of object and action categories across the human brain.
*Neuron*,*76*(6), 1210–1224.CrossRefPubMedPubMedCentralGoogle Scholar - Johansson, C., & Lansner, A. (2007). Imposing biological constraints onto an abstract neocortical attractor network model.
*Neural Comput.*,*19*(7), 1871–1896.CrossRefPubMedGoogle Scholar - Kalisman, N., Silberberg, G., & Markram, H. (2005). The neocortical microcircuit as a tabula rasa.
*Proc Natl Acad Sci U S A*,*102*(3), 880–885.CrossRefPubMedPubMedCentralGoogle Scholar - Knoblauch, A., Palm, G., & Sommer, F.T. (2010). Memory capacities for synaptic and structural plasticity.
*Neural Computation*,*22*(2), 289–341.CrossRefPubMedGoogle Scholar - Kropff, E., & Treves, A. (2005). The storage capacity of Potts models for semantic memory retrieval.
*J. Stat. Mech.*,*8*, P08010.Google Scholar - Loewenstein, Y., Kuras, A., & Rumpel, S. (2011). Multiplicative dynamics underlie the emergence of the log-normal distribution of spine sizes in the neocortex in vivo.
*J. Neurosci.*,*31*(26), 9481–9488.CrossRefPubMedGoogle Scholar - Mari, C.F. (2004). Extremely dilute modular neuronal networks: Neocortical memory retrieval dynamics.
*Journal of Computational Neuroscience*,*17*, 57–79.CrossRefPubMedGoogle Scholar - Mari, C.F., & Treves, A. (1998). Modeling neocortical areas with a modular neural network.
*Biosystems*,*48*(1), 47–55.CrossRefGoogle Scholar - Markram, H., Lubke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs.
*Science*,*275*, 213–215.CrossRefPubMedGoogle Scholar - Meli, C., & Lansner, A. (2013). A modular attractor associative memory with patchy connectivity and weight pruning.
*Network*,*24*, 129–150.PubMedGoogle Scholar - Miller, E.K., Erickson, C.A., & Desimone, R. (1996). Neural mechanisms of visual working memory in prefrontal cortex of the macaque.
*J. Neurosci.*,*16*, 5154–5167.PubMedGoogle Scholar - Miyashita, Y. (1988). Neuronal correlate of visual associative long-term memory in the primate temporal cortex.
*Nature*,*335*, 817–820.CrossRefPubMedGoogle Scholar - Montgomery, J.M., & Madison, D.V. (2004). Discrete synaptic states define a major mechanism of synapse plasticity.
*Trends Neurosci.*,*27*(12), 744–750.CrossRefPubMedGoogle Scholar - Nadal, J.-P. (1991). Associative memory: on the (puzzling) sparse coding limit.
*J. Phys. A: Math. Gen.*,*24*, 1093–1101.CrossRefGoogle Scholar - O’Connor, D.H., Wittenberg, G.M., & Wang, S. S.-H. (2005). Graded bidirectional synaptic plasticity is composed of switch-like unitary events.
*Proc Natl Acad Sci U S A*,*102*, 9679– 9684.CrossRefPubMedPubMedCentralGoogle Scholar - O’Kane, D. & Treves, A. (1992). Short-and long-range connections in autoassociative memory.
*Journal of Physics A: Mathematical and General*,*25*, 5055.CrossRefGoogle Scholar - Perin, R., Berger, T.K., & Markram, H. (2011). A synaptic organizing principle for cortical neuronal groups.
*Proc. Natl. Acad. Sci. U.S.A.*,*108*, 5419–5424.CrossRefPubMedPubMedCentralGoogle Scholar - Petersen, C.C., Malenka, R.C., Nicoll, R.A., & Hopfield, J.J. (1998). All-or-none potentiation at CA3-CA1 synapses.
*Proc.Natl.Acad.Sci.USA*,*95*, 4732–4737.CrossRefPubMedPubMedCentralGoogle Scholar - Pucak, M.L., Levitt, J.B., Lund, J.S., & Lewis, D.A. (1996). Patterns of intrinsic and associational circuitry in monkey prefrontal cortex.
*J. Comp. Neurol.*,*338*, 360–376.Google Scholar - Romo, R., Brody, C.D., Hernández, A., & Lemus, L. (1999). Neuronal correlates of parametric working memory in the prefrontal cortex.
*Nature*,*399*, 470–474.CrossRefPubMedGoogle Scholar - Roudi, Y., & Treves, A. (2004). An associative network with spatially organized connectivity.
*Journal of Statistical Mechanics: Theory and Experiment*,*2004*(07), P07010.CrossRefGoogle Scholar - Roudi, Y., & Treves, A. (2006). Localized activity profiles and storage capacity of rate-based autoassociative networks.
*Physical Review E, 73(6)*,*061904*.Google Scholar - Sjöström, P.J., Turrigiano, G.G., & Nelson, S. (2001). Rate, timing, and cooperativity jointly determine cortical synaptic plasticity.
*Neuron*,*32*, 1149–1164.CrossRefPubMedGoogle Scholar - Stepanyants, A., Martinez, L.M., Ferecsko, A.S., & Kisvarday, Z.F. (2009). The fractions of short-and long-range connections in the visual cortex.
*Proceedings of the National Academy of Sciences*,*106*(9), 3555–3560.CrossRefGoogle Scholar - Tsao, D.Y., Freiwald, W.A., Knutsen, T.A., Mandeville, J.B., & Tootell, R.B.H. (2003). Faces and objects in macaque cerebral cortex.
*Nature Neuroscience*,*6*(9), 989–995.CrossRefPubMedGoogle Scholar - van Vreeswijk, C., & Sompolinsky, H. (2003). Irregular activity in large networks of neurons. In
*Methods and Models in Neurophysics, Volume Session LXXX: Lecture Notes of the Les Houches Summer School, pages 341–402*.Google Scholar - Wang, X.-J. (2001). Synaptic reverberation underlying mnemonic persistent activity.
*Trends Neurosci.*,*24*, 455–463.CrossRefPubMedGoogle Scholar - Willshaw, D., Buneman, O.P., & Longuet-Higgins, H. (1969). Non-holographic associative memory.
*Nature*,*222*, 960–962.CrossRefPubMedGoogle Scholar