# Short term memory properties of sensory neural architectures

- 10 Downloads

## Abstract

A functional role of the cerebral cortex is to form and hold representations of the sensory world for behavioral purposes. This is achieved by a sheet of neurons, organized in modules called cortical columns, that receives inputs in a peculiar manner, with only a few neurons driven by sensory inputs through thalamic projections, and a vast majority of neurons receiving mainly cortical inputs. How should cortical modules be organized, with respect to sensory inputs, in order for the cortex to efficiently hold sensory representations in memory? To address this question we investigate the memory performance of trees of recurrent networks (TRN) that are composed of recurrent networks, modeling cortical columns, connected with each others through a tree-shaped feed-forward backbone of connections, with sensory stimuli injected at the root of the tree. On these sensory architectures two types of short-term memory (STM) mechanisms can be implemented, STM via transient dynamics on the feed-forward tree, and STM via reverberating activity on the recurrent connectivity inside modules. We derive equations describing the dynamics of such networks, which allow us to thoroughly explore the space of possible architectures and quantify their memory performance. By varying the divergence ratio of the tree, we show that serial architectures, where sensory inputs are successively processed in different modules, are better suited to implement STM via transient dynamics, while parallel architectures, where sensory inputs are simultaneously processed by all modules, are better suited to implement STM via reverberating dynamics.

## Keywords

Modular network Short-term memory Cortical hierarchy## Notes

### Acknowledgements

I would like to thank Nicolas Brunel and Haim Sompolinsky for useful discussions throughout the course of this work. I also would like to thank Gaetan Bouchet for his help with the figures.

### Compliance with Ethical Standards

### **Conflict of interests**

The authors declare that they have no conflict of interest.

## Supplementary material

## References

- Amit, D.J. (1989).
*Modeling brain function*. Cambridge University Press.Google Scholar - Amit, D.J., & Brunel, N. (1997). Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex.
*Cerebral Cortex*,*7*, 237–252.CrossRefGoogle Scholar - Bosking, W., Zhang, Y., Schofield, B., Fitzpatrick, D. (1997). Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex.
*Journal of Neuroscience*,*17*, 2112–2127.CrossRefGoogle Scholar - Braitenberg, V., & Schütz, A. (1991).
*Anatomy of the cortex*. Springer.Google Scholar - Brunel, N. (2016). Is cortical connectivity optimized for storing information [quest]?
*Nature Neuroscience*,*19*(5), 749–755.CrossRefGoogle Scholar - Coolen, A., & Viana, L. (1996). Feed-forward chains of recurrent attractor neural networks near saturation.
*Journal of Physics A: Mathematical and General*,*29*(24), 7855.CrossRefGoogle Scholar - DeFelipe, J., Conley, M., Jones, E.G. (1986). Long-range focal collateralization of axons arising from corticocortical cells in monkey sensory-motor cortex.
*Journal of Neuroscience*,*6*, 3749–3766.CrossRefGoogle Scholar - Dubreuil, A.M., & Brunel, N. (2016). Storing structured sparse memories in a multi-modular cortical network model.
*Journal of Computational Neuroscience*,*40*(2), 157–175.CrossRefGoogle Scholar - Evans, M. (1989). Random dilution in a neural network for biased patterns.
*Journal of Physics A: Mathematical and General*,*22*(12), 2103.CrossRefGoogle Scholar - Fuster, J.M. (1995).
*Memory in the cerebral cortex*. MIT Press.Google Scholar - Ganguli, S, Huh, D, Sompolinsky, H. (2008). Memory traces in dynamical systems.
*Proceedings of the National Academy of Sciences of the United States of America*,*105*, 18970–18975.CrossRefGoogle Scholar - Gilbert, C.D., & Wiesel, T. (1989). Columnar specificity of intrinsic horizontal and corticocortical connections in cat visual cortex.
*Journal of Neuroscience*,*9*, 2432–2442.CrossRefGoogle Scholar - Girard, P., & Bullier, J. (1989). Visual activity in area v2 during reversible inactivation of area 17 in the macaque monkey.
*Journal of neurophysiology*,*62*(6), 1287–1302.CrossRefGoogle Scholar - Hopfield, J.J. (1982). Neural networks and physical systems with emergent collective computational abilities.
*Proceedings of the National Academy of Sciences of the United States of America*,*79*, 2554–2558.CrossRefGoogle Scholar - Jaeger, H., & Haas, H. (2004). Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication.
*Science*,*304*(5667), 78–80.CrossRefGoogle Scholar - Johansson, C., & Lansner, A. (2007). Imposing biological constraints onto an abstract neocortical attractor network model.
*Neural Computation*,*19*(7), 1871–1896. https://doi.org/10.1162/neco.2007.19.7.1871 https://doi.org/10.1162/neco.2007.19.7.1871.CrossRefGoogle Scholar - Klampfl, S., David, S.V., Yin, P., Shamma, S.A., Maass, W. (2012). A quantitative analysis of information about past and present stimuli encoded by spikes of a1 neurons.
*Journal of Neurophysiology*,*108*(5), 1366–1380.CrossRefGoogle Scholar - Lim, S., & Goldman, M.S. (2012). Noise tolerance of attractor and feedforward memory models.
*Neural Computation*,*24*(2), 332–390.CrossRefGoogle Scholar - Maass, W., Natschläger, T, Markram, H. (2002). Real-time computing without stable states: a new framework for neural computation based on perturbations.
*Neural Computation*,*14*(11), 2531–2560.CrossRefGoogle Scholar - Mari, C.F., & Treves, A. (1998). Modeling neocortical areas with a modular neural network.
*Bio Systems*,*48*(1), 47–55.CrossRefGoogle Scholar - Markov, N., Misery, P., Falchier, A., Lamy, C., Vezoli, J., Quilodran, R., Gariel, M., Giroud, P., Ercsey-Ravasz, M., Pilaz, L., et al. (2010). Weight consistency specifies regularities of macaque cortical networks.
*Cerebral Cortex*,*21*(6), 1254–1272.CrossRefGoogle Scholar - Nikolić, D, Häusler, S, Singer, W., Maass, W. (2009). Distributed fading memory for stimulus properties in the primary visual cortex.
*PLoS Biology*,*7*(12), e1000260.CrossRefGoogle Scholar - O’Kane, D, & Treves, A. (1992). Short-and long-range connections in autoassociative memory.
*Journal of Physics A: Mathematical and General*,*25*, 5055.CrossRefGoogle Scholar - Pucak, M.L., Levitt, J.B., Lund, J.S., Lewis, D.A. (1996). Patterns of intrinsic and associational circuitry in monkey prefrontal cortex.
*Journal of Comparative Neurology*,*338*, 360–376.Google Scholar - Sejnowski, T.J. (1977). Storing covariance with nonlinearly interacting neurons.
*Journal of Mathematical Biology*,*4*, 303–.Google Scholar - Sompolinsky, H., & White, O. (2003). Theory of large recurrent networks: from spikes to behavior. In
*Methods and models in neurophysics, volume session LXXX: lecture notes of the Les Houches Summer School*(p Chapter 8).Google Scholar - Thorpe, S.J., & Fabre-Thorpe, M. (2001). Seeking categories in the brain.
*Science*,*291*(5502), 260–263.CrossRefGoogle Scholar - Tkačik, G, Prentice, J.S., Balasubramanian, V., Schneidman, E. (2010). Optimal population coding by noisy spiking neurons.
*Proceedings of the National Academy of Sciences*,*107*(32), 14419–14424.CrossRefGoogle Scholar - Tsodyks, M., & Feigel’man, M.V. (1988). The enhanced storage capacity in neural networks with low activity level.
*Europhysics Letters*,*6*, 101–105.CrossRefGoogle Scholar - White, O.L., Lee, D.D., Sompolinsky, H. (2004). Short-term memory in orthogonal neural networks.
*Physical Review Letters*,*92*(14), 148102.CrossRefGoogle Scholar