Skip to main content
SpringerLink
Log in

Neuronal Firing Rate As Code Length: a Hypothesis

  • Published:
Computational Brain & Behavior Aims and scope Submit manuscript

Abstract

Many theories assume that a sensory neuron’s higher firing rate indicates a greater probability of its preferred stimulus. However, this contradicts (1) the adaptation phenomena where prolonged exposure to, and thus increased probability of, a stimulus reduces the firing rates of cells tuned to the stimulus; and (2) the observation that unexpected (low probability) stimuli capture attention and increase neuronal firing. Other theories posit that the brain builds predictive/efficient codes for reconstructing sensory inputs. However, they cannot explain that the brain preserves some information while discarding other. We propose that in sensory areas, projection neurons’ firing rates are proportional to optimal code length (i.e., negative log estimated probability), and their spike patterns are the code, for useful features in inputs. This hypothesis explains adaptation-induced changes of V1 orientation tuning curves and bottom-up attention. We discuss how the modern minimum-description-length (MDL) principle may help understand neural codes. Because regularity extraction is relative to a model class (defined by cells) via its optimal universal code (OUC), MDL matches the brain’s purposeful, hierarchical processing without input reconstruction. Such processing enables input compression/understanding even when model classes do not contain true models. Top-down attention modifies lower-level OUCs via feedback connections to enhance transmission of behaviorally relevant information. Although OUCs concern lossless data compression, we suggest possible extensions to lossy, prefix-free neural codes for prompt, online processing of most important aspects of stimuli while minimizing behaviorally relevant distortion. Finally, we discuss how neural networks might learn MDL’s normalized maximum likelihood (NML) distributions from input data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Adrian, E. D. (1926). The impulses produced by sensory nerve endings. The Journal of Physiology, 61, 49–72.

    PubMed  PubMed Central  Google Scholar 

  • Albrecht, D. G., & Geisler, W. S. (1991). Motion selectivity and the contrast-response function of simple cells in the visual cortex. Visual Neuroscience, 7, 531–546.

    PubMed  Google Scholar 

  • Allman, J., Miezin, F., & McGuinness, E. (1985). Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. Annual Review of Neuroscience, 8, 407–430.

    PubMed  Google Scholar 

  • Anderson, J. S., Carandini, M., & Ferster, D. (2000). Orientation tuning of input conductance, excitation, and inhibition in cat primary visual cortex. Journal of Neurophysiology, 84, 909–926.

    PubMed  Google Scholar 

  • Atick, J. J., & Redlich, A. N. (1990). Towards a theory of early visual processing. Neural Computation, 2, 308–320.

    Google Scholar 

  • Balasubramanian, V., Berry, M. J., & Kimber, D. (2001). Metabolically efficient information processing. Neural Computation, 13, 799–816.

    PubMed  Google Scholar 

  • Barlow, H. (1972). Single units and sensation: a neuron doctrine for perceptual psychology? Perception, 1(4), 371–394.

    PubMed  Google Scholar 

  • Barlow, H., & Foldiak, P. (1989). Adaptation and decorrelation in the cortex. In R. Durbin, C. Miall, & G. Mitchinson (Eds.), The computing neuron (pp. 54–72). New York: Addison-Wesley.

    Google Scholar 

  • Barron, A., Rissanen, J., & Yu, B. (1998). The minimum description length principle in coding and modeling. IEEE Transactions on Information Theory, 44, 2743–2760.

    Google Scholar 

  • Bell, A. J., & Sejnowski, T. J. (1997). The “independent components” of natural scenes are edge filters. Vision Research, 37, 3327–3338.

    PubMed  PubMed Central  Google Scholar 

  • Blahut, R. E. (1972). Computation of channel capacity and rate-distortion functions. IEEE Transactions on Information Theory, 18, 460–473.

    Google Scholar 

  • Blakemore, C., & Campbell, F. W. (1969). Adaptation to spatial stimuli. The Journal of Physiology, 200, 11P–13P.

    PubMed  Google Scholar 

  • Bolz, J., & Gilbert, C. D. (1986). Generation of end-inhibition in the visual cortex via interlaminar connections. Nature, 320, 362–365.

    PubMed  Google Scholar 

  • Carandini, M., & Heeger, D. J. (2012). Normalization as a canonical neural computation. Nature Reviews. Neuroscience, 13, 51–62.

    Google Scholar 

  • Carandini, M., Heeger, D. J., & Movshon, J. A. (1997). Linearity and normalization in simple cells of the macaque primary visual cortex. The Journal of Neuroscience, 17, 8621–8644.

    PubMed  PubMed Central  Google Scholar 

  • Chen, L. (1982). Topological structure in visual perception. Science, 218, 699–700.

    PubMed  Google Scholar 

  • Dayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995). The Helmholtz machine. Neural Computation, 7, 889–904.

    PubMed  Google Scholar 

  • Ding, S., Cueva, C. J., Tsodyks, M., & Qian, N. (2017). Visual perception as retrospective Bayesian decoding from high- to low-level features. Proceedings of the National Academy of Sciences, 114, E9115–E9124.

    Google Scholar 

  • Dragoi, V., Sharma, J., & Sur, M. (2000). Adaptation-induced plasticity of orientation tuning in adult visual cortex. Neuron, 28, 287–298.

    PubMed  Google Scholar 

  • Dragoi, V., Rivadulla, C., & Sur, M. (2001). Foci of orientation plasticity in visual cortex. Nature, 411, 80–86.

    PubMed  Google Scholar 

  • Fang, F., Murray, S. O., Kersten, D., & He, S. (2005). Orientation-tuned fMRI adaptation in human visual cortex. Journal of Neurophysiology, 94, 4188–4195.

    PubMed  Google Scholar 

  • Felsen, G., Shen, Y. S., Yao, H., Spor, G., Li, C., & Dan, Y. (2002). Dynamic modification of cortical orientation tuning mediated by recurrent connections. Neuron, 36, 945–954.

    PubMed  Google Scholar 

  • Field, D. J. (1987). Relations between the statistics of natural images and the response properties of cortical-cells. Journal of the Optical Society of America. A, Optics, Image Science, and Vision, 4, 2379–2394.

    Google Scholar 

  • Flash, T., & Hogan, N. (1985). The coordination of arm movements - an experimentally confirmed mathematical-model. The Journal of Neuroscience, 5, 1688–1703.

    PubMed  PubMed Central  Google Scholar 

  • Friston (2010). The free-energy principle: a unified brain theory? Nature Reviews Neuroscience, 11, 127–138.

    PubMed  Google Scholar 

  • Gallant, J. L., Connor, C. E., & Van Essen, D. C. (1998). Neural activity in areas V1, V2 and V4 during free viewing of natural scenes compared to controlled viewing. Neuroreport, 9, 2153–2158.

    PubMed  Google Scholar 

  • Ganmor, E., Segev, R., & Schneidman, E. (2011). The architecture of functional interaction networks in the retina. The Journal of Neuroscience, 31, 3044–3054.

    PubMed  PubMed Central  Google Scholar 

  • Geisler, W. S., Perry, J. S., Super, B. J., & Gallogly, D. P. (2001). Edge co-occurrence in natural images predicts contour grouping performance. Vision Research, 41, 711–724.

    PubMed  Google Scholar 

  • Georgopoulos, A. P., Schwartz, A. B., & Kettner, R. E. (1986). Neuronal population coding of movement direction. Science, 233, 1416–1419.

    PubMed  Google Scholar 

  • Gibson, J. J., & Radner, M. (1937). Adaptation, after-effect and contrast in the perception of tilted lines. I. Quantitative studies. Journal of Experimental Psychology, 20, 453–467.

    Google Scholar 

  • Gottlieb, J. P., Kusunoki, M., & Goldberg, M. E. (1998). The representation of visual salience in monkey parietal cortex. Nature, 391, 481–484.

    PubMed  Google Scholar 

  • Grunwald, P. D. (2007). The minimum description length principle. Cambridge: MIT Press.

    Google Scholar 

  • Grunwald, P. D., Myung, I. J., & Pitt, M. A. (2005). Advances in minimum description length: theory and applications. Cambridge: MIT Press.

    Google Scholar 

  • Han, S., & Vasconcelos, N. (2010). Biologically plausible saliency mechanisms improve feedforward object recognition. Vision Research, 50, 2295–2307.

    PubMed  Google Scholar 

  • Harpur, G. F., & Prager, R. W. (1996). Development of low entropy coding in a recurrent network*. Network: Computation in Neural Systems, 7, 277–284.

    Google Scholar 

  • Heeger, D. J. (1992). Normalization of cell responses in cat striate cortex. Visual Neuroscience, 9, 181–197.

    PubMed  Google Scholar 

  • Hinton, G., Dayan, P., Frey, B., & Neal, R. (1995). The “wake-sleep” algorithm for unsupervised neural networks. Science, 268, 1158–1161.

    PubMed  Google Scholar 

  • Hubel, D. H., & Wiesel, T. N. (1968). Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology, 195, 215–243.

    PubMed  PubMed Central  Google Scholar 

  • Itti, L., & Koch, C. (2001). Computational modelling of visual attention. Nature Reviews. Neuroscience, 2, 194–203.

    PubMed  Google Scholar 

  • Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014). Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Computational Biology, 10, e1003915.

    PubMed  PubMed Central  Google Scholar 

  • Kingma DP, Welling M. (2013). Auto-encodrwing variational bayes. arXiv preprint arXiv:1312.6114.

  • Levitt, J. B., & Lund, J. S. (1997). Contrast dependence of contextual effects in primate visual cortex. Nature, 387, 73–76.

    PubMed  Google Scholar 

  • Li, W., & Gilbert, C. D. (2002). Global contour saliency and local colinear interactions. Journal of Neurophysiology, 88, 2846–2856.

    PubMed  Google Scholar 

  • Li, C.-Y., & Li, W. (1994). Extenstive integration field beyond the classical receptive field of cat’s striate cortical neurons - classification and tuning properties. Vision Research, 34, 2337–2355.

    PubMed  Google Scholar 

  • Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006). Bayesian inference with probabilistic population codes. Nature Neuroscience, 9, 1432–1438.

    PubMed  Google Scholar 

  • Meng, X., & Qian, N. (2005). The oblique effect depends on perceived, rather than physical, orientation and direction. Vision Research, 45, 3402–3413.

    PubMed  Google Scholar 

  • Motoyoshi, I., Sy, N., Sharan, L., & Adelson, E. H. (2007). Image statistics and the perception of surface qualities. Nature, 447, 206–209.

    PubMed  Google Scholar 

  • Myung, J. I., Navarro, D. J., & Pitt, M. A. (2006). Model selection by normalized maximum likelihood. Journal of Mathematical Psychology, 50, 167–179.

    Google Scholar 

  • Navon, D. (1977). Forest before trees: the precedence of global features in visual perception. Cognitive Psychology, 9, 353–383.

    Google Scholar 

  • Nelson, J. I., & Frost, B. J. (1978). Orientation-selective inhibition from beyond the classic visual receptive field. Brain Research, 139, 359–365.

    PubMed  Google Scholar 

  • Nowak, L. G., Munk, M. H., Nelson, J. I., James, A. C., & Bullier, J. (1995). Structural basis of cortical synchronization. I. Three types of interhemispheric coupling. Journal of Neurophysiology, 74, 2379–2400.

    PubMed  Google Scholar 

  • Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381, 607–609.

    PubMed  Google Scholar 

  • Olshausen, B. A., & Field, D. J. (2004). Sparse coding of sensory inputs. Current Opinion in Neurobiology, 14, 481–487.

    PubMed  Google Scholar 

  • Paradiso, M. A. (1988). A theory for the use of visual orientation information which exploits the columnar structure of striate cortex. Biological Cybernetics, 58, 35–49.

    PubMed  Google Scholar 

  • Pashler, H. (1988). Familiarity and visual change detection. Attention, Perception, & Psychophysics, 44, 369–378.

    Google Scholar 

  • Poggio, T., Torre, V., & Koch, C. (1985). Computational vision and regularization theory. Nature, 317, 314–319.

    PubMed  Google Scholar 

  • Qian, N. (1994). Computing stereo disparity and motion with known binocular cell properties. Neural Computation, 6, 390–404.

    Google Scholar 

  • Qian, N. (1997). Binocular disparity and the perception of depth. Neuron, 18, 359–368.

    PubMed  Google Scholar 

  • Rao, R. P. N., & Ballard, D. H. (1997). Dynamic model of visual recognition predicts neural response properties in the visual cortex. Neural Computation, 9, 721–763.

    PubMed  Google Scholar 

  • Rao, R. P. N., & Ballard, D. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2, 79–87.

    PubMed  Google Scholar 

  • Reid, R. C., & Alonso, J. M. (1995). Specificity of monosynaptic connections from thalamus to visual cortex. Nature, 378, 281–284.

    PubMed  Google Scholar 

  • Reynolds, J. H., & Heeger, D. J. (2009). The normalization model of attention. Neuron, 61, 168–185.

    PubMed  PubMed Central  Google Scholar 

  • Rissanen, J. (1978). Modeling by the shortest data description. Automata, 14, 465–471.

    Google Scholar 

  • Rissanen, J. (1983). A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11, 416–431.

    Google Scholar 

  • Rissanen, J. (1996). Fisher information and stochastic complexity. IEEE Transactions on Information Theory, 42, 40–47.

    Google Scholar 

  • Rissanen, J. (2001). Strong optimality of the normalized ML models as universal codes and information in data. IEEE Information Theory, 47, 1712–1717.

    Google Scholar 

  • Ruderman, D. L. (1994). The statistics of natural images. Network: Computation in Neural Systems, 5, 517–548.

    Google Scholar 

  • Sanger, T. D. (1996). Probability density estimation for the interpretation of neural population codes. Journal of Neurophysiology, 76, 2790–2793.

    PubMed  Google Scholar 

  • Schiller, P. H., Finlay, B. L., & Volman, S. F. (1976). Quantitative studies of single-cell properties in monkey striate cortex. II. Orientation specificity and ocular dominance. Journal of Neurophysiology, 39(6), 1320–1333.

  • Schneidman, E., Berry, M. J., Segev, R., & Bialek, W. (2006). Weak pairwise correlations imply strongly correlated network states in a neural population. Nature, 440, 1007–1012.

    PubMed  PubMed Central  Google Scholar 

  • Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423.

    Google Scholar 

  • Shlens, J., Field, G. D., Gauthier, J. L., Grivich, M. I., Petrusca, D., Sher, A., Litke, A. M., & Chichilnisky, E. J. (2006). The structure of multi-neuron firing patterns in primate retina. The Journal of Neuroscience, 26, 8254–8266.

    PubMed  PubMed Central  Google Scholar 

  • Sigman, M., Cecchi, G. A., Gilbert, C. D., & Magnasco, M. O. (2001). On a common circle: natural scenes and gestalt rules. Proceedings of the National Academy of Sciences of the United States of America, 98, 1935–1940.

    PubMed  PubMed Central  Google Scholar 

  • Simoncelli, E. P., & Heeger, D. J. (1998). A model of neuronal responses in visual area MT. Vision Research, 38, 743–761.

    PubMed  Google Scholar 

  • Simoncelli, E. P., & Olshausen, B. A. (2001). Natural image statistics and neural representation. Annual Review of Neuroscience, 24, 1193–1216.

    PubMed  Google Scholar 

  • Sims, C. R. (2018). Efficient coding explains the universal law of generalization in human perception. Science, 360, 652–656.

    PubMed  Google Scholar 

  • Stocker, A. A., & Simoncelli, E. P. (2006). Sensory adaptation within a Bayesian framework for perception. In Y. Weiss, B. Scholkopf, & J. Platt (Eds.), Advances in neural information processing systems. Cambridge: MIT Press.

    Google Scholar 

  • Tanaka, H., Krakauer, J. W., & Qian, N. (2006). An optimization principle for determining movement duration. Journal of Neurophysiology, 95, 3875–3886.

    PubMed  Google Scholar 

  • Teich, A. F., & Qian, N. (2003). Learning and adaptation in a recurrent model of V1 orientation selectivity. Journal of Neurophysiology, 89, 2086–2100.

    PubMed  Google Scholar 

  • Teich, A. F., & Qian, N. (2006). Comparison among some models of orientation selectivity. Journal of Neurophysiology, 96, 404–419.

    PubMed  Google Scholar 

  • Teich, A. F., & Qian, N. (2010). V1 orientation plasticity is explained by broadly tuned feedforward inputs and intracortical sharpening. Visual Neuroscience, 27, 57–73.

    PubMed  PubMed Central  Google Scholar 

  • Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 381, 520–522.

    PubMed  Google Scholar 

  • Tishby N, Pereira FC, Bialek W. (2000). The information bottleneck method. arXiv:physics/0004057.

  • van Kan, P. L. E., Scobey, R. P., & Gabor, A. J. (1985). Response covariance in cat visual cortex. Experimental Brain Research, 60, 559–563.

    PubMed  Google Scholar 

  • Webster, M. A., & De Valois, R. L. (1985). Relationship between spatial-frequency and orientation tuning of striate-cortex cells. Journal of the Optical Society of America A. Optics and Image Science, 2, 1124–1132.

    PubMed  Google Scholar 

  • Weiss, Y., Simoncelli, E. P., & Adelson, E. H. (2002). Motion illusions as optimal percepts. Nature Neuroscience, 5, 598–604.

    PubMed  Google Scholar 

  • Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014). Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111, 8619–8624.

    Google Scholar 

  • Yang, Z., & Purves, D. (2003). A statistical explanation of visual space. Nature Neuroscience, 6, 632–640.

    PubMed  Google Scholar 

  • Yenduri PK, Zhang J, Gilbert A. (2012) Proceedings of the Third International Conference on Intelligent Control and Information Processing, ICICIP 20122012.

  • Yuille, A., & Kersten, D. (2006). Vision as Bayesian inference: analysis by synthesis? Trends in Cognitive Sciences, 10, 301–308.

    PubMed  Google Scholar 

  • Zhang, J. (2011). Model selection with informative normalized maximal likelihood: data prior and model prior. In E. N. Dzhafarov & L. Perry (Eds.), Descriptive and normative approaches to human behavior (pp. 303–319). Hackensack: World Scientific.

    Google Scholar 

  • Zhaoping, L. (2002). A saliency map in primary visual cortex. Trends in Cognitive Sciences, 6, 9–16.

    Google Scholar 

  • Zhaoping, L. (2014). Understanding vision: theory, models, and data. London: Oxford University Press.

    Google Scholar 

Download references

Acknowledgments

We thank Dr. Jay Myung for encouraging us to explore this topic and for his many helpful comments and suggestions.

Funding

This work was supported by AFOSR FA9550-15-1-0439 and NSF 1754211.

Author information

Authors and Affiliations

  1. Department of Neuroscience, Zuckerman Institute, and Department of Physiology & Cellular Biophysics, Columbia University, New York, NY, 10027, USA

    Ning Qian

  2. Department of Psychology and Department of Mathematics, University of Michigan, Ann Arbor, MI, 48109, USA

    Jun Zhang

Authors
  1. Ning Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ning Qian.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qian, N., Zhang, J. Neuronal Firing Rate As Code Length: a Hypothesis. Comput Brain Behav 3, 34–53 (2020). https://doi.org/10.1007/s42113-019-00028-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42113-019-00028-z

Keywords

Search

Navigation