Abstract
Many theories assume that a sensory neuron’s higher firing rate indicates a greater probability of its preferred stimulus. However, this contradicts (1) the adaptation phenomena where prolonged exposure to, and thus increased probability of, a stimulus reduces the firing rates of cells tuned to the stimulus; and (2) the observation that unexpected (low probability) stimuli capture attention and increase neuronal firing. Other theories posit that the brain builds predictive/efficient codes for reconstructing sensory inputs. However, they cannot explain that the brain preserves some information while discarding other. We propose that in sensory areas, projection neurons’ firing rates are proportional to optimal code length (i.e., negative log estimated probability), and their spike patterns are the code, for useful features in inputs. This hypothesis explains adaptation-induced changes of V1 orientation tuning curves and bottom-up attention. We discuss how the modern minimum-description-length (MDL) principle may help understand neural codes. Because regularity extraction is relative to a model class (defined by cells) via its optimal universal code (OUC), MDL matches the brain’s purposeful, hierarchical processing without input reconstruction. Such processing enables input compression/understanding even when model classes do not contain true models. Top-down attention modifies lower-level OUCs via feedback connections to enhance transmission of behaviorally relevant information. Although OUCs concern lossless data compression, we suggest possible extensions to lossy, prefix-free neural codes for prompt, online processing of most important aspects of stimuli while minimizing behaviorally relevant distortion. Finally, we discuss how neural networks might learn MDL’s normalized maximum likelihood (NML) distributions from input data.
Similar content being viewed by others
References
Adrian, E. D. (1926). The impulses produced by sensory nerve endings. The Journal of Physiology, 61, 49–72.
Albrecht, D. G., & Geisler, W. S. (1991). Motion selectivity and the contrast-response function of simple cells in the visual cortex. Visual Neuroscience, 7, 531–546.
Allman, J., Miezin, F., & McGuinness, E. (1985). Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. Annual Review of Neuroscience, 8, 407–430.
Anderson, J. S., Carandini, M., & Ferster, D. (2000). Orientation tuning of input conductance, excitation, and inhibition in cat primary visual cortex. Journal of Neurophysiology, 84, 909–926.
Atick, J. J., & Redlich, A. N. (1990). Towards a theory of early visual processing. Neural Computation, 2, 308–320.
Balasubramanian, V., Berry, M. J., & Kimber, D. (2001). Metabolically efficient information processing. Neural Computation, 13, 799–816.
Barlow, H. (1972). Single units and sensation: a neuron doctrine for perceptual psychology? Perception, 1(4), 371–394.
Barlow, H., & Foldiak, P. (1989). Adaptation and decorrelation in the cortex. In R. Durbin, C. Miall, & G. Mitchinson (Eds.), The computing neuron (pp. 54–72). New York: Addison-Wesley.
Barron, A., Rissanen, J., & Yu, B. (1998). The minimum description length principle in coding and modeling. IEEE Transactions on Information Theory, 44, 2743–2760.
Bell, A. J., & Sejnowski, T. J. (1997). The “independent components” of natural scenes are edge filters. Vision Research, 37, 3327–3338.
Blahut, R. E. (1972). Computation of channel capacity and rate-distortion functions. IEEE Transactions on Information Theory, 18, 460–473.
Blakemore, C., & Campbell, F. W. (1969). Adaptation to spatial stimuli. The Journal of Physiology, 200, 11P–13P.
Bolz, J., & Gilbert, C. D. (1986). Generation of end-inhibition in the visual cortex via interlaminar connections. Nature, 320, 362–365.
Carandini, M., & Heeger, D. J. (2012). Normalization as a canonical neural computation. Nature Reviews. Neuroscience, 13, 51–62.
Carandini, M., Heeger, D. J., & Movshon, J. A. (1997). Linearity and normalization in simple cells of the macaque primary visual cortex. The Journal of Neuroscience, 17, 8621–8644.
Chen, L. (1982). Topological structure in visual perception. Science, 218, 699–700.
Dayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995). The Helmholtz machine. Neural Computation, 7, 889–904.
Ding, S., Cueva, C. J., Tsodyks, M., & Qian, N. (2017). Visual perception as retrospective Bayesian decoding from high- to low-level features. Proceedings of the National Academy of Sciences, 114, E9115–E9124.
Dragoi, V., Sharma, J., & Sur, M. (2000). Adaptation-induced plasticity of orientation tuning in adult visual cortex. Neuron, 28, 287–298.
Dragoi, V., Rivadulla, C., & Sur, M. (2001). Foci of orientation plasticity in visual cortex. Nature, 411, 80–86.
Fang, F., Murray, S. O., Kersten, D., & He, S. (2005). Orientation-tuned fMRI adaptation in human visual cortex. Journal of Neurophysiology, 94, 4188–4195.
Felsen, G., Shen, Y. S., Yao, H., Spor, G., Li, C., & Dan, Y. (2002). Dynamic modification of cortical orientation tuning mediated by recurrent connections. Neuron, 36, 945–954.
Field, D. J. (1987). Relations between the statistics of natural images and the response properties of cortical-cells. Journal of the Optical Society of America. A, Optics, Image Science, and Vision, 4, 2379–2394.
Flash, T., & Hogan, N. (1985). The coordination of arm movements - an experimentally confirmed mathematical-model. The Journal of Neuroscience, 5, 1688–1703.
Friston (2010). The free-energy principle: a unified brain theory? Nature Reviews Neuroscience, 11, 127–138.
Gallant, J. L., Connor, C. E., & Van Essen, D. C. (1998). Neural activity in areas V1, V2 and V4 during free viewing of natural scenes compared to controlled viewing. Neuroreport, 9, 2153–2158.
Ganmor, E., Segev, R., & Schneidman, E. (2011). The architecture of functional interaction networks in the retina. The Journal of Neuroscience, 31, 3044–3054.
Geisler, W. S., Perry, J. S., Super, B. J., & Gallogly, D. P. (2001). Edge co-occurrence in natural images predicts contour grouping performance. Vision Research, 41, 711–724.
Georgopoulos, A. P., Schwartz, A. B., & Kettner, R. E. (1986). Neuronal population coding of movement direction. Science, 233, 1416–1419.
Gibson, J. J., & Radner, M. (1937). Adaptation, after-effect and contrast in the perception of tilted lines. I. Quantitative studies. Journal of Experimental Psychology, 20, 453–467.
Gottlieb, J. P., Kusunoki, M., & Goldberg, M. E. (1998). The representation of visual salience in monkey parietal cortex. Nature, 391, 481–484.
Grunwald, P. D. (2007). The minimum description length principle. Cambridge: MIT Press.
Grunwald, P. D., Myung, I. J., & Pitt, M. A. (2005). Advances in minimum description length: theory and applications. Cambridge: MIT Press.
Han, S., & Vasconcelos, N. (2010). Biologically plausible saliency mechanisms improve feedforward object recognition. Vision Research, 50, 2295–2307.
Harpur, G. F., & Prager, R. W. (1996). Development of low entropy coding in a recurrent network*. Network: Computation in Neural Systems, 7, 277–284.
Heeger, D. J. (1992). Normalization of cell responses in cat striate cortex. Visual Neuroscience, 9, 181–197.
Hinton, G., Dayan, P., Frey, B., & Neal, R. (1995). The “wake-sleep” algorithm for unsupervised neural networks. Science, 268, 1158–1161.
Hubel, D. H., & Wiesel, T. N. (1968). Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology, 195, 215–243.
Itti, L., & Koch, C. (2001). Computational modelling of visual attention. Nature Reviews. Neuroscience, 2, 194–203.
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014). Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Computational Biology, 10, e1003915.
Kingma DP, Welling M. (2013). Auto-encodrwing variational bayes. arXiv preprint arXiv:1312.6114.
Levitt, J. B., & Lund, J. S. (1997). Contrast dependence of contextual effects in primate visual cortex. Nature, 387, 73–76.
Li, W., & Gilbert, C. D. (2002). Global contour saliency and local colinear interactions. Journal of Neurophysiology, 88, 2846–2856.
Li, C.-Y., & Li, W. (1994). Extenstive integration field beyond the classical receptive field of cat’s striate cortical neurons - classification and tuning properties. Vision Research, 34, 2337–2355.
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006). Bayesian inference with probabilistic population codes. Nature Neuroscience, 9, 1432–1438.
Meng, X., & Qian, N. (2005). The oblique effect depends on perceived, rather than physical, orientation and direction. Vision Research, 45, 3402–3413.
Motoyoshi, I., Sy, N., Sharan, L., & Adelson, E. H. (2007). Image statistics and the perception of surface qualities. Nature, 447, 206–209.
Myung, J. I., Navarro, D. J., & Pitt, M. A. (2006). Model selection by normalized maximum likelihood. Journal of Mathematical Psychology, 50, 167–179.
Navon, D. (1977). Forest before trees: the precedence of global features in visual perception. Cognitive Psychology, 9, 353–383.
Nelson, J. I., & Frost, B. J. (1978). Orientation-selective inhibition from beyond the classic visual receptive field. Brain Research, 139, 359–365.
Nowak, L. G., Munk, M. H., Nelson, J. I., James, A. C., & Bullier, J. (1995). Structural basis of cortical synchronization. I. Three types of interhemispheric coupling. Journal of Neurophysiology, 74, 2379–2400.
Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381, 607–609.
Olshausen, B. A., & Field, D. J. (2004). Sparse coding of sensory inputs. Current Opinion in Neurobiology, 14, 481–487.
Paradiso, M. A. (1988). A theory for the use of visual orientation information which exploits the columnar structure of striate cortex. Biological Cybernetics, 58, 35–49.
Pashler, H. (1988). Familiarity and visual change detection. Attention, Perception, & Psychophysics, 44, 369–378.
Poggio, T., Torre, V., & Koch, C. (1985). Computational vision and regularization theory. Nature, 317, 314–319.
Qian, N. (1994). Computing stereo disparity and motion with known binocular cell properties. Neural Computation, 6, 390–404.
Qian, N. (1997). Binocular disparity and the perception of depth. Neuron, 18, 359–368.
Rao, R. P. N., & Ballard, D. H. (1997). Dynamic model of visual recognition predicts neural response properties in the visual cortex. Neural Computation, 9, 721–763.
Rao, R. P. N., & Ballard, D. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2, 79–87.
Reid, R. C., & Alonso, J. M. (1995). Specificity of monosynaptic connections from thalamus to visual cortex. Nature, 378, 281–284.
Reynolds, J. H., & Heeger, D. J. (2009). The normalization model of attention. Neuron, 61, 168–185.
Rissanen, J. (1978). Modeling by the shortest data description. Automata, 14, 465–471.
Rissanen, J. (1983). A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11, 416–431.
Rissanen, J. (1996). Fisher information and stochastic complexity. IEEE Transactions on Information Theory, 42, 40–47.
Rissanen, J. (2001). Strong optimality of the normalized ML models as universal codes and information in data. IEEE Information Theory, 47, 1712–1717.
Ruderman, D. L. (1994). The statistics of natural images. Network: Computation in Neural Systems, 5, 517–548.
Sanger, T. D. (1996). Probability density estimation for the interpretation of neural population codes. Journal of Neurophysiology, 76, 2790–2793.
Schiller, P. H., Finlay, B. L., & Volman, S. F. (1976). Quantitative studies of single-cell properties in monkey striate cortex. II. Orientation specificity and ocular dominance. Journal of Neurophysiology, 39(6), 1320–1333.
Schneidman, E., Berry, M. J., Segev, R., & Bialek, W. (2006). Weak pairwise correlations imply strongly correlated network states in a neural population. Nature, 440, 1007–1012.
Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423.
Shlens, J., Field, G. D., Gauthier, J. L., Grivich, M. I., Petrusca, D., Sher, A., Litke, A. M., & Chichilnisky, E. J. (2006). The structure of multi-neuron firing patterns in primate retina. The Journal of Neuroscience, 26, 8254–8266.
Sigman, M., Cecchi, G. A., Gilbert, C. D., & Magnasco, M. O. (2001). On a common circle: natural scenes and gestalt rules. Proceedings of the National Academy of Sciences of the United States of America, 98, 1935–1940.
Simoncelli, E. P., & Heeger, D. J. (1998). A model of neuronal responses in visual area MT. Vision Research, 38, 743–761.
Simoncelli, E. P., & Olshausen, B. A. (2001). Natural image statistics and neural representation. Annual Review of Neuroscience, 24, 1193–1216.
Sims, C. R. (2018). Efficient coding explains the universal law of generalization in human perception. Science, 360, 652–656.
Stocker, A. A., & Simoncelli, E. P. (2006). Sensory adaptation within a Bayesian framework for perception. In Y. Weiss, B. Scholkopf, & J. Platt (Eds.), Advances in neural information processing systems. Cambridge: MIT Press.
Tanaka, H., Krakauer, J. W., & Qian, N. (2006). An optimization principle for determining movement duration. Journal of Neurophysiology, 95, 3875–3886.
Teich, A. F., & Qian, N. (2003). Learning and adaptation in a recurrent model of V1 orientation selectivity. Journal of Neurophysiology, 89, 2086–2100.
Teich, A. F., & Qian, N. (2006). Comparison among some models of orientation selectivity. Journal of Neurophysiology, 96, 404–419.
Teich, A. F., & Qian, N. (2010). V1 orientation plasticity is explained by broadly tuned feedforward inputs and intracortical sharpening. Visual Neuroscience, 27, 57–73.
Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 381, 520–522.
Tishby N, Pereira FC, Bialek W. (2000). The information bottleneck method. arXiv:physics/0004057.
van Kan, P. L. E., Scobey, R. P., & Gabor, A. J. (1985). Response covariance in cat visual cortex. Experimental Brain Research, 60, 559–563.
Webster, M. A., & De Valois, R. L. (1985). Relationship between spatial-frequency and orientation tuning of striate-cortex cells. Journal of the Optical Society of America A. Optics and Image Science, 2, 1124–1132.
Weiss, Y., Simoncelli, E. P., & Adelson, E. H. (2002). Motion illusions as optimal percepts. Nature Neuroscience, 5, 598–604.
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014). Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111, 8619–8624.
Yang, Z., & Purves, D. (2003). A statistical explanation of visual space. Nature Neuroscience, 6, 632–640.
Yenduri PK, Zhang J, Gilbert A. (2012) Proceedings of the Third International Conference on Intelligent Control and Information Processing, ICICIP 20122012.
Yuille, A., & Kersten, D. (2006). Vision as Bayesian inference: analysis by synthesis? Trends in Cognitive Sciences, 10, 301–308.
Zhang, J. (2011). Model selection with informative normalized maximal likelihood: data prior and model prior. In E. N. Dzhafarov & L. Perry (Eds.), Descriptive and normative approaches to human behavior (pp. 303–319). Hackensack: World Scientific.
Zhaoping, L. (2002). A saliency map in primary visual cortex. Trends in Cognitive Sciences, 6, 9–16.
Zhaoping, L. (2014). Understanding vision: theory, models, and data. London: Oxford University Press.
Acknowledgments
We thank Dr. Jay Myung for encouraging us to explore this topic and for his many helpful comments and suggestions.
Funding
This work was supported by AFOSR FA9550-15-1-0439 and NSF 1754211.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Qian, N., Zhang, J. Neuronal Firing Rate As Code Length: a Hypothesis. Comput Brain Behav 3, 34–53 (2020). https://doi.org/10.1007/s42113-019-00028-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42113-019-00028-z