Skip to main content
SpringerLink
Log in

Information, Novelty, and Surprise in Brain Theory

  • Chapter
  • First Online:
Novelty, Information and Surprise

  • 883 Accesses

Abstract

In biological research it is common to assume that each organ of an organism serves a definite purpose. The purpose of the brain seems to be the coordination and processing of information which the animal obtains through its sense organs about the outside world and about its own internal state (Bateson 1972). An important aspect of this is the storage of information in memory and the use of the stored information in connection with the present sensory stimuli.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abbott, L. F. (1994). Decoding neuronal firing and modeling neural networks. Quarterly Reviews of Biophysics, 27, 291–331.

    Google Scholar 

  2. Abeles, M. (1991). Corticonics: Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press.

    Google Scholar 

  3. Abeles, M., & Gerstein, G. L. (1988). Detecting spatiotemporal firing patterns among simultaneously recorded single neurons. Journal of Neurophysiology, 60(3), 909–924.

    Google Scholar 

  4. Abeles, M., & Lass, Y. (1975). Transmission of information by the axon: II. The channel capacity. Biological Cybernetics, 19(3), 121–125.

    Google Scholar 

  5. Abeles, M., Bergman, H., Margalit, E., & Vaadia, E. (1993). Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. Journal of Neurophysiology, 70(4), 1629–1638.

    Google Scholar 

  6. Abeles, M., Bergman, H., Gat, I., Meilijson, I., Seidemann, E., Tishby, N., & Vaadia, E. (1995). Cortical activity flips among quasi stationary states. Proceedings of the National Academy of Sciences of the United States of America, 92, 8616–8620.

    Google Scholar 

  7. Adelman, T. L., Bialek, W., & Olberg, R. M. (2003). The information content of receptive fields. Neuron, 40(13), 823–833.

    Google Scholar 

  8. Aertsen, A. M. H. J., & Johannesma, P. I. M. (1981). The spectro-temporal receptive field. A functional characteristic of auditory neurons. Biological Cybernetics, 42(2), 133–143.

    Google Scholar 

  9. Aertsen, A. M. H. J., Gerstein, G. L., Habib, M. K., & Palm, G. (1989). Dynamics of neuronal firing correlation: Modulation of “effective connectivity”. Journal of Neurophysiology, 61(5), 900–917.

    Google Scholar 

  10. Amari, S.-i., & Nakahara, H. (2005). Difficulty of singularity in population coding. Neural Computation, 17, 839–858.

    Google Scholar 

  11. Amari, S., & Nakahara, H. (2006). Correlation and independence in the neural code. Neural Computation, 18(6), 1259–1267.

    Google Scholar 

  12. Arieli, A., Sterkin, A., Grinvald, A., & Aertsen, A. M. H. J. (1996). Dynamics of ongoing activity: Explanation of the large variability in evoked cortical responses. Science, 273(5283), 1868–1871.

    Google Scholar 

  13. Atick, J. J. (1992). Could information theory provide an ecological theory of sensory processing? Network: Computation in Neural Systems, 3, 213–251.

    Google Scholar 

  14. Atick, J. J., & Redlich, A. N. (1990). Towards a theory of early visual processing. Neural Computation, 2(3), 308–320.

    Google Scholar 

  15. Atick, J. J., & Redlich, A. N. (1992). What does the retina know about natural scenes? Cambridge: MIT Press.

    Google Scholar 

  16. Attneave, F. (1959). Applications of information theory to psychology. New York: Holt, Rinehart and Winston.

    Google Scholar 

  17. Bach, M., & Krüger, J. (1986). Correlated neuronal variability in monkey visual cortex revealed by a multi-microelectrode. Experimental Brain Research, 61(3), 451–456.

    Google Scholar 

  18. Bair, W., & Koch, C. (1996). Temporal precision of spike trains in extrastriate cortex of the behaving macaque monkey. Neural Computation, 8(6), 1185–1202.

    Google Scholar 

  19. Baker, S. N., & Lemon, R. N. (2000). Precise spatiotemporal repeating patterns in monkey primary and supplementary motor areas occur at chance levels. Journal of Neurophysiology, 84, 1770–1780.

    Google Scholar 

  20. Bar-Hillel, Y., & Carnap, R. (1953). Semantic information. In London information theory symposium (pp. 503–512). New York: Academic.

    Google Scholar 

  21. Barlow, H. B. (1961). Possible principles underlying the transformation of sensory messages. Cambridge: MIT Press.

    Google Scholar 

  22. Barlow, H. B. (1989). Unsupervised learning. Neural Computation, 1, 295–311.

    Google Scholar 

  23. Barlow, H. B., & Földiák, P. (1989). Adaptation and decorrelation in the cortex. In C. Miall, R. M. Durbin, & G. J. Mitcheson (Eds.), The computing neuron (pp. 54–72). USA: Addison-Wesley.

    Google Scholar 

  24. Barlow, H. B., Kaushal, T. P., & Mitchison, G. J. (1989). Finding minimum entropy codes. Neural Computation, 1(3), 412–423.

    Google Scholar 

  25. Barnard, G. A. (1955). Statistical calculation of word entropies for four Western languages. IEEE Transactions on Information Theory, 1(1), 49–53.

    Google Scholar 

  26. Bateson, G. (1972). Steps to an ecology of mind. London: Intertext Books.

    Google Scholar 

  27. Bell, A. J., & Sejnowski, T. J. (1995). An information-maximisation approach to blind separation and blind deconvolution. Neural Computation, 7, 1129–1159.

    Google Scholar 

  28. Bethge, M., Rotermund, D., & Pawelzik, K. (2002). Optimal short-term population coding: When Fisher information fails. Neural Computation, 14, 2317–2351.

    Google Scholar 

  29. Bi, G.-Q., & Poo, M.-M. (1998). Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type. The Journal of Neuroscience, 18, 10464–10472.

    Google Scholar 

  30. Bialek, W., de Ruyter van Steveninck, R. R., & Tishby, N. (2007). Efficient representation as a design principle for neural coding and computation. Neural Computation, 19(9), 2387-2432.

    Google Scholar 

  31. Bialek, W., Reike, F., de Ruyter van Steveninck, R. R., & Warland, D. (1991). Reading a neural code. Science, 252, 1854–1857.

    Google Scholar 

  32. Bliss, T. V. P., & Collingridge, G. L. (1993). A synaptic model of memory: Long-term potentiation in the hippocampus. Nature, 361, 31–39.

    Google Scholar 

  33. Borst, A., & Theunissen, F. E. (1999). Information theory and neural coding. Nature Neuroscience, 2(11), 947–957.

    Google Scholar 

  34. Brenner, N., Strong, S., Koberle, R., Bialek, W., & de Ruyter van Steveninck, R. (2000). Synergy in a neural code. Neural Computation, 12(7), 1531–1552.

    Google Scholar 

  35. Brown, E. N., Kass, R. E., & Mitra, P. P. (2004). Multiple neural spike train data analysis: State-of-the-art and future challenges. Nature Neuroscience, 7, 456–461. doi: 10.1038/nn1228.

    Google Scholar 

  36. Brunel, N., & Nadal, J.-P. (1998). Mutual information, Fisher information, and population coding. Neural Computation, 10(7), 1731–1757.

    Google Scholar 

  37. Butts, D. A. (2003). How much information is associated with a particular stimulus? Network: Computation in Neural Systems, 14(2), 177–187.

    Google Scholar 

  38. Butts, D. A., & Goldman, M. (2006). Tuning curves, neuronal variability and sensory coding. PLOS Biology, 4, 639–646.

    Google Scholar 

  39. Butts, D. A., Weng, C., Jin, J., Yeh, C.-I., Lesica, N. A., Alonso, J.-M., & Stanley, G. B. (2007). Temporal precision in the neural code and the timescales of natural vision. Nature, 449(7158), 92–95.

    Google Scholar 

  40. Cessac, B., Rostro-González, H., Vasquez, J.-C., & Viéville, T. (2008). To which extend is the “neural code” a metric? In Proceedings of the conference NeuroComp 2008. Informal publication.

    Google Scholar 

  41. Cherry, C. (1966). On human communication. Cambridge: MIT Press.

    Google Scholar 

  42. Christodoulou, C., & Bugmann, G. (2001). Coefficient of variation (CV) vs mean inter-spike-interval (ISI) curves: What do they tell us about the brain? Neurocomputing, 38–40, 1141–1149.

    Google Scholar 

  43. Coulter, W. K., Hillar, C. J., & Sommer, F. T. (2009). Adaptive compressed sensing—a new class of self-organizing coding models for neuroscience.

    Google Scholar 

  44. Dan, Y., & Poo, M.-M. (2006). Spike timing-dependent plasticity: From synapse to perception. Physiology Review, 86, 1033–1048.

    Google Scholar 

  45. Dan, Y., Atick, J. J., & Reid, R. C. (1996). Efficient coding of natural scenes in the lateral geniculate nucleus: Experimental test of a computational theory. Journal of Neuroscience, 16(10), 3351–3362.

    Google Scholar 

  46. Dayhoff, J. E., & Gerstein, G. L. (1983a). Favored patterns in spike trains. I. Detection. Journal of Neurophysiology, 49(6), 1334–1348.

    Google Scholar 

  47. Dayhoff, J. E., & Gerstein, G. L. (1983b). Favored patterns in spike trains. II. Application. Journal of Neurophysiology, 49(6), 1349–1363.

    Google Scholar 

  48. Deadwyler, S. A., & Hampson, R. E. (1997). The significance of neural ensemble codes during behavior and cognition. Annual Review of Neuroscience, 20, 217–244.

    Google Scholar 

  49. Dean, I., Harper, N. S., & D. McAlpine (2005). Neural population coding of sound level adapts to stimulus statistics. Nature Neuroscience, 8(12), 1684–1689.

    Google Scholar 

  50. Denève, S. (2008). Bayesian spiking neurons I: Inference. Neural Computation, 20, 91–117.

    Google Scholar 

  51. Dong, D. W., & Atick, J. J. (1995). Statistics of natural time-varying images. Network, 6(3), 345–358.

    Google Scholar 

  52. Doob, J. L. (1953). Stochastic Processes. New York: Wiley.

    Google Scholar 

  53. Eckhorn, R. (1999). Neural mechanisms of scene segmentation: Recordings from the visual cortex suggest basic circuits for linking field models. IEEE Transactions on Neural Networks, 10(3), 464–479.

    Google Scholar 

  54. Eckhorn, R., Grüsser, O.-J., Kröller, J., Pellnitz, K., & Pöpel, B. (1976). Efficiency of different neuronal codes: Information transfer calculations for three different neuronal systems. Biological Cybernetics, 22(1), 49–60.

    Google Scholar 

  55. Edelman, G. M., & Tononi, G. (2000). A universe of consciousness: How matter becomes imagination. New York: Basic Books.

    Google Scholar 

  56. Engel, A., Fries, P., & Singer, W. (2001). Dynamic predictions: Oscillations and synchrony in top-down processing. Nature Reviews Neuroscience, 2(10), 704–716.

    Google Scholar 

  57. Field, G. D., & Chichilnisky, E. J. (2007). Information processing in the primate retina: Circuitry and coding. Annual Review of Neuroscience, 30, 1–30.

    Google Scholar 

  58. Furber, S. B., Brown, G., Bose, J., Cumpstey, J. M., Marshall, P., & Shapiro, J. L. (2007). Sparse distributed memory using rank-order neural codes. IEEE Transactions on Neural Networks, 18, 648–659.

    Google Scholar 

  59. Gerstein, G. L., & Aertsen, A. M. (1985). Representation of cooperative firing activity among simultaneously recorded neurons. Journal of Neurophysiology, 54(6), 1513–1528.

    Google Scholar 

  60. Gerstein, G. L., & Mandelbrot, B. (1964). Random walk models for the spike activity of a single neuron. Biophysical Journal, 4(1), 41–68.

    Google Scholar 

  61. Gerstner, W., & Kistler, W. M. (2002). Spiking Neuron Models. New York: Cambridge University Press.

    Google Scholar 

  62. Gerstner, W., Kreiter, A. K., Markram, H., & Herz, A. V. M. (1997). Neural codes: Firing rates and beyond. Proceedings of the National Academy of Sciences of the United States of America, 94(24), 12740–12741.

    Google Scholar 

  63. Golomb, D., Hertz, J., Panzeri, S., Treves, A., & Richmond, B. (1997). How well can we estimate the information carried in neuronal responses from limited samples? Neural Computation, 9(3), 649–665.

    Google Scholar 

  64. Grossberg, S. (1999). How does the cerebral cortex work? Learning, attention and grouping by the laminar circuits of visual cortex. Spatial Vision, 12, 163–186.

    Google Scholar 

  65. Grün, S., Aertsen, A. M. H. J., Abeles, M., Gerstein, G., & Palm, G. (1994a). Behavior-related neuron group activity in the cortex. In Proceedings 17th Annual Meeting European Neuroscience Association. Oxford. Oxford University Press.

    Google Scholar 

  66. Grün, S., Aertsen, A. M. H. J., Abeles, M., Gerstein, G., & Palm, G. (1994b). On the significance of coincident firing in neuron group activity. In N. Elsner, & H. Breer (Eds.), Sensory transduction (p. 558). Thieme: Stuttgart.

    Google Scholar 

  67. Grün, S., Diesmann, M., & Aertsen, A. (2002a). Unitary events in multiple single-neuron spiking activity: I. Detection and significance. Neural Computation, 14(1), 43–80.

    Google Scholar 

  68. Grün, S., Diesmann, M., & Aertsen, A. (2002b). Unitary events in multiple single-neuron spiking activity: II. Nonstationary data. Neural Computation, 14(1), 81–119.

    Google Scholar 

  69. Grün, S., Diesmann, M., Grammont, F., Riehle, A., & Aertsen, A. (1999). Detecting unitary events without discretization of time. Journal of Neuroscience, 94(1), 121–154.

    Google Scholar 

  70. Grün, S., & Rotter, S. (Eds.) (2010). Analysis of spike trains. New York: Springer.

    Google Scholar 

  71. Gütig, R., Aertsen, A., & Rotter, S. (2002). Statistical significance of coincident spikes: Count-based versus rate-based statistics. Neural Computation, 14(1), 121–153.

    Google Scholar 

  72. Gutnisky, D. A., & Dragoi, V. (2008). Adaptive coding of visual information in neural populations. Nature, 452(7184), 220–224.

    Google Scholar 

  73. Guyonneau, R., VanRullen, R., & Thorpe, S. J. (2004). Temporal codes and sparse representations: A key to understanding rapid processing in the visual system. Journal of Physiology – Paris, 98, 487–497.

    Google Scholar 

  74. Haft, M., & van Hemmen, J. L. (1998). Theory and implementation of infomax filters for the retina. Network, 9, 39–71.

    Google Scholar 

  75. Hansel, D., & Sompolinsky, H. (1996). Chaos and synchrony in a model of a hypercolumn in visual cortex. Journal of Computational Neuroscience, 3(1), 7–34.

    Google Scholar 

  76. Hawkins, J., & Blakeslee, S. (2004). On intelligence. New York: Times Books, Henry Holt and Company.

    Google Scholar 

  77. Hebb, D. O. (1949). The organization of behavior: A neuropsychological theory. New York: Wiley.

    Google Scholar 

  78. Hecht-Nielsen, R. (2007). Confabulation theory. The mechanism of thought. Berlin: Springer.

    Google Scholar 

  79. Holden, A. V. (1976). Models of the stochastic activity of neurons. New York: Springer.

    Google Scholar 

  80. Hosaka, R., Araki, O., & Ikeguchi, T. (2008). STDP provides the substrate for igniting synfire chains by spatiotemporal input patterns. Neural Computation, 20(2), 415–435.

    Google Scholar 

  81. Hoyer, P. O., & Hyvärinen, A. (2002). A multi-layer sparse coding network learns contour coding from natural images. Vision Research, 42(12), 1593–1605.

    Google Scholar 

  82. Hyvärinen, A., & Hoyer, P. O. (2001). A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vision Research, 41(18), 2413–2423.

    Google Scholar 

  83. Hyvärinen, A., Hurri, J., & Hoyer, P. O. (2009). Natural Image Statistics. New York: Springer.

    Google Scholar 

  84. Hyvärinen, A., & Karhunen, J. (2001). Independent Component Analysis. New York: Wiley.

    Google Scholar 

  85. Izhikevich, E. M. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral Cortex, 17, 2443–2452.

    Google Scholar 

  86. Izhikevich, E. M., & Desai, N. S. (2003). Relating STDP to BCM. Neural Computation, 15, 15111523.

    Google Scholar 

  87. Johannesma, P. I. M. (1981). Neural representation of sensory stimuli and sensory interpretation of neural activity. Advanced Physiological Science, 30, 103–125.

    Google Scholar 

  88. Kamimura, R. (2002). Information theoretic neural computation. New York: World Scientific.

    Google Scholar 

  89. Kang, K., & Sompolinsky, H. (2001). Mutual information of population codes and distance measures in probability space. Physical Review Letter, 86(21), 4958–4961.

    Google Scholar 

  90. Kempter, R., Gerstner, W., & van Hemmen, J. L. (1999). Hebbian learning and spiking neurons. Physical Review E, 59, 4498–4514.

    Google Scholar 

  91. Kjaer, T. W., Hertz, J. A., & Richmond, B. J. (1994). Decoding cortical neuronal signals: Network models, information estimation, and spatial tuning. Journal of Computational Neuroscience, 1, 109–139.

    Google Scholar 

  92. Knoblauch, A., & Palm, G. (2004). What is Signal and What is Noise in the Brain? BioSystems, 79, 83–90.

    Google Scholar 

  93. Koepsell, K., & Sommer, F. T. (2008). Information transmission in oscillatory neural activity. Biological Cybernetics, 99, 403–416.

    Google Scholar 

  94. Koepsell, K., Wang, X., Vaingankar, V., Wei, Y., Wang, Q., Rathbun, D. L., Usrey, W. M., Hirsch, J. A., & Sommer, F. T. (2009). Retinal oscillations carry visual information to cortex. Frontiers in Systems Neuroscience, 3, 1–18.

    Google Scholar 

  95. König, P., Engel, A. K., & Singer, W. (1995). Relation between oscillatory activity and long-range synchronization in cat visual cortex. In Proceedings of the National Academy of Sciences of the United States of America, 92, 290–294.

    Google Scholar 

  96. Kostal, L., Lansky, P., & Rospars, J.-P. (2007). Neuronal coding and spiking randomness. European Journal of Neuroscience, 26(10), 2693–2701.

    Google Scholar 

  97. Krone, G., Mallot, H., Palm, G., & Schüz, A. (1986). Spatiotemporal receptive fields: A dynamical model derived from cortical architectonics. Proceedings of the Royal Society of London. Series B, Biological Sciences, 226(1245), 421–444.

    Google Scholar 

  98. Krüger, J., & Bach, M. (1981). Simultaneous recording with 30 microelectrodes in monkey visual cortex. Experimental Brain Research, 41(2), 191–194.

    Google Scholar 

  99. Legéndy, C. (2009). Circuits in the brain—a model of shape processing in the primary visual cortex. New York: Springer.

    Google Scholar 

  100. Legéndy, C. R. (1975). Three principles of brain function and structure. International Journal of Neuroscience, 6, 237–254.

    Google Scholar 

  101. Legéndy, C. R., & Salcman, M. (1985). Bursts and recurrences of bursts in the spike trains of spontaneously active striate cortex neurons. Journal of Neurophysiology, 53(4), 926–939.

    Google Scholar 

  102. Letvin, J. Y., Maturana, H. R., McCulloch, W. S., & Pitts, W. H. (1959). What the frog’s eye tells the frog’s brain. Proceedings of the IRE, 47(11), 1940–1951.

    Google Scholar 

  103. Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21, 105–117.

    Google Scholar 

  104. Linsker, R. (1989a). An application of the principle of maximum information preservation to linear systems. In D. S. Touretzky (Ed.), Advances in Neural Information Processing Systems (Vol. 1) (pp. 186–194). San Mateo: Morgan Kaufmann.

    Google Scholar 

  105. Linsker, R. (1989b). How to generate ordered maps by maximizing the mutual information between input and output signals. Neural Computation, 1(3), 402–411.

    Google Scholar 

  106. Linsker, R. (1992). Local synaptic learning rules suffice to maximize mutual information in a linear network. Neural Computation, 4, 691–702.

    Google Scholar 

  107. Linsker, R. (1997). A local learning rule that enables information maximization for arbitrary input distributions. Neural Computation, 9, 1661–1665.

    Google Scholar 

  108. Lisman, J., & Spruston, N. (2005). Postsynaptic depolarization requirements for LTP and LTD: A critique of spike timing-dependent plasticity. Nature Neuroscience, 8(7), 839–841.

    Google Scholar 

  109. Loiselle, S., Rouat, J., Pressnitzer, D., & Thorpe, S. J. (2005). Exploration of rank order coding with spiking neural networks for speech recognition. Proceedings of International Joint Conference on Neural Networks, 4, 2076–2078.

    Google Scholar 

  110. MacGregor, R. J. (1987). Neural and brain modeling. New York: Academic.

    Google Scholar 

  111. MacKay, D. M., & McCulloch, W. S. (1952). The limiting information capacity of a neuronal link. Bulletin of Mathematical Biology, 14(2), 127–135.

    Google Scholar 

  112. Mainen, Z. F., & Sejnowski, T. J. (1995). Reliability of spike timing in neocortical neurons. Science, 268(5216), 1503–1506.

    Google Scholar 

  113. Markram, H., Luebke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, 275, 213–215.

    Google Scholar 

  114. Martignon, L., Deco, G., Laskey, K., Diamond, M., Freiwald, W. A., & Vaadia, E. (2000). Neural coding: Higher-order temporal patterns in the neurostatistics of cell assemblies. Neural Computation, 12(11), 2621–2653.

    Google Scholar 

  115. Martignon, L., von Hasseln, H., Grün, S., Aertsen, A. M. H. J., & Palm, G. (1995). Detecting higher-order interactions among the spiking events in a group of neurons. Biological Cybernetics, 73(1), 69–81.

    Google Scholar 

  116. Martignon, L., von Hasseln, H., Grün, S., & Palm, G. (1994). Modelling the interaction in a set of neurons implicit in their frequency distribution: A possible approach to neural assemblies. In F. Allocati, C. Musio, & C. Taddei-Ferretti (Eds.), Biocybernetics (Cibernetica Biologica) (pp. 268–288). Torino: Rosenberg & Sellier.

    Google Scholar 

  117. Masquelier, T., Guyonneau, R., & Thorpe, S. (2009). Competitive STDP-based spike pattern learning. Neural Computation, 21(5), 1259–1276.

    Google Scholar 

  118. Massaro, D. W. (1975). Experimental psychology and human information processing. Chicago: Rand McNally & Co.

    Google Scholar 

  119. McClurkin, J. W., Gawne, T. J., Optican, L. M., & Richmond, B. J. (1991). Lateral geniculate neurons in behaving priimates II. Encoding of visual information in the temporal shape of the response. Journal of Neurophysiology, 66(3), 794–808.

    Google Scholar 

  120. Miller, J. G. (1962). Information input overload. In M. C. Yovits, G. T. Jacobi, & G. D. Goldstein (Eds.), Self-Organizing Systems (pp. 61–78). Washington DC: Spartan Books.

    Google Scholar 

  121. Morrison, A., Aertsen, A., & Diesmann, M. (2007). Spike-timing-dependent plasticity in balanced random networks. Neural Computation, 19(6), 1437–1467.

    Google Scholar 

  122. Morrison, A., Diesmann, M., & Gerstner, W. (2008). Phenomenological models of synaptic plasticity based on spike timing. Biological Cybernetics, 98, 459–478.

    Google Scholar 

  123. Nakahara, H., & Amari, S. (2002). Information geometric measure for neural spikes. Neural Computation, 14, 2269–2316.

    Google Scholar 

  124. Nakahara, H., Amari, S., & Richmond, B. J. (2006). A comparison of descriptive models of a single spike train by information geometric measure. Neural Computation, 18, 545–568.

    Google Scholar 

  125. Nemenman, I., Lewen, G. D., Bialek, W., & de Ruyter van Steveninck, R. R. (2008). Neural coding of natural stimuli: Information at sub-millisecond resolution. PLoS Computational Biology, 4(3), e1000025.

    Google Scholar 

  126. Nirenberg, S., & Latham, P. (2003). Decoding neural spike trains: How important are correlations? Proceedings of the National Academy of Science of the United States of America, 100, 7348–7353.

    Google Scholar 

  127. Nirenberg, S., & Latham, P. (2005). Synergy, redundancy and independence in population codes. Journal of Neuroscience, 25, 5195–5206.

    Google Scholar 

  128. Optican, L. M., Gawne, T. J., Richmond, B. J., & Joseph, P. J. (1991). Unbiased measures of transmitted information and channel capacity from multivariate neuronal data. Biological Cybernetics, 65(5), 305–310.

    Google Scholar 

  129. Optican, L. M., & Richmond, B. J. (1987). Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. III. Information theoretic analysis. Journal of Neurophysiology, 57(1), 162–178.

    Google Scholar 

  130. Osborne, L. C., Palmer, S. E., Lisberger, S. G., & Bialek, W. (2008). The neural basis for combinatorial coding in a cortical population response. Journal of Neuroscience, 28(50), 13522–13531.

    Google Scholar 

  131. Palm, G. (1980). On associative memory. Biological Cybernetics, 36, 167–183.

    Google Scholar 

  132. Palm, G. (1981). Evidence, information and surprise. Biological Cybernetics, 42(1), 57–68.

    Google Scholar 

  133. Palm, G. (1982). Neural assemblies, an alternative approach to artificial intelligence. New York: Springer.

    Google Scholar 

  134. Palm, G. (1985). Information und entropie. In H. Hesse (Ed.), Natur und Wissenschaft. Tubingen: Konkursbuch Tübingen.

    Google Scholar 

  135. Palm, G. (1987a). Associative memory and threshold control in neural networks. In J. L. Casti, & A. Karlqvist (Eds.), Real brains: artificial minds (pp. 165–179). New York: Elsevier.

    Google Scholar 

  136. Palm, G. (1987b). Computing with neural networks. Science, 235, 1227–1228.

    Google Scholar 

  137. Palm, G. (1992). On the information storage capacity of local learning rules. Neural Computation, 4, 703–711.

    Google Scholar 

  138. Palm, G., Aertsen, A. M. H. J., & Gerstein, G. L. (1988). On the significance of correlations among neuronal spike trains. Biological Cybernetics, 59(1), 1–11.

    Google Scholar 

  139. Palm, G., & Sommer, F. T. (1992). Information capacity in recurrent McCulloch–Pitts networks with sparsely coded memory states. Network, 3(2), 177–186.

    Google Scholar 

  140. Panzeri, S., & Schultz, S. R. (2001). A unified approach to the study of temporal, correlational, and rate coding. Neural Computation, 13(6), 1311–1349.

    Google Scholar 

  141. Panzeri, S., Schultz, S. R., Treves, A., & Rolls, E. T. (1999). Correlations and the encoding of information in the nervous system. Proceedings of the Royal Society of London Series B; Biological Science, 266(1423), 1001–1012.

    Google Scholar 

  142. Perkel, D. H., & Bullock, T. H. (1967). Neural coding. Neurosciences Research Program Bulletin, 6(3), 223–344.

    Google Scholar 

  143. Perrinet, L., Samuelides, M., & Thorpe, S. J. (2003). Coding static natural images using spike event times: Do neurons cooperate? IEEE Transactions on Neural Networks, 15, 1164–1175.

    Google Scholar 

  144. Pfaffelhuber, E. (1972). Learning and information theory. International Journal of Neuroscience, 3, 83.

    Google Scholar 

  145. Pfister, J.-P., & Gerstner, W. (2006). Triplets of spikes in a model of spike timing-dependent plasticity. The Journal of Neuroscience, 26(38), 9673–9682.

    Google Scholar 

  146. Prut, Y., Vaadia, E., Bergman, H., Haalman, I., Slovin, H., & Abeles, M. (1998). Spatiotemporal structure of cortical activity: Properties and behavioral relevance. Journal of Neurophysiology, 79(6), 2857–2874.

    Google Scholar 

  147. Quastler, H. (1956a). Information theory in psychology: Problems and methods. Glencoe: Free Press.

    Google Scholar 

  148. Quastler, H. (1956b). Studies of human channel capacity. In E. Cherry (Ed.), Information theory, 3rd London symposium (p. 361). London: Butterworths.

    Google Scholar 

  149. Rieke, F., Warland, D., de Ruyter van Steveninck, R., & Bialek, W. (1997). Spikes: Exploring the neural code. Cambridge: MIT Press.

    Google Scholar 

  150. Rolls, E. T., Treves, A., & Tovee, M. J. (1997). The representational capacity of the distributed encoding of information provided by populations of neurons in primate temporal visual cortex. Experimental Brain Research, 114(1), 149–162.

    Google Scholar 

  151. Schneideman, E., Bialek, W., & M. J. II. Berry (2003). Synergy, redundancy, and independence in population codes. Journal of Neuroscience, 23, 11539–11553.

    Google Scholar 

  152. Seriès, P., Latham, P., & Pouget, A. (2004). Tuning curve sharpening for orientation slectivity: Coding efficiency and the impact of correlations. Nature Neurosience, 7(10), 1129–1135.

    Google Scholar 

  153. Shadlen, M. N., & Newsome, W. T. (1994). Noise, neural codes and cortical organization. Current Opinion in Neurobiology, 4(4), 569–579.

    Google Scholar 

  154. Shadlen, M. N., & Newsome, W. T. (1998). The variable discharge of cortical neurons: Implications for connectivity, computation, and information coding. Journal of Neuroscience, 18(10), 3870–3896.

    Google Scholar 

  155. Shannon, C. E. (1948). A mathematical theory of communication. Bell Systems Technical Journal, 27, 379–423, 623–656.

    Google Scholar 

  156. Shaw, G., & Palm, G. (Eds.) (1988). Brain Theory Reprint Volume. Singapore: World Scientific.

    Google Scholar 

  157. Softky, W., & Koch, C. (1992). Cortical cells should fire regularly, but do not. Neural Computation, 4, 643–646.

    Google Scholar 

  158. Softky, W. R. (1995). Simple codes versus efficient codes. Current Opinion in Neurobiology, 5(2), 239–247.

    Google Scholar 

  159. Softky, W. R., & Koch, C. (1993). The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. Journal of Neuroscience, 13(1), 334–350.

    Google Scholar 

  160. Song, S., Miller, K. D., & Abbott, L. F. (2000). Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience, 3, 919–926.

    Google Scholar 

  161. Srinivasan, M. V., Laughlin, S. B., & Dubs, A. (1982). Predictive coding: A fresh view of inhibition in the retina. Proceedings of the Royal Society of London Series B; Biological Science, 216(1205), 427–459.

    Google Scholar 

  162. Stevens, C. F., & Zador, A. M. (1998). Input synchrony and the irregular firing of cortical neurons. Nature Neuroscience, 1(3), 210–217.

    Google Scholar 

  163. Tetko, I. V., & Villa, A. E. P. (1992). Fast combinitorial methods to estimate the probability of complex temporal patterns of spikes. Biological Cybernetics, 76, 397–407.

    Google Scholar 

  164. Thorpe, S. J., Guyonneau, R., Guilbaud, N., Allegraud, J.-M., & VanRullen, R. (2004). Spikenet: Real-time visual processing with one spike per neuron. Neurocomputing, 58–60, 857–864.

    Google Scholar 

  165. Tononi, G., Sporns, O., & Edelman, G. M. (1992). Reentry and the problem of integrating multiple cortical areas: Simulation of dynamic integration in the visual system. Cerebral Cortex, 2(4), 310–335.

    Google Scholar 

  166. Tononi, G., Sporns, O., & Edelman, G. M. (1994). A measure for brain complexity: Relating functional segregation and integration in the nervous system. Neurobiology, 91, 5033–5037.

    Google Scholar 

  167. Treves, A., & Panzeri, S. (1995). The upward bias in measures of information derived from limited data samples. Neural Computation, 7, 399–407.

    Google Scholar 

  168. Tsodyks, M., & Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter releaseprobability. Proceedings of the National Academy of Sciences of the United States of America, 94(2), 719–723.

    Google Scholar 

  169. Tsodyks, M., Uziel, A., & Markram, H. (2000). Synchrony generation in recurrent networks with frequency-dependent synapses. The Journal of Neuroscience, 20, 1–5.

    Google Scholar 

  170. Uttley, A. M. (1979). Information Transmission in the Nervous System. London: Academic.

    Google Scholar 

  171. Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., & Aertsen, A. M. H. J. (1995). Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, 373, 515–518.

    Google Scholar 

  172. van Essen, D. C., Olshausen, B., Anderson, C. H., & Gallant, J. L. (1991). Pattern recognition, attention and information bottlenecks in the primate visual system. Proceedings of SPIE Conference on Visual Information Processing: From Neurons to Chips, 1473, 17–27.

    Google Scholar 

  173. van Rossum, M. C. W., Bi, G. Q., & Turrigiano, G. G. (2000). Stable Hebbian learning from spike timing-dependent plasticity. The Journal of Neuroscience, 20, 8812–8821.

    Google Scholar 

  174. Wang, X., Hirsch, J. A., & Sommer, F. T. (2010). Recoding of sensory information across the retinothalamic synapse. The Journal of Neuroscience, 30, 13567–13577.

    Google Scholar 

  175. Wenzel, F. (1961). Über die Erkennungszeit beim Lesen. Biological Cybernetics, 1(1), 32–36.

    Google Scholar 

  176. Yang, H. H., & Amari, S. (1997). Adaptive online learning algorithms for blind separation: Maximum entropy and minimum mutual information. Neural Computation, 9, 1457–1482.

    Google Scholar 

  177. Yovits, M. C., Jacobi, G. T., & Goldstein, G. D. (Eds.) (1962). Self-organizing systems. Proceedings of the Conference on Self-Organizing Systems held on May 22, 23, and 24, 1962 in Chicago, Illinois. Washington: Spartan Books.

    Google Scholar 

  178. Zemel, R. S., & Hinton, G. E. (1995). Learning population codes by minimizing description length. Neural Computation, 7, 549–564.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Neural Information Processing, University of Ulm, James-Franck-Ring, Ulm, Germany

    Günther Palm

Authors
  1. Günther Palm
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Palm, G. (2012). Information, Novelty, and Surprise in Brain Theory. In: Novelty, Information and Surprise. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29075-6_12

Download citation

Publish with us

Policies and ethics

Search

Navigation