Abstract
In the field of computational neuroscience, spiking neural network models are generally preferred over rate-based models due to their ability to model biological dynamics. Within AI, rate-based artificial neural networks have seen success in a wide variety of applications. In simplistic spiking models, information between neurons is transferred through discrete spikes, while rate-based neurons transfer information through continuous firing-rates. Here, we argue that while the spiking neuron model, when viewed in isolation, may be more biophysically accurate than rate-based models, the roles reverse when we also consider information transfer between neurons. In particular we consider the biological importance of continuous synaptic signals. We show how synaptic conductance relates to the common rate-based model, and how this relation elevates these models in terms of their biological soundness. We shall see how this is a logical relation by investigating mechanisms known to be present in biological synapses. We coin the term ‘conductance-outputting neurons’ to differentiate this alternative view from the standard firing-rate perspective. Finally, we discuss how this fresh view of rate-based models can open for further neuro-AI collaboration.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Partly due to this not being the main priority of these models.
- 2.
With rate-based models being the 2nd generation and threshold perceptrons being the 1st.
- 3.
Apart from a few special versions e.g. continuous ANNs.
- 4.
Although several successful but less biologically motivated activation functions have come about in recent years [20].
- 5.
We have simplified here by disregarding depletion of neurotransmitters: i.e. we assume that neurotransmitters re-uptake is able to keep up with the pace of release.
References
Aaser, P., et al.: Towards making a cyborg: a closed-loop reservoir-neuro system. In: Proceedings of the 14th European Conference on Artificial Life ECAL 2017, pp. 430–437. MIT Press, Cambridge (2017). https://doi.org/10.7551/ecal_a_072
Attneave, F., B., M., Hebb, D.O.: The organization of behavior: a neuropsychological theory. Am. J. Psychol. 63(4), 633 (2006). https://doi.org/10.2307/1418888
Brette, R.: Philosophy of the spike: rate-based vs. spike-based theories of the brain. Front. Syst. Neurosci. 9, 151 (2015). https://doi.org/10.3389/fnsys.2015.00151
Buchanan, K.A., Mellor, J.: The activity requirements for spike timing-dependent plasticity in the hippocampus. Front. Synaptic Neurosci. 2, 11 (2010). https://doi.org/10.3389/fnsyn.2010.00011
Burkitt, A.N.: A review of the integrate-and-fire neuron model: I. homogeneous synaptic input. Biol. Cybern. 95(1), 1–19 (2006). https://doi.org/10.1007/s00422-006-0068-6
Clopath, C., Gerstner, W.: Voltage and spike timing interact in STDP - a unified model. Front. Synaptic Neurosci. 2, 25 (2010). https://doi.org/10.3389/fnsyn.2010.00025
Diehl, P.U., Cook, M.: Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Front. Comput. Neurosci. 9, 99 (2015). https://doi.org/10.3389/fncom.2015.00099
Gerstner, W., Kreiter, A.K., Markram, H., Herz, A.V.: Neural codes: firing rates and beyond. Proc. Nat. Acad. Sci. U.S.A. 94(24), 12740–1 (1997). https://doi.org/10.1073/PNAS.94.24.12740
Honoré, T., Lauridsen, J., Krogsgaard-Larsen, P.: The binding of [3H]AMPA, a structural analogue of glutamic acid, to rat brain membranes. J. Neurochem. (1982). https://doi.org/10.1111/j.1471-4159.1982.tb10868.x
Izhikevich, E.M.: Which model to use for cortical spiking neurons? IEEE Trans. Neural Netw. 15(5), 1063–1070 (2004). https://doi.org/10.1109/TNN.2004.832719
de Kamps, M., van der Velde, F.: From artificial neural networks to spiking neuron populations and back again. Neural Netw. 14(6–7), 941–953 (2001). https://doi.org/10.1016/S0893-6080(01)00068-5
Kandel, E.R., Schwartz, J.H., Jessell, T.M.: Principles of Neural Science, vol. 4. McGraw-Hill Education, New York (2013). https://doi.org/10.1036/0838577016
Kheradpisheh, S.R., Ganjtabesh, M., Masquelier, T.: Bio-inspired unsupervised learning of visual features leads to robust invariant object recognition. Neurocomputing 205, 382–392 (2016). https://doi.org/10.1016/j.neucom.2016.04.029
Kheradpisheh, S.R., Ghodrati, M., Ganjtabesh, M., Masquelier, T.: Deep network scan resemble human feed-forward vision in invariant object recognition. Sci. Rep. 6 (2016). https://doi.org/10.1038/srep32672
Maass, W.: Networks of spiking neurons: the third generation of neural network models. Neural Netw. 10(9), 1659–1671 (1997). https://doi.org/10.1016/S0893-6080(97)00011-7
Mainen, Z.F., Seinowski, T.J.: Reliability of spike timing in neocortical neurons. Science (1995). https://doi.org/10.1126/science.7770778
Markram, H., Gerstner, W., Sjøstrøm, P.J.: Spike-timing-dependent plasticity: a comprehensive overview. Front. Res. Topics 4, 2010–2012 (2012). https://doi.org/10.3389/fnsyn.2012.00002
Medium: Google brain’s co-inventor tells why he’s building Chinese neural networks: Andrew Ng on the state of deep learning at Baidu. Medium (2015)
Meldrum, B.S.: Glutamate as a neurotransmitter in the brain: review of physiology and pathology. J. Nutr. 130, 1007S-15S (2000). 10736372
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (2010). https://doi.org/10.1.1.165.6419
Numenta. https://numenta.com/
Rumelhart, D.E., Widrow, B., Lehr, M.A.: The basic ideas in neural networks. Commun. ACM (1994). https://doi.org/10.1145/175247.175256
Shouval, H.Z., Wang, S.S.H., Wittenberg, G.M.: Spike timing dependent plasticity: a consequence of more fundamental learning rules. Front. Comput. Neurosci. 4, 1–13 (2010). https://doi.org/10.3389/fncom.2010.00019
Sompolinsky, H.: Computational neuroscience: beyond the local circuit. Current Opinion Neurobiol. 25, xiii–xviii (2014). https://doi.org/10.1016/J.CONB.2014.02.002
Song, S., Miller, K.D., Abbott, L.F.: Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neurosci. 3(9), 919–926 (2000). https://doi.org/10.1038/78829
Sterratt, D., Graham, B., Gillies, A., Willshaw, D.: Principles of Computational Modelling in Neuroscience. Cambridge University Press (2011). https://doi.org/10.1109/MPUL.2012.2196841
Wolfram, S.: Cellular automata as models of complexity. Nature 311(5985), 419–424 (1984). https://doi.org/10.1038/311419a0
Wurtz, R.H.: Visual receptive fields of striate cortex neurons in awake monkeys. J. Neurophysiol. (1969). https://doi.org/10.1152/jn.1969.32.5.727
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Knudsen, M., Hendseth, S., Tufte, G., Sandvig, A. (2019). Viewing Rate-Based Neurons as Biophysical Conductance Outputting Models. In: McQuillan, I., Seki, S. (eds) Unconventional Computation and Natural Computation. UCNC 2019. Lecture Notes in Computer Science(), vol 11493. Springer, Cham. https://doi.org/10.1007/978-3-030-19311-9_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-19311-9_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-19310-2
Online ISBN: 978-3-030-19311-9
eBook Packages: Computer ScienceComputer Science (R0)