The Extended Theory of Cognitive Creativity pp 267-283 | Cite as

# Neuromodulation and Neural Circuit Performativity: Adequacy Conditions for Their Computational Modelling

## Abstract

An understanding of the functional repertoire of neural circuits and their plasticity requires knowledge of neural connectivity diagrams and their dynamical evolution. However, one must additionally take into account the *fast* and *reversible* functional effects induced by neuromodulatory mechanisms which do not alter neural circuit diagrams. Neuromodulators contribute crucially to determine the *performativity* of a neural circuit, that is, its ability to change behavior, and especially behavioral changes occurring under temporal constraints that are incompatible with the longer time scales of Hebbian learning and other forms of neural learning. This paper focuses on two properties of neuromodulatory action that have been relatively neglected so far. These properties are the *functional soundness* of neuromodulated circuits and the *robustness* of neuromodulatory action. Both properties are analyzed here as sources of functional specifications for the computational modeling of neural circuit performativity. In particular, taking dynamical systems that are based on CTRNNs (Continuous Time Recurrent Neural Networks) as an exemplary class of computational models, it is argued that robustness is suitably modeled there by means of a hysteresis process, and functional soundness by means of a multiplicity of stable fixed points.

## Keywords

Computational neurosciences Performativity of neural circuits Neuromodulation Computational models of neuromodulation Robustness of neuromodulatory action## Notes

### Acknowledgments

Research work on which this article is based was partially supported by grant 2015TM24JS in the framework of the PRIN 2015 program of MIUR (Italian Ministry of University, Research and Education).

## References

- Abbott, L.F. (1990). Modulation of function and gated learning in a network memory.
*Proceedings of the National Academy of Sciences of the United States of America, 87*(23), 9241–9245.CrossRefGoogle Scholar - Bargmann, C.I. (2012). Beyond the connectome: How neuromodulators shape neural circuits.
*BioEssays, 34*, 458–465.CrossRefGoogle Scholar - Beer, R.D. (1995). On the dynamics of small continuous-time recurrent neural networks.
*Adaptive Behavior, 3*, 469–509.CrossRefGoogle Scholar - Butera, R.J., Clark, J.W., Canavier, C.C., Baxter, D.A., & Byrne, J.H. (1995). Analysis of the effects of modulatory agents on a modeled bursting neuron: Dynamic interactions between voltage and calcium dependent systems.
*Journal of Computational Neuroscience, 2*(1), 19–44.CrossRefGoogle Scholar - Cox, B.R., & Krichmar, J.L. (2009). Neuromodulation as a robot controller: A brain-inspired strategy for controlling autonomous robots.
*IEEE Robotics and Automation Magazine, 16*, 1115–1129.CrossRefGoogle Scholar - Donnarumma, F., Prevete, R., & Trautteur, G. (2012). Programming in the brain: A neural network theoretical framework.
*Connection Science, 24*(2–3), 71–90.CrossRefGoogle Scholar - Donnarumma, F., Prevete, R., Chersi, F., & Pezzulo, G. (2015). A programmer–interpreter neural network architecture for prefrontal cognitive control.
*International Journal of Neural Systems, 25*(06), 1550017.CrossRefGoogle Scholar - Donnarumma, F., Prevete, R., de Giorgio, A., Montone, G., & Pezzulo, G. (2016). Learning programs is better than learning dynamics: A programmable neural network hierarchical architecture in a multi-task scenario.
*Adaptive Behavior, 24*(1), 27–51.CrossRefGoogle Scholar - Fellous, J.M., & Linster, C. (1998). Computational models of neuromodulation.
*Neural Computation, 10*(4), 771–805.CrossRefGoogle Scholar - Finnis, J.C., & Neal, M. (2016, August). UESMANN: A feed-forward network capable of learning multiple functions. In
*International conference on simulation of adaptive behavior*(pp. 101–112). Cham: Springer.Google Scholar - Flamm, R.E., & Harris-Warrick, R.M. (1986).
*Journal of Neurophysiology, 55*, 866–881.CrossRefGoogle Scholar - Goldman, M.S., Golowasch, J., Marder, E., & Abbott, L.F. (2001). Global structure, robustness, and modulation of neuronal models.
*The Journal of Neuroscience, 21*, 5229–5238.CrossRefGoogle Scholar - Hassani, V., Jahjowidodo, T.T., & Do, T.N. (2014). A survey on hysteresis modeling, identification and control.
*Mechanical Systems and Signal Processing, 49*, 209–233.CrossRefGoogle Scholar - Hooper, S.L., & Marder, E. (1987).
*The Journal of Neuroscience, 7*, 2097–2112.CrossRefGoogle Scholar - Husbands, P. (1998). Evolving robot behaviours with diffusing gas networks. In P. Husbands & J.-A. Meyer (Eds.),
*Evolutionary robotics: Proc. EvoRob’98, LNCS 1468*(pp. 71–86). Springer.Google Scholar - Ishiguro, A., Fujii, A., & Eggenberger, P. (2003). Neuromodulated control of bipedal locomotion using a polymorphic CPG circuit.
*Adaptive Behavior, 11*, 7–17.CrossRefGoogle Scholar - Kier, R.J., Ames, J.C., Beer, R.D., & Harrison, R.R. (2006). Design and implementation of multipattern generators in analog VLSI.
*IEEE Transactions on Neural Networks, 17*, 1025–1038.CrossRefGoogle Scholar - Kondo, T., Ishiguro, A., Tokura, S., Uchikawa, Y., & Eggenberger, P. (1999). Realization of robust controllers in evolutionary robotics: A dynamically-rearranging neural network approach. In
*Proceedings of the 1999 congress of evolutionary computation*(Vol. 1, pp. 366–373).Google Scholar - Krichmar, J.L. (2012, June). A biologically inspired action selection algorithm based on principles of neuromodulation. In
*Neural Networks (IJCNN), The 2012 International Joint Conference on*(pp. 1–8). IEEE.Google Scholar - Magg, S., & Philippides, A. (2006, September). GasNets and CTRNNs–a comparison in terms of evolvability. In
*International conference on simulation of adaptive behavior*(pp. 461–472). Berlin/Heidelberg: Springer.Google Scholar - Marder, E. (2012). Neuromodulation of neuronal circuits: Back to the future.
*Neuron, 76*, 1–11.CrossRefGoogle Scholar - Marder, E., O’Leary, T., & Shruti, S. (2014). Neuromodulation of circuits with variable parameters: Single neurons and small circuits reveal principles of state-dependent and robust neuromodulation.
*Annual Review of Neuroscience, 37*, 329–346.CrossRefGoogle Scholar - McCormick, D.A. (1989). Cholinergic and noradrenergic modulation of thalamocortical processing.
*Trends in Neuroscience, 12*(6), 215–221.CrossRefGoogle Scholar - McCormick, D.A., & Bal, T. (1997). Sleep and arousal: Thalamocortical mechanisms.
*Annual Review of Neuroscience, 20*(1), 185–215.CrossRefGoogle Scholar - McHale, G., & Husbands, P. (2004). GasNets and other evolvable neural networks applied to bipedal locomotion. In S. Schaal et al. (Eds.),
*From animals to Animats 8: Proceedings of the eighth international conference on simulation of adaptive behaviour (SAB’2004)*(pp. 163–172). MIT Press.Google Scholar - Meng, Y., Jin, Y., Yin, J., & Conforth, M. (2010). Human activity detection using spiking neural networks regulated by a gene regulatory network. In IEEE international joint conference on neural networks (IJCNN 2010).Google Scholar
- Montague, P.R., Dayan, P., & Sejnowski, T.J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning.
*The Journal of Neuroscience, 16*(5), 1936–1947.CrossRefGoogle Scholar - Nusbaum, M.P., & Marder, E. (1989).
*The Journal of Neuroscience, 9*, 1591–1599.CrossRefGoogle Scholar - Sterzer, P., Kleinschmidt, A., & Rees, G. (2009). The neural bases of multistable perception.
*Trends in Cognitive Sciences, 13*(7), 310–318.CrossRefGoogle Scholar - Swensen, A.M., & Marder, E. (2000).
*The Journal of Neuroscience, 20*, 6752–6759.CrossRefGoogle Scholar - Szücs, A., & Selverston, A.I. (2006). Consistent dynamics suggests tight regulation of biophysical parameters in a small network of bursting neurons.
*Journal of Neurobiology, 66*, 1584–1601.CrossRefGoogle Scholar - Taylor, A.L., Goaillard, J.M., & Marder, E. (2009). How multiple conductances determine electrophysiological properties in a multicompartment model.
*The Journal of Neuroscience, 29*, 5573–5586.CrossRefGoogle Scholar - Weimann, J.M., Skiebe, P., Heinzel, H.G., Soto, C., Kopell, N., et al. (1997). Modulation of oscillator interactions in the crab stomatogastric ganglion by crustacean cardioactive peptide.
*The Journal of Neuroscience, 17*, 1748–1760.CrossRefGoogle Scholar - Williams, A.H., Calkins, A., O’Leary, T., Symonds, R., Marder, E., & Dickinson, P.S. (2013). The neuromuscular transform of the lobster cardiac system explains the opposing effects of a neuromodulator on muscle output.
*The Journal of Neuroscience, 33*, 16565–16575.CrossRefGoogle Scholar - Zhang, L., & Suganthan, P.N. (2016). A survey of randomized algorithms for training neural networks.
*Information Sciences, 364*, 146–155.Google Scholar - Ziemke, T. (1999). Remembering how to behave: Recurrent neural networks for adaptive robot behavior. In L.R. Medsker & L.C. Jain (Eds.),
*Recurrent neural networks: Design and applications*(pp. 359–389). Boca Raton: CRC Press.Google Scholar - Ziemke, T., & Thieme, M. (2002). Neuromodulation of reactive sensorimotor mappings as a short-term memory mechanism in delayed response tasks.
*Adaptive Behavior, 10*, 185–198.CrossRefGoogle Scholar