# Introducing double bouquet cells into a modular cortical associative memory model

- 91 Downloads

## Abstract

We present an electrophysiological model of double bouquet cells and integrate them into an established cortical columnar microcircuit model that has previously been used as a spiking attractor model for memory. Learning in that model relies on a Hebbian-Bayesian learning rule to condition recurrent connectivity between pyramidal cells. We here demonstrate that the inclusion of a biophysically plausible double bouquet cell model can solve earlier concerns about learning rules that simultaneously learn excitation and inhibition and might thus violate Dale’s principle. We show that learning ability and resulting effective connectivity between functional columns of previous network models is preserved when pyramidal synapses onto double bouquet cells are plastic under the same Hebbian-Bayesian learning rule. The proposed architecture draws on experimental evidence on double bouquet cells and effectively solves the problem of duplexed learning of inhibition and excitation by replacing recurrent inhibition between pyramidal cells in functional columns of different stimulus selectivity with a plastic disynaptic pathway. We thus show that the resulting change to the microcircuit architecture improves the model’s biological plausibility without otherwise impacting the model’s spiking activity, basic operation, and learning abilities.

## Keywords

BCPNN learning rule Cortical microcircuit Disynaptic inhibition Double bouquet cells Electrophysiological modeling Hebbian plasticity## 1 Introduction

We examine and build on a cortical microcircuit model, previously used in a working memory model by Fiebig and Lansner (2017) that implemented a BCPNN (Bayesian Confidence Propagation Neural Network) learning rule. We then expand on this functional columnar architecture by integrating GABAergic double bouquet cells (DBCs), which may play a key modulatory role in the cortical microcircuit (Krimer et al. 2005; Kelsom and Lu 2013). Generally speaking, the BCPNN learning rule processes spike trains of pre- and postsynaptic neurons, and computes synaptic traces of activation and coactivation, which are then used to calculate the updated weights (see Section 2.3). In other words, the BCPNN is based on spike train correlations (Tully et al. 2014).

The foremost points of this work are first, the electrophysiological modeling of the inhibitory DBCs, and secondly their integration with the previous cortical memory model and its learning rule, which yields a novel model with improved biological plausibility and maintained functionality. The previous implementation suffers from the problem that learned weights among excitatory pyramidal cells in competing functional columns become negative (inhibitory), thus violating Dale’s Principle which states that neurons release the same neurotransmitters at all of their synapses (Strata and Harvey 1999). The biological plausibility of our model is here improved by the introduction of DBCs that provide disynaptic inhibition among pyramidal cells (Silberberg and Markram 2007).

## 2 Materials and methods

### 2.1 Neuron models

*V*

_{m}and the adaptation current

*I*

_{w}, given by the following equations:

*V*

_{m}represents cell membrane potential,

*I*

_{w}stands for the adaptation current,

*C*

_{m}is the membrane capacitance,

*g*

_{L}is the leak conductance, E

_{L}is the leak reversal potential, V

_{t}is the spiking threshold,

*Δ*

_{τ}is the spike slope factor, b is the spike-triggered adaptation,

*τ*

_{w}is the adaptation recovery time constant and

*I*

_{ext}is the stimulation current.

Adaptation enriches neural dynamics particularly in pyramidal cells (Brette and Gerstner 2005), and we take advantage of this in modeling DBCs as well. As in the model we extend, the neuron model was somewhat simplified by excluding the subthreshold adaptation. AdEx models provide a phenomenological description of the neural behaviour, yet feature limitations in predicting the precise time course of the voltage during and after a spike or the underlying biophysical causes of electrical activity (Gerstner and Naud 2009).

### 2.2 Synapse model

*I*

_{ext}, neurons receive synaptic currents \(I_{\text {syn}_{j}}\) from AMPA and GABA synapses summed at the membrane. The model features conductance based AMPA (reversal potential

*E*

^{AMPA}) and GABA (reversal potential E

^{GABA}) synapses:

### 2.3 BCPNN learning rule

Excitatory AMPA weights develop according to the BCPNN learning rule (Lansner and Ekeberg 1989; Sandberg et al. 2002; Tully et al. 2014). This is a Hebbian type of learning rule used in many previous works, most recently in Fiebig and Lansner (2017). It was derived from Bayes rule, assuming that a postsynaptic neuron employs some form of probabilistic inference to decide whether to emit a spike or not. It is a more complex learning rule than the more standard spike-timing-dependent plasticity (STDP) learning rule (Caporale and Dan 2008), but can replicate the main features of such plasticity. As other spiking synaptic learning rules, it is so far insufficiently validated against quantitative experimental data on biological synaptic plasticity.

A BCPNN synapse calculates three synaptic memory traces, *P*_{i}, *P*_{j} and *P*_{ij}, implemented as exponentially weighted moving averages of pre-, post- and co-activation. As old memories deteriorate they are gradually replaced by newly learned patterns, so exponentially moving averages prioritize recent patterns. Specifically, BCPNN implements a three-stage procedure of exponential filters which defines Z, E and P traces. The method then estimates *P*_{i} (normalized presynaptic firing rate), *P*_{j} (normalized postsynaptic firing rate) and also *P*_{ij} (coactivation) from these traces. In the final stage, *P*_{i}, *P*_{j} and *P*_{ij} update the Bayesian weights *w*_{ij} and biases *β*_{j}. It is worth adding that E traces that enable delayed reward learning, are not used here because such conditions are not applicable. Some of the key equations are highlighted in this chapter; yet for further information and deeper understanding of the BCPNN learning rule, see Tully et al. (2014).

*S*

_{i},

*S*

_{j}) so as to calculate the traces

*Z*

_{i}and

*Z*

_{j}:

*𝜖*is the lowest attainable probability estimate,

*t*

_{spike}denotes the spike duration while \(\tau _{\mathrm {z_{\mathrm {i}}}}=\tau _{\mathrm {z_{\mathrm {j}}}}\) are the pre- and postsynaptic time constants respectively (here 5

*ms*).

*κ*adjusts learning speed, and by setting

*κ*= 0 there are no weight changes. To give prominence to the stability of memory networks with BCPNN learning rule, we set

*κ*= 1 during the whole simulation.

*P*

_{i},

*P*

_{j}and

*P*

_{ij}are used to calculate intrinsic excitability

*β*

_{j}and synaptic weights

*w*

_{ij}:

### 2.4 Columnar network architecture

### 2.5 Simulation tools

We use NEST (Neural Simulation Tool) version 2.4.2, and a custom-built BCPNN learning rule module (Tully et al. 2014). NEST simulates the dynamics of spiking neural models and features a convenient Python interface (PyNEST) to NEST’s simulation kernel (Gewaltig and Diesmann 2007). Simulation code and the custom module is available online in ModelDB (McDougal et al. 2017) at http://modeldb.yale.edu/257610.

## 3 Results

### 3.1 Double bouquet cells

Modeling DBCs is a key contribution of this work since the suggested cortical microcircuit model learns disynaptic inhibition through them and thereby modulates the neural activity of neurons in competing MCs. DBCs are GABAergic interneurons which may play an important role in shaping neural activity and circuitry (Krimer et al. 2005; Kelsom and Lu 2013) and are mainly located in Layer II/III featuring a bitufted dendritic conformation (Markram et al. 2004). They contact the dendrites of targeted cells (Markram et al. 2004) innervating spines (69.2*%* ± 4.2*%*) and shafts (30.8*%* ± 4.2*%*) (Tamas et al. 1997). DBCs are characterized by vertically oriented descending axons (María and DeFelipe 1995), which are generally termed “bundles” or “horse-tails” (Yáñez et al. 2005; DeFelipe et al. 1989).

The majority of double bouquet cells appear to be situated in upper layers (DeFelipe et al. 1989) and one of their unique feature is a horse-tail that fit well within the minicolumn (vertical cyclinder of tissue with a diameter of roughly 25 − 50 *μ**m*). Due to this morphology they create strong connections with pyramidal cells within their local column (DeFelipe et al. 2006). Neuroanatomical data suggests that each minicolumn contains one DBC (DeFelipe et al. 2006).

DBCs present a small degree of adaptation (Tamas et al. 1997) and unlike Basket Cells, they are characterized by sustained spiking activity (Krimer et al. 2005), which under strong stimulation is at 43 ± 13 *H**z* (Zaitsev et al. 2008). DBCs are intermediate spiking (IS) cells (Krimer et al. 2005) and subclassified as RSNP cells (Regular-spiking nonpyramical neurons) according to studies of their physiological properties (Kawaguchi and Kubota 1996; 1997).

*I*

*n*

*vitro*experiments report DBC resting potential at − 76 ± 6

*m*

*V*, spiking threshold

*V*

_{th}at − 44 ± 8

*m*

*V*and input resistance at 626 ± 312

*M*

*Ω*. They are considered inhibitory neurons with high input resistance (Zaitsev et al. 2008) and low capacitance (see Table 1). In addition, they exhibit membrane time constants (17.1 ± 7.7

*ms*), slope factor (− 0.64 ± 2.22) and action potential amplitude (66 ± 12.4

*m*

*V*) (Krimer et al. 2005).

DBC model parameters

Parameters | Symbol | Value |
---|---|---|

Adaptation current | b | 3 |

Adaptation time constant | | 200 |

Membrane capacitance | | 15 |

Leak reversal potential | | − 76 |

Leak conductance | | 1.52 |

Upstroke slope factor | | 1 |

Spike threshold | | − 44 |

Spike reset potential | | − 60 |

Refractory period | | 2 |

Input Resistance | | 660 |

We align the simulation model for DBCs with biological findings, yet tune some factors such as adaptation (*b*), leak conductance (*g*_{L}), slope factor (*Δ*_{t}) and refractory period (*τ*_{ref}) to achieve satisfactory electrophysiological fidelity, reproducing spike patterns under sweeps of increasing suprathreshold current steps and other typically reported activity. Figure 1c displays the membrane voltage of a stimulated DBC. The resulting model parameters are broadly consistent with experimentally reported values (see Table 1).

### 3.2 Adding DBCs to the columnar architecture

In the new model, connections among pyramidal cells in competing MCs are now mediated by DBCs as an additional local microcircuit component (see Fig. 1a). Their functional role is to deliver the same amount of inhibition to the respective MCs as the previous model but now entirely disynaptically, without principally changing established network learning and neural dynamics.

This extended cortical microcircuit model now contains three classes of neurons; pyramidal cells, basket cells and DBCs (see Fig. 1a). We use parameters for pyramidal and basket cells from a previous model implementation (Fiebig and Lansner 2017) and derive parameters for DBCs through electrophysiological modeling and tuning based on reported *in**vitro* characterizations (see Section 3.1). We simulate a small network of two reduced HCs from the previous larger network model.

### 3.3 BCPNN plasticity

Stimulation of the columnar network changes the efficacy of the plastic BCPNN synapses. To show the learned connectivity, we read out connection weights after an one second long initialization with zero mean noise (Initial Weight Distribution, IWD) and finally, after learning of the two stimuli (Learned Weight Distribution, LWD), see Fig. 1b.

Two important changes can be identified in Fig. 2c and d, wherein disynaptic inhibition is introduced by DBCs. Even though the strength of the learned connections onto DBCs is weak, DBCs also feature low capacitance and dense connections with local pyramidal cells, and thus deliver comparable inhibition (see Section 3.4).

This result shows the effectiveness of DBCs involvement in the microcircuit network as they learn to mediate disynaptic inhibition between pyramidal cells in competing MCs. This outcome looks promising, but we yet have to verify its functional efficacy with regards to the total inhibition delivered (see Section 3.4).

### 3.4 Functionality verification

The cortical model learns as expected and the competing MCs inhibit each other by disynaptic inhibition mediated by DBCs and basket cells. But is this inhibition equivalent to the mono-synaptic inhibition learned by the previous model?

We tested learning in both the new and previous model using the same stimulation pattern and recorded the total inhibitory input current received by pyramidal cells in MC0 (see Fig. 3b). The proposed cortical model effectively delivers the same amount of disynaptic inhibition via basket cells and DBCs. The new model has thus improved biological credibility while maintaining the same functionality.

## 4 Discussion

This work aims at giving prominence to the double bouquet cells and their use as an integral part of a cortical microcircuit model. The population of DBCs is limited compared to other GABAergic neurons; however, they may play a key role in shaping neural activity. By integrating them into an established model, the new model now obeys Dale’s principle with maintained function. Indirectly, this result also verifies the biological plausibility of recent network models (Lansner 2009; Tully et al. 2016; Fiebig and Lansner 2017). The newly integrated DBCs effectively learn to mediate disynaptic inhibition between pyramidal cells thus eliminating negative learned weights between pyramidal cells which violate Dale’s principle.

In conclusion, the successful integration of an electrophysiological DBC model into an established cortical microcircuit design yields a novel functionally equivalent learning network with improved biological plausibility. This model suggests that DBCs have a quite well defined role in cortical memory networks.

## Notes

### Acknowledgements

Funding from T.E.I of Athens, Erasmus Placement [NC], EuroSPIN Erasmus Mundus Doctoral Programme [FF], and Swedish E-Science Research Center (SeRC) [AL] is gratefully acknowledged.

### Funding Information

Open access funding provided by Stockholm University.

### Compliance with Ethical Standards

### **Conflict of interests**

The authors declare that they have no conflict of interest.

## References

- Binzegger, T., Douglas, R., Martin, K. (2009). Topology and dynamics of the canonical circuit of cat v1.
*Neural Networks*,*22*(8), 1071–1078. Cortical Microcircuits.CrossRefGoogle Scholar - Brette, R., & Gerstner, W. (2005). Adaptive exponential integrate-and-fire model as an effective description of neuronal activity.
*Journal of Neurophysiology*,*94*(5), 3637–3642. PMID: 16014787.CrossRefGoogle Scholar - Caporale, N., & Dan, Y. (2008). Spike timing–dependent plasticity: a Hebbian learning rule.
*Annual Review of Neuroscience*,*31*, 25–46.*International Journal of Neural Systems*, 01(01):77–87.CrossRefGoogle Scholar - DeFelipe, J., Ballesteros-Yáñez, I., Inda, M.C., Muñoz, A. (2006). Double-bouquet cells in the monkey and human cerebral cortex with special reference to areas 17 and 18.
*Progress in Brain Research*,*154*, 15–32.CrossRefGoogle Scholar - DeFelipe, J., Hendry, S., Jones, E. (1989). Synapses of double bouquet cells in monkey cerebral cortex visualized by Calbindin immunoreactivity.
*Brain Research*,*503*(1), 49–54.CrossRefGoogle Scholar - Fiebig, F., & Lansner, A. (2017). A spiking working memory model based on Hebbian short-term potentiation.
*Journal of Neuroscience*,*37*(1), 83–96.CrossRefGoogle Scholar - Gerstner, W., & Naud, R. (2009). How good are neuron models?
*Science*,*326*(5951), 379–380.CrossRefGoogle Scholar - Gewaltig, M.-O., & Diesmann, M. (2007). Nest (neural simulation tool).
*Scholarpedia*,*2*(4), 1430.CrossRefGoogle Scholar - Kawaguchi, Y., & Kubota, Y. (1996). Physiological and morphological identification of somatostatin-or vasoactive intestinal polypeptide-containing cells among GABAergic cell subtypes in rat frontal cortex.
*Journal of Neuroscience*,*16*(8), 2701–2715.CrossRefGoogle Scholar - Kawaguchi, Y., & Kubota, Y. (1997). GABAergic cell subtypes and their synaptic connections in rat frontal cortex.
*Cerebral Cortex*,*7*(6), 476–86.CrossRefGoogle Scholar - Kelsom, C., & Lu, W. (2013). Development and specification of GABAergic cortical interneurons.
*Cell & Bioscience*,*3*(1), 19.CrossRefGoogle Scholar - Krimer, L.S., Zaitsev, A.V., Czanner, G., Kroner, S., González-Burgos, G., Povysheva, N.V., Iyengar, S., Barrionuevo, G., Lewis, D.A. (2005). Cluster analysis–based physiological classification and morphological properties of inhibitory neurons in layers 2–3 of monkey dorsolateral prefrontal cortex.
*Journal of Neurophysiology*,*94*(5), 3009–3022.CrossRefGoogle Scholar - Lansner, A. (2009). Associative memory models: from the cell-assembly theory to biophysically detailed cortex simulations.
*Trends in Neurosciences*,*32*(3), 178–186.CrossRefGoogle Scholar - Lansner, A., & Ekeberg, Ã. (1989). A one-layer feedback artificial neural network with a Bayesian learning rule.Google Scholar
- María, R., & DeFelipe, J. (1995). A light and electron microscopic study of calbindin d-28k immunoreactive double bouquet cells in the human temporal cortex.
*Brain Research*,*690*(1), 133– 140.CrossRefGoogle Scholar - Markram, H., Toledo-Rodriguez, M., Wang, Y., Gupta, A., Silberberg, G., Wu, C. (2004). Interneurons of the neocortical inhibitory system.
*Nature Reviews Neuroscience*,*5*(10), 793–807.CrossRefGoogle Scholar - McDougal, R.A., Morse, T.M., Carnevale, T., Marenco, L., Wang, R., Migliore, M., Miller, P.L., Shepherd, G.M., Hines, M.L. (2017). Twenty years of modeldb and beyond: building essential modeling tools for the future of neuroscience.
*Journal of Computational Neuroscience*,*42*(1), 1–10.CrossRefGoogle Scholar - Muir, D.R., Da Costa, N.M., Girardin, C.C., Naaman, S., Omer, D.B., Ruesch, E., Grinvald, A., Douglas, R.J. (2011). Embedding of cortical representations by the superficial patch system.
*Cerebral Cortex*,*21*(10), 2244–2260.CrossRefGoogle Scholar - Sandberg, A., Lansner, A., Petersson, K.M. (2002). Ekeberg a Bayesian attractor network with incremental learning.
*Network: Computation in Neural Systems*,*13*(2), 179–194.CrossRefGoogle Scholar - Silberberg, G., & Markram, H. (2007). Disynaptic inhibition between neocortical pyramidal cells mediated by Martinotti cells.
*Neuron*,*53*(5), 735–746.CrossRefGoogle Scholar - Stettler, D.D., Das, A., Bennett, J., Gilbert, C.D. (2002). Lateral connectivity and contextual interactions in macaque primary visual cortex.
*Neuron*,*36*(4), 739–750.CrossRefGoogle Scholar - Strata, P., & Harvey, R. (1999). Dale’s principle.
*Brain Research Bulletin*,*50*(5-6), 349–350.CrossRefGoogle Scholar - Tamas, G., Buhl, E., Somogyi, P. (1997). Fast ipsps elicited via multiple synaptic release sites by different types of GABAergic neurone in the cat visual cortex.
*The Journal of Physiology*,*500*(3), 715–738.CrossRefGoogle Scholar - Thomson, A.M., West, D.C., Wang, Y., Bannister, A.P. (2002). Synaptic connections and small circuits involving excitatory and inhibitory neurons in layers 2–5 of adult rat and cat neocortex: triple intracellular recordings and biocytin labelling
*in vitro*.*Cerebral Cortex*,*12*(9), 936–953.CrossRefGoogle Scholar - Tsodyks, M.V., & Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability.
*Proceedings of the National Academy of Sciences*,*94*(2), 719–723.CrossRefGoogle Scholar - Tully, P.J., Hennig, M.H., Lansner, A. (2014). Synaptic and nonsynaptic plasticity approximating probabilistic inference.
*Frontiers in Synaptic Neuroscience*, (6):8.Google Scholar - Tully, P.J., Lindén, H., Hennig, M.H., Lansner, A. (2016). Spike-based Bayesian-Hebbian learning of temporal sequences.
*PLoS Computational Biology*,*12*(5), 1–35.CrossRefGoogle Scholar - Yáñez, I.B., Muñoz, A., Contreras, J., Gonzalez, J., Rodríguez-Veiga, E., DeFelipe, J. (2005). Double bouquet cell in the human cerebral cortex and a comparison with other mammals.
*The Journal of Comparative Neurology*,*486*(4), 344– 360.CrossRefGoogle Scholar - Yoshimura, Y., & Callaway, E.M. (2005). Fine-scale specificity of cortical networks depends on inhibitory cell type and connectivity.
*Nature Neuroscience*,*8*(11), 1552.CrossRefGoogle Scholar - Zaitsev, A.V., Povysheva, N.V., Gonzalez-Burgos, G., Rotaru, D., Fish, K.N., Krimer, L.S., Lewis, D.A. (2008). Interneuron diversity in layers 2–3 of monkey prefrontal cortex.
*Cerebral Cortex*,*19*(7), 1597–1615.CrossRefGoogle Scholar

## Copyright information

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.