Neuromorphic Synapses for Artificial Dendrites

  • Wayne C. Westerman
  • David P. M. Northmore
  • John. G. Elias
Part of the The Springer International Series in Engineering and Computer Science book series (SECS, volume 447)


In the past few years several researchers have developed general-purpose spiking silicon neurons, or neuromorphs, which attempt to capture various functional behaviors of biological neurons [19, 23, 24]. These CMOS analog VLSI implementations are intended to be the computational elements linking sensory input to motor output in artificial nervous systems. Systems consisting of thousands or millions of neuromorphs may be required to realize certain basic behaviors. However, until appropriate methods are found to specify and adapt network parameters (e.g., connectivity, synapse weights, dynamics, morphology), these systems will remain beyond our grasp. Though various Hebbian or correlative adaptation rules have been applied to perceptron based networks (e.g., [77, 11, 141]) and single-compartment model neurons [12, 26], these rules are virtually untested with multi-compartment spiking neuromorphs that model the spatially extensive dendrites found in biological systems. This paper discusses the VLSI design and performance of just one of the adaptive network parameters: variable synaptic conductances that mimic the fast chemical synapses found on dendrites throughout high-order nervous systems.


Synaptic Weight Hebbian Learning Synaptic Conductance Analog VLSI Synapse Activation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    C. Acar and M. S. Ghausi. Fully integrated active-rc filters using MOS and non-balanced structure. Int. J. of Circuit Theory and Applications, 15:105–121, 1987.CrossRefGoogle Scholar
  2. [2]
    C. Allen and C. F. Stevens. An evaluation of causes for unreliability of synapt ic transmission. In Proc. Natl. Acad. Sci. USA, volume 91, pages 10380–10383, 1994.CrossRefGoogle Scholar
  3. [3]
    P. O. Anderson. Properties of hippocampal synapses of importance for integration and memory. In G. M. Edelman, W. E. Gall, and W. M. Cowan, editors, Synaptic Function, pages 403–430. John Wiley & Sons, 1987.Google Scholar
  4. [4]
    Y. Arima, M. Murasaki, T. Yamada, A. Maeda, and H. Shinohara. A refreshable analog VLSI neural network chip with 400 neurons and 40k synapses. IEEE J. of Solid State Circuits, 27:1854–1861, 1992.CrossRefGoogle Scholar
  5. [5]
    M. Banu and Y. Tsividis. Floating voltage-controlled resistors in CMOS technology. Electron. Lett., 18:678–679, 1982.CrossRefGoogle Scholar
  6. [6]
    J. N. Barrett and W. E. Crill. Specific membrane properties of cat motoneurones. J. of Physiol., 239:301–324, 74.Google Scholar
  7. [7]
    M. Bawdry and J. L. Davis. Long-Term Potentiation, volume 2. The MIT Press, Cambridge, MA, 1994.Google Scholar
  8. [8]
    J. M. Bekkers. Quantal analysis of synaptic transmission in the central nervous system. Current Opinion in Neurobiology, 4:360–365, 1994.CrossRefGoogle Scholar
  9. [9]
    T. V. P. Bliss and G. L. Collingridge. A synaptic model of memory: Long-term potentiation in the hippocampus. Nature, 361:31–39, 1993.CrossRefGoogle Scholar
  10. [10]
    K. A. Boahen and A. G. Andreou. A contrast sensitive silicon retina with reciprocal synapses. Advances in Neural Information Processing Systems, 4:764–772, 1992.Google Scholar
  11. [11]
    T. H. Brown, E. W. Kairiss, and C. L. Keenan. Hebbian synapses: Biophysical mechanisms and algorithms. Annu. Rev. of Neurosci., 13:475–511, 1990.CrossRefGoogle Scholar
  12. [12]
    D. V. Buonomano and M. M. Merzenich. Temporal information transformed into a spatial code by a neural network with realistic properties. Science, 267:1028–1030, 1995.CrossRefGoogle Scholar
  13. [13]
    H. A. Castro, S. M. Tam, and M. A. Holler. Implementation and preformance of an analog nonvolatile neural network. Analog Integrated Circuits and Signal Processing, 4:97–113, 1993.CrossRefGoogle Scholar
  14. [14]
    J. D. Clements and S. J. Redman. Cable properties of cat spinal motoneurons measured by combining voltage clamp, current clamp and intracellular staining. J. of Physiol., 409:63–87, 1989.Google Scholar
  15. [15]
    M. H. Cohen and A. G. Andreou. Current-node subthreshold MOS implementation of the herault-jutten autoadaptive network. IEEE J. of Solid State Circuits, 27:714–727, 1992.CrossRefGoogle Scholar
  16. [16]
    G. L. Collingridge and T. V. P. Bliss. Memories of nmda receptors and ltp. Trends in Neurosci., 18:54–56, 1995.CrossRefGoogle Scholar
  17. [17]
    T. Delbrück. Silicon retina with correlation-based velocity-tuned pixels. IEEE Transactions on Neural Networks, 4(3):529–541, May 1993.CrossRefGoogle Scholar
  18. [18]
    J. Van der Spiegel, P. Mueller, D. Blackman, P. Chance, C. Donham, R. Etienne-Cummings, and P. Kinget. An analog neural computer with modular architecture for real-time dynamic computations. IEEE J. of Solid-State Circuits, 27:82–92, 1992.CrossRefGoogle Scholar
  19. [19]
    R. Douglas and M. Mahowald. A constructor set for silicon neurons. In S. F. Zornetzer, J. Davis, and T. McKenna, editors, An Introduction to Neural and Electronic Networks. Academic Press, 1994.Google Scholar
  20. [20]
    S. Eberhardt, T. Duong, and A. Thakoor. Design of parallel hardware neural network systems from custom analog VLSI ‘building blocks’ chips. In Int. Jt. Conf. Neural Network, volume 2, pages 183–190, 1988.Google Scholar
  21. [21]
    G. M. Edelman, W. E. Gall, and W. M. Cowan. Synaptic Function. John Wiley & Sons, 1987.Google Scholar
  22. [22]
    F. A. Edwards. Ltp-a structural model to explain the inconsistencies. Trends in Neurosci., 18:250–255, 1995.CrossRefGoogle Scholar
  23. [23]
    J. G. Elias. Artificial dendritic trees. Neural Computation, 5:648–663, 1993.Google Scholar
  24. [24]
    J. G. Elias and D. P. M. Northmore. Switched-capacitor neuromorphs with wide-range variable dynamics. IEEE Trans. on Neural Networks, 6:1542–1548, 1995.CrossRefGoogle Scholar
  25. [25]
    J. G. Elias, D. P. M. Northmore, and W. Westerman. An analog memory device for spiking silicon neurons. Neural Computation, 9:419–440, 1997.CrossRefGoogle Scholar
  26. [26]
    W. Gerstner, R. Ritz, and J. Leo van Hemmen. Why spikes? hebbian learning and retrieval of time-resolved excitation patterns. Biol. Cybern., 69:503–515, 1993.MATHGoogle Scholar
  27. [27]
    H. P. Graf, E. Sackinger, and L. D. Jackel. Recent developments of electronic neural nets in north america. J. of VLSI Signal Processing, 5:19–31, 1993.CrossRefGoogle Scholar
  28. [28]
    P. Hasler, C. Diorio, B. A. Minch, and C. Mead. Single transistor learning synapses with long term storage. In IEEE Intl. Symp. on Circuits and Systems, volume 3, pages 1660–1663, 1995.Google Scholar
  29. [29]
    N. A. Hessler, A. M. Shirke, and R. Malinow. The probability of transmitter release at a mammalian central synapse. Nature, 366:569–573, 1993.CrossRefGoogle Scholar
  30. [30]
    B. Hille. Ionic Channels of Excitable Membranes. Sinauer Associates Inc., 2 edition, 1992.Google Scholar
  31. [31]
    Y. Hirai. Recent VLSI neural networks in japan. J. of VLSI Signal Processing, 6:7–18, 1993.CrossRefGoogle Scholar
  32. [32]
    A. J. Holmes, R. A. G. Gibson, J. Hajto, A. F. Murray, A. E. Owen, M. J. Rose, and A. J. Snell. Use of a-si:h memory devices for non-volatile weight storage in artificial neural networks. J. of Non-Crystalline Solids, 164–166:817–820, 1993.CrossRefGoogle Scholar
  33. [33]
    T. M. Jessell and E. R. Kandel. Synaptic transmission: A bidirectional and self-modifiable form of cell-cell comunication. Cell, 72:1–30, 1993. Neuron Vol. 10 (Suppl.) pp. 1–30.CrossRefGoogle Scholar
  34. [34]
    E. R. Kandel. Part III: Elementary interactions between neurons: Synaptic transmission. In J. H. Schwartz and T. M. Jessell, editors, Principles of Neural Science, pages 123–260. Appleton and Lange, Norwalk, Connecticut, 3 edition, 1991.Google Scholar
  35. [35]
    E. R. Kandel, M. Klein, B. Hochner, M. Shuster, S. A. Siegelbaum, R. D. Hawkins, D. L. Glanzman, V. F. Castellucci, and T. W. Abrams. Synaptic modulation and learning: New insights into synaptic transmission from the study of behavior. In G. M. Edelman, W. E. Gall, and W. M. Cowan, editors, Synaptic Function, pages 471–518. John Wiley & Sons, 1987.Google Scholar
  36. [36]
    C. Koch and T. Poggio. The biophysical properties of spines as a basis for their electrical function: A comment on kawato and tsakahara (1983). J. Theor. Biol., 113:225–230, 1985.CrossRefGoogle Scholar
  37. [37]
    C. Koch and T. Poggio. Biophysics of computation: Neurons, synapses, and membranes. In G. M. Edelman, W. E. Gall, and W. M. Cowan, editors, Synaptic Function, pages 637–698. ohn Wiley & Sons, 1987.Google Scholar
  38. [38]
    C. Koch and T. Poggio. Multiplying with synapses and neurons. In T. McKenna, J. Davis, and S. F. Zornetzer, editors, Single Neuron Computation, pages 315–346. Academic Press, Inc., 1992.Google Scholar
  39. [39]
    H. Korn and D. S. Faber. Regulation and significance of probablistic release mechanisms at central synapses. In G. M. Edelman, W. E. Gall, and W. M. Cowan, editors, Synaptic Function, pages 57–108. John Wiley & Sons, 1987.Google Scholar
  40. [40]
    H. Korn and A. Mallet. Transformation of binomial input by the postsynaptic membrane at a central synapse. Science, 225:1157–1159, 1984.CrossRefGoogle Scholar
  41. [41]
    D. M. Kullmann and R. A. Nicoll. Long-term potentiation is associated with increases in quantal content and quantal amplitude. Nature, 357:240–244, 1992.CrossRefGoogle Scholar
  42. [42]
    A. Larkman, T. Hannay, K. Stratford, and J. Jack. Presynaptic release probability influences the locus of long-term potentiation. Nature, 360:70–73, 1992.CrossRefGoogle Scholar
  43. [43]
    J. P. Lazzaro and C. Mead. Circuit models of sensory transduction in the cochlea. In Mead and Ismail, editors, Analog VLSI Implementation of Neural Systems, pages 85–101. Kluwer Academic Publishers, Norwell, MA, 1989.Google Scholar
  44. [44]
    B. W. Lee, B. J. Sheu, and H. Yang. Analog floating-gate synapses for general-purpose VLSI neural computation. IEEE Trans. on Circuits and Systems, 38:654–658, 1991.CrossRefGoogle Scholar
  45. [45]
    B. W. Lee and B. J. Sheu. General-purpose neural chips with electrically programmable synapses and gain-adjustable neurons. IEEE J. of Solid-State Circuits, 27:1299–1302, 1992.CrossRefGoogle Scholar
  46. [46]
    D. Liao, N. A. Hessler, and R. Malinow. Activation of post synaptically silent synapses during pairing-induced ltp in ca1 region of hippocampal slice. Nature, 375:400–404, 1995.CrossRefGoogle Scholar
  47. [47]
    B. Linares-Barranco, E. Sánchez-Sinencio, A. Rodriguez-Vazquez, and J. L. Huertas. A CMOS implementation of fitzhugh-nagumo neuron model. IEEE Solid-State Circuits, 26(7):956–965, 1991.CrossRefGoogle Scholar
  48. [48]
    B. Linares-Barranco, E. Sánchez-Sinencio, A. Rodriguez-Vazquez, and J. L. Huertas. A CMOS analog adaptive bam with on-chip learning and weight refreshing. IEEE Trans. on Neural Networks, 4:445–457, 1993.CrossRefGoogle Scholar
  49. [49]
    P. J. Mackenzie, M. Umemiya, and T. H. Murphy. a2+ imaging of cns axons in culture indicates reliable coupling between single action potentials and distal functional release sites. Neuron, 16:783–795, 1996.CrossRefGoogle Scholar
  50. [50]
    K. L. Magleby. Short-term changes in synaptic efficacy. In G. M. Edelman, W. E. Gall, and W. M. Cowan, editors, Synaptic Function, pages 21–56. John Wiley & Sons, 1987.Google Scholar
  51. [51]
    M. Mahowald. VLSI Analogs of Neuronal Visual Processing: a Synthesis of Form and Function. Computation and neural systems, California Institute of Technology, 1992.Google Scholar
  52. [52]
    M. Mahowald and R. Douglas. A silicon neuron. Nature, 354:515–518, 1991.CrossRefGoogle Scholar
  53. [53]
    G. Major, A. U. Larkman, P. Jones, B. Sakmann, and J. J. B. Jack. Detailed passive cable models of whole-cell recorded ca3 pyramidal neurons in rat hippocampal slices. J. of Neurosci., 14(8):4613–4638, 1994.Google Scholar
  54. [54]
    R. C. Malenka, J. A. Kauer, D. J. Perkel, M. D. Mauk, P. T. Kelly, R. A. Nicoll, and M. N. Waxham. An essential role for postsynaptic calmodulin and protein kinase activity in long-term potentiation. Nature, 340:554–557, 1989.CrossRefGoogle Scholar
  55. [55]
    R. C. Malenka, J. A. Kauer, R. S. Zucker, and R. A. Nicoll. Postsynaptic calcium is sufficient for potentiation of hippocampal synaptic transmission. Science, 242:81–84, 1988.CrossRefGoogle Scholar
  56. [56]
    A. Malgaroli, A. E. Ting, B. Wendland, A. Bergamaschi, A. Villa, R.W. Tsien, and R.H. Scheller. Presynaptic component of long-term potentiation visualized at individual hippocampal synapses. Science, 268:1624–1628, 1995.CrossRefGoogle Scholar
  57. [57]
    J. L. Martinez and B. E. Derrick. Long-term potentiation and learning. Annu. Rev. of Psychology, 47:173–203, 1996.CrossRefGoogle Scholar
  58. [58]
    C. A. Mead. Analog VLSI and Neural Systems. Addison-Wesley, Reading, MA, 1989.MATHGoogle Scholar
  59. [59]
    C. A. Mead and M. A. Mahowald. Silicon model of early visual processing. Neural Networks, 1:91–97, 1988.CrossRefGoogle Scholar
  60. [60]
    B. W. Mel. Information processing in dendritic trees. Neural Computation, 6:1031–1085, 1994.MATHGoogle Scholar
  61. [61]
    G. Moon, M. E. Zaghloul, and R. W. Newcomb. An enhancement-mode MOS voltage-controlled linear resistor with large dynamic range. IEEE Trans. on Circuits and Systems, 37:1284–1288, 1990.CrossRefGoogle Scholar
  62. [62]
    G. Moon, M. E. Zaghloul, and R. W. Newcomb. VLSI implementation of synaptic weighting and summing in pulse coded neural-type cells. EEE Trans. Neural Networks, 3:394–403, 1992.CrossRefGoogle Scholar
  63. [63]
    A. Mortara and E. A. Vittoz. A communication architecture tailored for analog VLSI neural networks: Intrinsic performance and limitations. IEEE Transactions on Neural Networks, TNN-5(3):459–466, May 1994.CrossRefGoogle Scholar
  64. [64]
    P. Mueller, J. V. D. Spiegel, D. Blackman, T. Chiu, T. Clare, J. Dao, C. Donham, T. Hsieh, and M. Loinaz. A general purpose analog neural computer. In Int. Jt. Conf. Neural Network, volume 2, pages 177–182, 1989.CrossRefGoogle Scholar
  65. [65]
    T. H. Murphy, J. M. Baraban, W. G. Weir, and L. A. Blatter. Visualization of quantal synaptic transmission by dendritic calcium imaging. Science, 263:529–532, 1994.CrossRefGoogle Scholar
  66. [66]
    A. F. Murray and P. J. Edwards. Enhanced mlp performance and fault tolerance resulting from synaptic weight noise during training. IEEE Trans. on Neural Networks, 5:792–802, 1994.CrossRefGoogle Scholar
  67. [67]
    A. F. Murray and L. Tarassenko. Analogue Neural VLSI: A Pulse Stream Approach. Chapman and Hall, London, England, 1994.Google Scholar
  68. [68]
    A.F. Murray, D. Del Corso, and L. Tarassenko. Pulse-stream VLSI neural networks mixing analog and digital techniques. IEEE Transactions on Neural Networks, 2:193–204, 1991.CrossRefGoogle Scholar
  69. [69]
    D. P. M. Northmore and J. G. Elias. Spike train processing by a silicon neuromorph: The role of sublinear summation in dendrites. Neural Computation, 8:1245–1265, 1996.CrossRefGoogle Scholar
  70. [70]
    N. Otmakhov, A. M. Shrike, and R. Malinow. Measuring the impact of probabilistic transmission on neuronal output. Neuron, 10:1101–1111, 1993.CrossRefGoogle Scholar
  71. [71]
    T. Poggio and V. Torre. A new approach to synaptic interactions. In H. Heim and G. Palm, editors, Lecture Notes in Biomathematics. Theorectical Approaches to Computer Systems, volume 2, pages 89–115. Heidelberg, New York, 1977.Google Scholar
  72. [72]
    W. Rall. Theoretical significance of dendritic trees for neuronal input-output relations. In R. F. Reiss, editor, Neural Theory and Modeling. Stanford University Press, Palo Alto, 1964.Google Scholar
  73. [73]
    W. Rall. Distinguishing theoretical synaptic potentials computed for different soma-dendritic distributions of synaptic inputs. J. of Neurophys., 30:1138–1168, 1967.Google Scholar
  74. [74]
    M. Rapp, I. Segev, and Y. Yarom. Physiology, morphology and detailed passived models of cerebellar purkinje cell. J. of Physiol, 474:101–118, 1994.Google Scholar
  75. [75]
    M. J. Rose, J. Hajto, P. G. Lecomber, S. M. Gage, W. K. Choi, A. J. Snell, and A. E. Owen. Amorphous silicon analogue memory devices. J. of Non-Crystalline Solids, 115:168–170, 1989.CrossRefGoogle Scholar
  76. [76]
    D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by error propagation. In D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, editors, Parallel Distributed Processing: Explorations in the Microstructures of Cognition, volume I: Foundations. MIT Press/Bradford Books, Cambridge, MA, 1986.Google Scholar
  77. [77]
    I. Segev, J. W. Fleshman, and R. E. Burke. Compartmental models of complex neurons. In C. Koch and I. Segev, editors, Methods in Neuronal Modeling: From Synapses to Networks, pages 63–96. The MIT Press, 1989.Google Scholar
  78. [78]
    D. P. Shelton. Membrane resistivity estimated for the purkinje neuron by means of a passive computer model. Neuroscience, 14:111–131, 1985.CrossRefGoogle Scholar
  79. [79]
    G. M. Shepherd. Canonical neurons and their computational organization. In J. Davis and S. F. Zornetzer, editors, Single Neuron Computation, pages 27–60. Academic Press, Inc., 1992.Google Scholar
  80. [80]
    G. M. Shepherd and C. Koch. Introduction to synaptic circuits. In G.M. Shepherd, editor, The Synaptic Organization of the Brain, pages 3–31. Oxford University Press, New York, 1990.Google Scholar
  81. [81]
    T. Shibata and T. Ohmi. A functional MOS transistor featuring gate-level weighted sum and threshold operations. IEEE Trans. on Electron Devices, 39:1444–1455, 1992.CrossRefGoogle Scholar
  82. [82]
    T. Shima, T. Kimura, Y. Kamatani, T. Itakura, Y. Fujita, and T. Iida. Neuro chips with on-chip back-propagation and/or hebbian learning. IEEE J. of Solid-State Circuits, 27(12):1868–1875, 1992.CrossRefGoogle Scholar
  83. [83]
    C. F. Stevens and Y. Wang. Changes in reliability of synaptic function as a mec hanism for plasticity. Nature, 371:704–707, 1994.CrossRefGoogle Scholar
  84. [84]
    K. J. Stratford, A. J. R. Mason, A. U. Larkman, G. Major, and J. J. B. Jack. The modelling of pyramidal neurones in the visual cortex. In R. Durbing, C. Miall, and G. Mitchison, editors, The computing neurone, pages 296–321. Addison-Wesley, 1989.Google Scholar
  85. [85]
    D. Thurbon, A. Field, and S. J. Redman. Electrotonic profiles of interneurones in stratum pyramidale of the ca1 region of rat hippocampus. J. of Neurophys., 71:1948–1958, 1994.Google Scholar
  86. [86]
    J. Tomberg. Synchronous pulse density modulation in neural network implementation. In M. E. Zaghloul, J. L. Meador, and R. W. Newcomb, editors, Silicon Implementation of Pulse Coded Neural Networks, pages 165–198. Kluwer Academic Publishers, 1994.Google Scholar
  87. [87]
    K. Y. Tsai, N. T. Carnevale, and T. H. Brown. Hebbian learning is jointly controlled by electronic and input structure. Network, 5:1–19, 1994.CrossRefGoogle Scholar
  88. [88]
    Y. P. Tsividis and D. Anastassiou. Switched-capacitor neural networks. Electron. Lett., 23:958–959, 1988.CrossRefGoogle Scholar
  89. [89]
    Z. Wang. Novel electronically-controlled floating resistors using MOS transistors operating in saturation. Electron. Lett., 27:188–189, 1991.CrossRefGoogle Scholar
  90. [90]
    L. Watts, D. Kerns, R. F. Lyon, and C. Mead. Improved implementation of the silicon cochlea. IEEE Journal Solid-State Circuits, 27(5):692–700, May 1992.CrossRefGoogle Scholar
  91. [91]
    A. Zador, C. Koch, and T. H. Brown. Biophysical model of a hebbian synapse. In Proc. Natl. Acad. Sci. U.S.A., pages 6718–6722, 1990.Google Scholar
  92. [92]
    M. E. Zaghloul. Silicon Implementation of Pulse Coded Neural Networks. Kluwer Academic Publishers, 1994.Google Scholar

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • Wayne C. Westerman
    • 1
  • David P. M. Northmore
    • 1
  • John. G. Elias
    • 1
  1. 1.Departments of Electrical Engineering and PsychologyUniversity of Delaware Newark

Personalised recommendations