Skip to main content
SpringerLink
Log in
Book cover

Advances in Computational Intelligence pp 167–178Cite as

Third Generation Neural Networks: Spiking Neural Networks

  • Conference paper

Part of the book series: Advances in Intelligent and Soft Computing ((AINSC,volume 116))

Abstract

Artificial Neural Networks (ANNs) are based on highly simplified brain dynamics and have been used as powerful computational tools to solve complex pattern recognition, function estimation, and classification problems. Throughout their development, ANNs have been evolving towards more powerful and more biologically realistic models. In the last decade, the third generation Spiking Neural Networks (SNNs) have been developed which comprise of spiking neurons. Information transfer in these neurons models the information transfer in biological neurons, i.e., via the precise timing of spikes or a sequence of spikes. Addition of the temporal dimension for information encoding in SNNs yields new insight into the dynamics of the human brain and has the potential to result in compact representations of large neural networks. As such, SNNs have great potential for solving complicated time-dependent pattern recognition problems defined by time series because of their inherent dynamic representation. This article presents an overview of the development of spiking neurons and SNNs within the context of feedforward networks, and provides insight into their potential for becoming the next generation neural networks.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   429.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   549.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adeli, H., Hung, S.L.: Machine Learning - Neural Networks, Genetic Algorithms, and Fuzzy Sets. John Wiley and Sons, New York (1995)

    Google Scholar 

  2. Adeli, H., Park, H.S.: Neurocomputing for Design Automation. CRC Press, Boca Raton (1998)

    Google Scholar 

  3. Adeli, H., Karim, A.: Fuzzy-wavelet RBFNN model for freeway incident detection. Journal of Transportation Engineering 126(6), 464–471 (2000)

    Article  Google Scholar 

  4. Adeli, H.: Neural networks in civil engineering: 1989-2000. Computer-Aided Civil and Infrastructure Engineering 16(2), 126–142 (2001)

    Article  Google Scholar 

  5. Ghosh-Dastidar, S., Adeli, H.: Wavelet-clustering-neural network model for freeway incident detection. Computer-Aided Civil and Infrastructure Engineering 18(5), 325–338 (2003)

    Article  Google Scholar 

  6. Adeli, H., Karim, A.: Wavelets in Intelligent Transportation Systems. John Wiley and Sons, Hoboken (2005)

    MATH  Google Scholar 

  7. Adeli, H., Ghosh-Dastidar, S., Dadmehr, N.: Alzheimer’s disease and models of computation: Imaging, classification, and neural models. Journal of Alzheimer’s Disease 7(3), 187–199 (2005a)

    Google Scholar 

  8. Adeli, H., Ghosh-Dastidar, S., Dadmehr, N.: Alzheimer’s disease: Models of computation and analysis of EEGs. Clinical EEG and Neuroscience 36(3), 131–140 (2005b)

    Google Scholar 

  9. Adeli, H., Jiang, X.: Dynamic fuzzy wavelet neural network model for structural system identification. Journal of Structural Engineering 132(1), 102–111 (2006)

    Article  Google Scholar 

  10. Ghosh-Dastidar, S., Adeli, H.: Neural network-wavelet microsimulation model for delay and queue length estimation at freeway work zones. Journal of Transportation Engineering 132(4), 331–341 (2006)

    Article  Google Scholar 

  11. Ghosh-Dastidar, S., Adeli, H.: Improved spiking neural networks for EEG classification and epilepsy and seizure detection. Integrated Computer-Aided Engineering 14(3), 187–212 (2007)

    Google Scholar 

  12. Adeli, H., Hung, S.L.: A concurrent adaptive conjugate gradient learning algorithm on MIMD machines. Journal of Supercomputer Applications 7(2), 155–166 (1993)

    Article  Google Scholar 

  13. Ghosh-Dastidar, S., Adeli, H., Dadmehr, N.: Mixed-band wavelet-chaos-neural network methodology for epilepsy and epileptic seizure detection. IEEE Transactions on Biomedical Engineering 54(9), 1545–1551 (2007)

    Article  Google Scholar 

  14. Adeli, H., Park, H.S.: Counter propagation neural network in structural engineering. Journal of Structural Engineering 121(8), 1205–1212 (1995a)

    Article  Google Scholar 

  15. Panakkat, A., Adeli, H.: Recurrent neural network for approximate earthquake time and location prediction using multiple seismicity indicators. Computer-Aided Civil and Infrastructure Engineering 24(4), 280–292 (2009)

    Article  Google Scholar 

  16. Karim, A., Adeli, H.: Comparison of the fuzzy-wavelet RBFNN freeway incident detection model with the california algorithm. Journal of Transportation Engineering 128(1), 21–30 (2002)

    Article  Google Scholar 

  17. Karim, A., Adeli, H.: Radial basis function neural network for work zone capacity and queue estimation. Journal of Transportation Engineering 129(5), 494–503 (2003)

    Article  Google Scholar 

  18. Ghosh-Dastidar, S., Adeli, H., Dadmehr, N.: Principal component analysis-enhanced cosine radial basis function neural network for robust epilepsy and seizure detection. IEEE Transactions on Biomedical Engineering 55(2), 512–518 (2008)

    Article  Google Scholar 

  19. Jiang, X., Adeli, H.: Dynamic wavelet neural network model for traffic flow forecasting. Journal of Transportation Engineering 131(10), 771–779 (2005)

    Article  Google Scholar 

  20. Jiang, X., Adeli, H.: Pseudospectra, MUSIC, and dynamic wavelet neural network for damage detection of highrise buildings. International Journal for Numerical Methods in Engineering 71(5), 606–629 (2007)

    Article  Google Scholar 

  21. Jiang, X., Adeli, H.: Dynamic fuzzy wavelet neuroemulator for nonlinear control of irregular highrise building structures. International Journal for Numerical Methods in Engineering 74(7), 1045–1066 (2008)

    Article  MATH  Google Scholar 

  22. Adeli, H., Hung, S.L.: An adaptive conjugate gradient learning algorithm for effective training of multilayer neural networks. Applied Mathematics and Computation 62(1), 81–102 (1994)

    Article  MATH  Google Scholar 

  23. Adeli, H., Park, H.S.: Optimization of space structures by neural dynamics. Neural Networks, Vol 8(5), 769–781 (1995b)

    Article  Google Scholar 

  24. Adeli, H., Karim, A.: Neural dynamics model for optimization of cold-formed steel beams. Journal of Structural Engineering 123(11), 1535–1543 (1997)

    Article  Google Scholar 

  25. Adeli, H., Jiang, X.: Neuro-fuzzy logic model for freeway work zone capacity estimation. Journal of Transportation Engineering 129(5), 484–493 (2003)

    Article  Google Scholar 

  26. Sejnowski, T.J.: Open questions about computation in the cerebral cortex. In: Rumelhart, D.E., McClelland, J.L. (eds.) Parallel Distributed Processing, vol. 2, pp. 372–389. MIT Press, Cambridge (1986)

    Google Scholar 

  27. Maass, W.: Lower bounds for the computational power of spiking neural networks. Neural Computation 8(1), 1–40 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  28. Maass, W.: Networks of spiking neurons: The third generation of spiking neural network models. Neural Networks 10(9), 1659–1671 (1997a)

    Article  Google Scholar 

  29. Bose, N.K., Liang, P.: Neural Network Fundamentals with Graphs, Algorithms, and Applications. McGraw-Hill, New York (1996)

    Google Scholar 

  30. Maass, W.: Noisy spiking neurons with temporal coding have more computational power than sigmoidal neurons. In: Mozer, M., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, pp. 211–217. MIT Press, Cambridge (1997b)

    Google Scholar 

  31. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Rumelhart, D.E., McClelland, J.L. (eds.) Parallel Distributed Processing, vol. 1, pp. 318–362. MIT Press, Cambridge (1986)

    Google Scholar 

  32. Hung, S.L., Adeli, H.: Parallel backpropagation learning algorithms on Cray Y-MP8/864 supercomputer. Neurocomputing 5(6), 287–302 (1993)

    Article  Google Scholar 

  33. Park, H.S., Adeli, H.: Distributed neural dynamics algorithms for optimization of large steel structures. Journal of Structural Engineering 123(7), 880–888 (1997)

    Article  Google Scholar 

  34. Adeli, H., Wu, M.: Regularization neural network for construction cost estimation. Journal of Construction Engineering and Management 124(1), 18–24 (1998)

    Article  Google Scholar 

  35. Adeli, H., Samant, A.: An adaptive conjugate gradient neural network - wavelet model for traffic incident detection. Computer-Aided Civil and Infrastructure Engineering 15(4), 251–260 (2000)

    Article  Google Scholar 

  36. Sirca, G., Adeli, H.: Neural network model for uplift load capacity of metal roof panels. Journal of Structural Engineering 127(11), 1276–1285 (2001)

    Article  Google Scholar 

  37. Sirca, G., Adeli, H.: Neural network model for uplift load capacity of metal roof panels - closure. Journal of Structural Engineering 129(4), 562–563 (2003)

    Article  Google Scholar 

  38. Dharia, A., Adeli, H.: Neural network model for rapid forecasting of freeway link travel time. Engineering Applications of Artificial Intelligence 16(7-8), 607–613 (2003)

    Article  Google Scholar 

  39. Panakkat, A., Adeli, H.: Neural network models for earthquake magnitude prediction using multiple seismicity indicators. International Journal of Neural Systems 17(1), 13–33 (2007)

    Article  Google Scholar 

  40. Jiang, X., Adeli, H.: Neuro-genetic algorithm for nonlinear active control of highrise buildings. International Journal for Numerical Methods in Engineering 75(8), 770–786 (2008)

    Article  Google Scholar 

  41. Kandel, E.R., Schwartz, J.H., Jessell, T.M.: Principles of Neural Science, 4th edn. McGraw-Hill, New York (2000)

    Google Scholar 

  42. Hodgkin, A.L., Huxley, A.F.: A quantitative description of ion currents and its applications to conduction and excitation in nerve membranes. Journal of Physiology 117, 500–544 (1952)

    Google Scholar 

  43. Rinzel, J., Ermentrout, G.B.: Analysis of neuronal excitability and oscillations. In: Koch, C., Segev, I. (eds.) Methods in Neuronal Modeling, pp. 135–169. MIT Press, Cambridge (1989)

    Google Scholar 

  44. Hille, B.: Ionic channels of excitable membranes, 2nd edn. Sinauer Associates, Sunderland (1992)

    Google Scholar 

  45. Ermentrout, G.B.: Type I membranes, phase resetting curves, and synchrony. Neural Computation 8, 979–1001 (1996)

    Article  Google Scholar 

  46. Hoppensteadt, F.C., Izhikevich, E.M.: Weakly Connected Neural Networks. Springer, New York (1997)

    Google Scholar 

  47. Izhikevich, E.M.: Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral Cortex 17, 2443–2452 (2007)

    Article  Google Scholar 

  48. Abbott, L.F., Kepler, T.B.: Model neurons: From Hodgkin-Huxley to Hopfield. In: Garrido, L. (ed.) Statistical Mechanics of Neural Networks. Springer, Berlin (1990)

    Google Scholar 

  49. Kepler, T.B., Abbott, L.F., Marder, E.: Reduction of conductance-based neuron models. Biological Cybernetics 66, 381–387 (1992)

    Article  MATH  Google Scholar 

  50. Ermentrout, G.B., Kopell, N.: Parabolic bursting in an excitable system coupled with a slow oscillation. SIAM Journal on Applied Mathematics 46, 233–253 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  51. Gerstner, W.: Time structure of the activity in neural network models. Physical Review E 51, 738–758 (1995)

    Article  Google Scholar 

  52. Kistler, W.M., Gerstner, W., van Hemmen, J.L.: Reduction of Hodgkin-Huxley equations to a single-variable threshold model. Neural Computation 9, 1015–1045 (1997)

    Article  Google Scholar 

  53. Izhikevich, E.M.: Resonate-and-fire neurons. Neural Networks 14, 883–894 (2001)

    Article  Google Scholar 

  54. Gerstner, W., Kistler, W.M.: Spiking Neuron Models. Single Neurons, Populations, Plasticity. Cambridge University Press, New York (2002)

    MATH  Google Scholar 

  55. Bohte, S.M., Kok, J.N., La Poutré, J.A.: Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48(1-4), 17–37 (2002)

    Article  MATH  Google Scholar 

  56. Natschläger, T., Ruf, B.: Spatial and temporal pattern analysis via spiking neurons. Network: Computation in Neural Systems 9(3), 319–332 (1998)

    Article  MATH  Google Scholar 

  57. Booij, O., Nguyen, H.T.: A gradient descent rule for multiple spiking neurons emitting multiple spikes. Information Processing Letters 95(6), 552–558 (2005)

    Article  MathSciNet  Google Scholar 

  58. Ghosh-Dastidar, S., Adeli, H.: A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection. Neural Networks 22 (in press, 2009)

    Google Scholar 

  59. Xin, J., Embrechts, M.J.: Supervised learning with spiking neural networks. In: Proceedings of the International Joint Conference on Neural Networks, Washington, DC, vol. 3, pp. 1772–1777 (2001)

    Google Scholar 

  60. McKennoch, S., Liu, D., Bushnell, L.G.: Fast modifications of the SpikeProp algorithm. In: Proceedings of the International Joint Conference on Neural Networks, Vancouver, Canada, pp. 3970–3977 (2006)

    Google Scholar 

  61. Silva, S.M., Ruano, A.E.: Application of Levenberg-Marquardt method to the training of spiking neural networks. In: Proceedings of the International Conference on Neural Networks and Brain, vol. 3, pp. 1354–1358 (2005)

    Google Scholar 

  62. Fahlman, S.E.: Faster-learning variations of back-propagation: An empirical study. In: Proceedings of the 1988 Connectionist Models Summer School, San Mateo, CA, pp. 38–51. Morgan Kaufmann, San Francisco (1988)

    Google Scholar 

  63. Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: The Rprop algorithm. In: IEEE International Conference on Neural Networks, San Francisco, CA, vol. 1, pp. 586–591 (1993)

    Google Scholar 

  64. Schrauwen, B., van Campenhout, J.: Extending SpikeProp. In: Proceedings of the International Joint Conference on Neural Networks, Budapest, pp. 471–476 (2004)

    Google Scholar 

  65. Moore, S.C.: Backpropagation in spiking neural networks. Master’s thesis, University of Bath (2002)

    Google Scholar 

  66. Maass, W.: Fast sigmoidal networks via spiking neurons. Neural Computation 9(2), 279–304 (1997c)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, The Ohio State University,  

    Samanwoy Ghosh-Dastidar

  2. Abba G. Lichtenstein Professor, Departments of Biomedical Engineering, Biomedical Informatics, Civil and Environmental Engineering and Geodetic Science, Electrical and Computer Engineering, and Neuroscience, The Ohio State University, 470 Hitchcock Hall, 2070 Neil Avenue, Columbus, Ohio, 43210

    Hojjat Adeli

Authors
  1. Samanwoy Ghosh-Dastidar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hojjat Adeli
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. CINVESTAV, Mexico City, Mexico

    Wen Yu

  2. CINVESTAV, Guadalajara, Mexico

    Edgar N. Sanchez

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Ghosh-Dastidar, S., Adeli, H. (2009). Third Generation Neural Networks: Spiking Neural Networks. In: Yu, W., Sanchez, E.N. (eds) Advances in Computational Intelligence. Advances in Intelligent and Soft Computing, vol 116. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03156-4_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-03156-4_17

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-03155-7

  • Online ISBN: 978-3-642-03156-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation