Skip to main content
SpringerLink
Log in

Audio- and Visual Information Processing in the Brain and Its Modelling with Evolving SNN

  • Chapter
  • First Online:
Time-Space, Spiking Neural Networks and Brain-Inspired Artificial Intelligence

Part of the book series: Springer Series on Bio- and Neurosystems ((SSBN,volume 7))

Abstract

This chapter presents first some background knowledge on how the human brain processes audio- and visual information. Then methods are presented for audio-, visual- and for the integrated audio and visual information processing using evolving spiking neural networks that include convolutional evolving spiking neural networks (CeSNN). Case studies are presented for person identification.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 299.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 299.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. N. Kasabov, Evolving Connectionist Systems: The Knowledge Engineering Approach (Springer, Heidelberg, 2007)

    Google Scholar 

  2. L. Benuskova, N. Kasabov, Computational Neurogenetic Modelling (Springer, Heidelberg, 2007)

    Google Scholar 

  3. D.D. Greenwood, Critical bandwidth and the frequency coordinates of the basilar membrane. J. Acoust. Soc. Am. 33(10) (1961). https://doi.org/10.1131/1.1908437

  4. D. D. Greenwood, A cochlear frequency-position function for several species–29 years later. J. Acoust. Soc. Am. 87(6), 2592–2605 (1990)

    Google Scholar 

  5. E. de Boer, H. R. de Jongh, On cochlear encoding: potentialities and limitations of the reverse-correlation technique. J. Acoust. Soc. Am. 63(1), 115–135 (1978)

    Google Scholar 

  6. J.B. Allen, How do humans process and recognize speech? IEEE Trans. Speech Audio Process. 2(4), 567–577 (1994)

    Google Scholar 

  7. E. Zwicker, Subdivision of the audible frequency range into critical bands (Frequenzgruppen). J. Acoust. Soc. Am. 33, 248 (1961)

    Article  Google Scholar 

  8. B.R. Glasberg, B.C. Moore, Derivation of auditory filter shapes from notched-noise data. Hear Res. 47(1–2), 103–138 (1990)

    Google Scholar 

  9. T.J. Cole, J.A. Blendy, A.P. Monaghan, K. Krieglstein, W. Schmid, A. Aguzzi, G. Fantuzzi, E. Hummler, K. Unsicker, G. Schütz, Targeted disruption of the glucocorticoid receptor gene blocks adrenergic chromaffin cell development and severely retards lung maturation. Genes Dev. 9(13), 1608–1621 (1995)

    Google Scholar 

  10. A.M. Aertsen, J.H. Olders, P.I. Johannesma, Spectro-temporal receptive fields of auditory neurons in the grassfrog. III. Analysis of the stimulus-event relation for natural stimuli. Biol. Cybern. 39(3), 195–209 (1981)

    Google Scholar 

  11. E.O. Postma, H.J. van den Herik, P.T.W. Hudson, Image recognition by brains and machines, in Brain-like Computing and Intelligent Information Systems, ed. by S. Amari, N. Kasabov (Springer, Singapore, 1998), pp. 25–47

    Google Scholar 

  12. E.O. Postma, H.J. van den Herik, P.T.W. Hudson, SCAN: a scalable model of covert attention. Neural Netw. 10, 993–1015 (1997)

    Google Scholar 

  13. K. Kim, N. Relkin, K.-M. Lee, J. Hirsch, Distinct cortical areas associated with native and second languages. Nature 388, 171–174 (1997)

    Google Scholar 

  14. S. Wysoski, L. Benuskova, N. Kasabov, Evolving spiking neural networks for audio-visual information processing. Neural Netw. 23(7), 819–835 (2010)

    Google Scholar 

  15. S. Wysoski, L. Benuskova, N. Kasabov, Fast and adaptive network of spiking neurons for multi-view visual pattern recognition. Neurocomputing 71(13–15), 2563–2575 (2008)

    Google Scholar 

  16. A. Ross, A.K. Jain, Information fusion in biometrics. Pattern Recognit. Lett. 24(13), 2115–2135 (2003)

    Google Scholar 

  17. C. Sanderson, K.K. Paliwal, Identity verification using speech and face information. Digital Signal Process. 14, 449–480 (2004)

    Article  Google Scholar 

  18. A. Sharkey, Combining Artificial Neural Nets: Ensemble and Modular Multi-net systems (Springer, Heidelberg, 1999)

    Google Scholar 

  19. B.E. Stein, M.A. Meredith, The Merging of the Senses (MIT Press, Cambridge, 1993)

    Google Scholar 

  20. S.G. Wysoski, L. Benuskova, N. Kasabov, Adaptive spiking neural networks for audiovisual pattern recognition, ICONIP. Lecture notes in computer science (Springer, 2007)

    Google Scholar 

  21. C. Sanderson, K.K. Paliwal, Identity verification using speech and face information. Digital Signal Process. 14, 449–480 (2004)

    Article  Google Scholar 

  22. P. Viola, M.J. Jones, Rapid object detection using a boosted cascade of simple features. Proceed. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 1(2001), 511–518 (2001)

    Google Scholar 

  23. A. Delorme, L. Perrinet, S. Thorpe, Networks of integrate-and-fire neurons using rank order coding. Neurocomputing. 38–48 (2001)

    Google Scholar 

  24. A. Delorme, S. Thorpe, Face identification using one spike per neuron: resistance to image degradation. Neural Netw. 14, 795–803 (2001)

    Article  Google Scholar 

  25. A.R. McIntosh, R.E. Cabeza, N.J. Lobaugh, Analysis of neural interactions explains the activation of occipital cortex by an auditory stimulus. J. Neurophysiol. 80(1998), 2790–2796 (1998)

    Article  Google Scholar 

  26. D. Gonzalo, T. Shallice, R. Dolan, Time-dependent changes in learning audiovisual associations: a single-trial fMRI study. NeuroImage 11, 243–255 (2000)

    Article  Google Scholar 

  27. A.M. Burton, V. Bruce, R.A. Johnston, Understanding face recognition with an interactive activation model. B. J. Psychol. 81, 361–380 (1990)

    Article  Google Scholar 

  28. A.W. Ellis, A. Young, D.C. Hay, Modelling the recognition of faces and words, in Modelling Cognition, ed. by P.E. Morris (Wiley, London, 1987), p. 1987

    Google Scholar 

  29. H.D. Ellis, D.M. Jones, N. Mosdell, Intra- and inter-modal repetition priming of familiar faces and voices. B. J. Psycol. 88, 143–156 (1997)

    Article  Google Scholar 

  30. K. Kriegstein, A. von, Giraud, Implicit multisensory associations influence voice recognition. PLoS Biol. 4(10), 1809–1820 (2006)

    Google Scholar 

  31. N. Kasabov, E. Postma, J. van den Herik, AVIS: a connectionist-based framework for integrated auditory and visual information processing. Inf. Sci. 133, 137–148 (2000)

    Google Scholar 

  32. N.Kasabov (ed) (2014) Springer Handbook of Bio-/Neuroinformatics (Springer, Heidelberg, 2014)

    Google Scholar 

  33. N. Kasabov, NeuCube: a spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data. Neural Netw. 52, 62–76 (2014). https://doi.org/10.1016/j.neunet.2014.01.006

    Article  Google Scholar 

  34. Wikipedia: http://www.wikipedia.com

  35. L. Benuskova, N. Kasabov, Computational Neurogenetic Modelling (Springer, Heidelberg, 2007)

    Google Scholar 

Download references

Acknowledgements

Some of the material in this chapter was first published in [1, 2, 31, 14]. I acknowledge the contribution of my co-authors of these publications Simei Wysoski, Lubica Benuskova, Eric Postma and Jaap van den Herik.

Author information

Authors and Affiliations

  1. Knowledge Engineering and Discovery Research Institute (KEDRI), Auckland University of Technology, Auckland, New Zealand

    Nikola K. Kasabov

Authors
  1. Nikola K. Kasabov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikola K. Kasabov .

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer-Verlag GmbH Germany, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Kasabov, N.K. (2019). Audio- and Visual Information Processing in the Brain and Its Modelling with Evolving SNN. In: Time-Space, Spiking Neural Networks and Brain-Inspired Artificial Intelligence . Springer Series on Bio- and Neurosystems, vol 7. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-57715-8_12

Download citation

Publish with us

Policies and ethics

Search

Navigation