From von Neumann Architecture and Atanasoffs ABC to Neuro-Morphic Computation and Kasabov’s NeuCube: Principles and Implementations

  • Neelava Sengupta
  • Josafath Israel Espinosa Ramos
  • Enmei Tu
  • Stefan Marks
  • Nathan Scott
  • Jakub Weclawski
  • Akshay Raj Gollahalli
  • Maryam Gholami Doborjeh
  • Zohreh Gholami Doborjeh
  • Kaushalya Kumarasinghe
  • Vivienne Breen
  • Anne Abbott
Chapter
Part of the Studies in Computational Intelligence book series (SCI, volume 756)

Abstract

During the 1940s John Atanasoff with the help of one of his students Clifford E. Berry, at Iowa State College, created the ABC (Atanasoff-Berry Computer) that was the first electronic digital computer. The ABC computer was not a general-purpose one, but still, it was the first to implement three of the most important ideas used in computers nowadays: binary data representation; using electronics instead of mechanical switches and wheels; using a von Neumann architecture, where the memory and the computations are separated. A new computational paradigm, named as Neuromorphic, utilises the above two principles, but instead of the von Neumann principle, it integrates the memory and the computation in a single module a spiking neural network structure. This chapter first reviews the principles of the earlier published work by the team on neuromorphic computational architecture NeuCube. NeuCube is not a general purpose machine but is still the first neuromorphic spatio/spectro-temporal data machine for learning, pattern recognition and understanding of spatio/spectro-temporal data. The chapter further presents the software/hardware implementation of the NeuCube as a development system for efficient applications on temporal or spatio/spectro-temporal across domain areas, including: brain data (EEG, fMRI), brain computer interfaces, robot control, multi-sensory data modelling, seismic stream data modelling and earthquake prediction, financial time series forecasting, climate data modelling and personalised, on-line risk of stroke prediction, and others. A limited version of the NeuCube software implementation is available from http://www.kedri.aut.ac.nz/neucube/.

Notes

Acknowledgements

The NeuCube development is funded by the Auckland University of Technology SRIF grant. Nikola Kasabov and Giacomo Indiveri from ETH and University of Zurich were granted an EU Marie Curie grant in 2011–2012 to start a preliminary research on SNN for spatio-temporal data. The research groups lead by Zeng-Guang Hou and Jie Yang from China contributed to the earlier software implementation of the NeuCube development system.

References

  1. 1.
    David, B.: The Advent of the Algorithm: The 300-Year Journey From an Idea to the Computer. Houghton Mifflin Harcourt (2001)Google Scholar
  2. 2.
    Warren, S., McCulloch, Walter, P.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophy. 5(4), 115–133 (1943)Google Scholar
  3. 3.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)Google Scholar
  4. 4.
    Robert, R.S.: Moore’s law: past, present and future. IEEE Spectr. 34(6), 52–59 (1997)Google Scholar
  5. 5.
    Brian R.: The Origins of Digital Computers: Selected Papers. Springer (2013)Google Scholar
  6. 6.
    Toumey, C.: Less is moore. Nature Nanotechnol. 11, 2–3 (2016)Google Scholar
  7. 7.
    Mead, C.: Neuromorphic electronic systems. Proc. IEEE 78(10), 1629–1636 (1990)Google Scholar
  8. 8.
    Nikola, K., Neelava, S., Nathan, S.: From von neumann, John atanasoff and abc to neuromorphic computation and the neucube spatio-temporal data machine. In: 2016 IEEE 8th International Conference on Intelligent Systems (IS), pp. 15–21. IEEE (2016)Google Scholar
  9. 9.
    Ivan, S.: Neuromorphic Computing: From Materials To Systems Architecture. Accessed 16 July 2016Google Scholar
  10. 10.
    Calimera, A., Macii, E., Poncino, M.: The human brain project and neuromorphic computing. Function. Neurol. 28(3), 191–196 (2013)Google Scholar
  11. 11.
    Hsu, J.: Ibm’s new brain [news]. IEEE Spectr. 51(10), 17–19 (2014)Google Scholar
  12. 12.
    Merolla, P.A., Arthur, J.V., Rodrigo, A.-I., Cassidy, A.S., Sawada, J., Akopyan, F., Jackson, B.L., Imam, N., Guo, C., Nakamura, Y. et al.: A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345(6197), 668–673 (2014)Google Scholar
  13. 13.
    Ben, V.B., Peiran, G., McQuinn, E., Swadesh, C., Anand, R.C., Bussat, J.-M., Rodrigo, A.-I., John, V.A., Paul, A.M., Kwabena, B.N.: A mixed-analog-digital multichip system for large-scale neural simulations. Proc. IEEE 102(5), 699–716 (2014)CrossRefGoogle Scholar
  14. 14.
    Steve, B., Furber, D., Lester, R., Luis, Plana, A., Jim, D., Garside, E.P., Steve, T., Andrew, D.B.: Overview of the spinnaker system architecture. IEEE Trans. Comput. 62(12), 2454–2467 (2013)Google Scholar
  15. 15.
    Indiveri, G., Linares-Barranco, Bernabé, H., Tara, J., Van Schaik, A., Etienne-Cummings, R., Delbruck, T., Liu, Shih-Chii, Dudek, Piotr, Häfliger, Philipp, Renaud, S., et al.: Neuromorphic silicon neuron circuits. Front. Neurosci. 5, 73 (2011)Google Scholar
  16. 16.
    Indiveri, Giacomo: Liu, S.-C. Memory and information processing in neuromorphic systems. Proc. IEEE 103(8), 1379–1397 (2015)CrossRefGoogle Scholar
  17. 17.
    Maass, W.: Networks of spiking neurons: the third generation of neural network models. Neural Netw. 10(9), 1659–1671 (1997)CrossRefGoogle Scholar
  18. 18.
    Maass, W., Bishop, C.M.: Pulsed Neural Networks. MIT press, New york (2001)MATHGoogle Scholar
  19. 19.
    Elisa, C., Nikola, K., Grace, Y., Wang et al.: Analysis of connectivity in neucube spiking neural network models trained on EEG data for the understanding of functional changes in the brain: a case study on opiate dependence treatment. Neural Netw. 68:62–77 (2015)Google Scholar
  20. 20.
    Maryam Gholami, D., Elisa, C., Nikola, K.: Classification and segmentation of fmri spatio-temporal brain data with a neucube evolving spiking neural network model. In: 2014 IEEE Symposium on Evolving and Autonomous Learning Systems (EALS), pp. 73–80. IEEE (2014)Google Scholar
  21. 21.
    Delbruck, T., Patrick, L.: Fast sensory motor control based on event-based hybrid neuromorphic-procedural system. In: Circuits and Systems, 2007. ISCAS 2007. IEEE International Symposium on, pp. 845–848. IEEE (2007)Google Scholar
  22. 22.
    Nikola, K., Nathan, M.S., Enmei, T., Stefan, M., Neelava, S., Elisa, C., Muhaini, O., Maryam, G., Doborjeh, Norhanifah M., Reggio, H., et al.: Evolving spatio-temporal data machines based on the neucube neuromorphic framework design methodology and selected applications. Neural Netw. 78, 1–14 (2016)Google Scholar
  23. 23.
    Eric, B.B.: On the capabilities of multilayer perceptrons. J. Complex. 4(3), 193–215 (1988)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Sandberg, W.: Universal approximation using radial-basis-function networks. Neural Computat 3(2), 246–257 (1991)Google Scholar
  25. 25.
    Donald, F.: Specht. Probabilistic neural networks. Neural networks 3(1), 109–118 (1990)CrossRefGoogle Scholar
  26. 26.
    Kohonen, T.: The self-organizing map. Neurocomputing 21(1), 1–6 (1998)Google Scholar
  27. 27.
    Nikola, K.: Evolving connectionist systems: the knowledge engineering approach. Springer Science & Business Media (2007)Google Scholar
  28. 28.
    Neucom: http://www.theneucom.com. Accessed 15 Aug 2015
  29. 29.
    Schaul, Tom: Bayer, Justin, Wierstra, Daan, Sun, Yi, Felder, Martin, Sehnke, Frank, Rückstieß, Thomas, Schmidhuber, Jürgen: Pybrain. J. Mach. Learn. Res. 11, 743–746 (2010)Google Scholar
  30. 30.
    Steffen, N., Evan, N.: Fast artificial neural network library. leenissen.dk/fann/html/files/fann-h.html. (2000)Google Scholar
  31. 31.
    Mark, H., Eibe, F., Geoffrey, H., Bernhard, P., Peter, R., Ian, H.W.: The Weka data mining software: an update. ACM SIGKDD Explorat. Newslett. 11(1):10–18 (2009)Google Scholar
  32. 32.
    Michael, R.B., Nicolas, C., Fabian, D., Thomas, R., Gabriel, Tobias, K., Thorsten, M., Peter, O., Christoph, S., Kilian, T., Bernd, W.K.: The konstanz information miner. In: Data Analysis, Machine Learning and Applications, pp. 319–326. Springer (2008)Google Scholar
  33. 33.
    Demšar, J., Zupan, B., Leban, G., Tomaz, C.: From experimental machine learning to interactive data mining. Springer, Orange (2004)Google Scholar
  34. 34.
    Michael, L., Hines, N., Carnevale, T.: The neuron simulation environment. Neural Comput. 9(6), 1179–1209 (1997)Google Scholar
  35. 35.
    Romain, B., Michelle, R., Ted, C., Hines, M., Beeman, D., Bower, J.M., Diesmann, M., Morrison, A., Goodman, P.H., Harris, Jr., Frederick, C., et al.: Simulation of networks of spiking neurons: a review of tools and strategies. J. Comput. Neurosci. 23(3), 349–398 (2007)Google Scholar
  36. 36.
    Jochen Martin, E., Moritz, H., Eilif, M., Markus, D., Marc-Oliver, G.: Pynest: a convenient interface to the nest simulator. Front. Neuroinformat. 2 (2008)Google Scholar
  37. 37.
    Dejan, P., Thomas, N.,Klaus, S.: Pcsim: a parallel simulation environment for neural circuits fully integrated with python. Front. Neuroinformat. 3 (2009)Google Scholar
  38. 38.
    Thomas, N., Henry, M., Wolfgang, M.: Computer models and analysis tools for neural microcircuits. In: Neuroscience Databases, pp. 123–138. Springer (2003)Google Scholar
  39. 39.
    Rich, D.: Brainlab: a toolkit to aid in the design, simulation, and analysis of spiking neural networks with the NCS environment. PhD thesis, University of Nevada Reno (2005)Google Scholar
  40. 40.
    E Courtenay, W.: Parallel implementation of a large scale biologically realistic neocortical neural network simulator. PhD thesis, University of Nevada Reno (2001)Google Scholar
  41. 41.
    Dejan, P.: Oger: Modular learning architectures for large-scale sequential processingGoogle Scholar
  42. 42.
    Goodman, Dan, F.M.: Code generation: a strategy for neural network simulators. Neuroinformatics 8(3), 183–196 (2010)Google Scholar
  43. 43.
    Goodman, Dan, F.M., Brette, R.: The brian simulator. Front. Neurosci. 3(2), 192 (2009)Google Scholar
  44. 44.
    Diesmann, M.: Gewaltig, Marc-Oliver, Aertsen, Ad: Stable propagation of synchronous spiking in cortical neural networks. Nature 402(6761), 529–533 (1999)Google Scholar
  45. 45.
    Nikola, K.: Kasabov. Neucube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data. Neural Netw. 52, 62–76 (2014)CrossRefGoogle Scholar
  46. 46.
    Neelava Sengupta, Nathan Scott, Nikola, K.: Framework for knowledge driven optimisation based data encoding for brain data modelling using spiking neural network architecture. In: Proceedings of the Fifth International Conference on Fuzzy and Neuro Computing (FANCCO-2015), pp. 109–118. Springer (2015)Google Scholar
  47. 47.
    Fusi, S.: Spike-driven synaptic plasticity for learning correlated patterns of mean firing rates. Rev. Neurosci. 14(1–2), 73–84 (2003)Google Scholar
  48. 48.
    Song, Sen: Kenneth D Miller, and Larry F Abbott. Competitive hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neurosci. 3(9), 919–926 (2000)Google Scholar
  49. 49.
    Kasabov, N.: Dhoble, K., Nuntalid, N., Indiveri, G.: Dynamic evolving spiking neural networks for on-line spatio-and spectro-temporal pattern recognition. Neural NetW. 41, 188–201 (2013)Google Scholar
  50. 50.
    Mohemmed, Ammar: Schliebs, Stefan, Matsuda, Satoshi, Kasabov, Nikola: Span: Spike pattern association neuron for learning spatio-temporal spike patterns. Int. J. Neural Syst. 22(04), 1250012 (2012)CrossRefGoogle Scholar
  51. 51.
    Nathan S., Nikola K., Giacomo Indiveri.: Neucube neuromorphic framework for spatio-temporal brain data and its python implementation. In: Neural Information Processing, pp. 78–84. Springer (2013)Google Scholar
  52. 52.
    Marks, S., Javier, E., Nathan, S.: Immersive Visualisation Of 3-dimensional Neural Network Structures. (2015)Google Scholar
  53. 53.
    Stefan Marks. Immersive Visualisation Of 3-dimensional Spiking Neural Networks. Evolving Syst. pp. 1–9 (2016)Google Scholar
  54. 54.
    Kasabov, N., Yingjie, H.: Integrated optimisation method for personalised modelling and case studies for medical decision support. Int. J. Function. Informat. Personal. Med. 3(3), 236–256 (2010)Google Scholar
  55. 55.
    Maryam Gholami, D., Nikola, K.: Personalised modelling on integrated clinical and EEG spatio-temporal brain data in the neucube spiking neural network system. In: 2016 International Joint Conference on Neural Networks (IJCNN), pp. 1373–1378. IEEE (2016)Google Scholar
  56. 56.
    Hadi, E., Emily, B., Renee, S.T., Amant, K.S, Doug, B.: Dark silicon and the end of multicore scaling. ACM SIGARCH Comput. Architect. News 39(3):365 (2011)Google Scholar
  57. 57.
    Perrin, D.: Complexity and high-end computing in biology and medicine. Advanc. Experiment. Med. Biol. 696, 377–84 (2011)Google Scholar
  58. 58.
    Furber, S.: To build a brain. IEEE Spect. 49(8), 44–49 (2012)Google Scholar
  59. 59.
    Indiveri, G., Linares-Barranco, Bernabé, Tara Julia, H., André van Schaik, Ralph Etienne-Cummings, Tobi Delbruck, Shih-Chii Liu, Piotr Dudek, Philipp, Häfliger, Sylvie, R., Johannes, S., Gert, C., John, A., Kai, H., Fopefolu, F., Sylvain, S., Teresa, S.-G., Jayawan, W., Yingxue, W., Kwabena, B.: Neuromorphic silicon neuron circuits. Front. Neurosci. 5, 73 (2011)Google Scholar
  60. 60.
    Andrew, P., Davison, D., Brüderle, Jochen, E., Jens, K., Eilif, M., Dejan, P., Laurent, P., Pierre, Y.: PyNN: A common interface for neuronal network simulators. Front. Neuroinformat. 211 (2008)Google Scholar
  61. 61.
    Bruckner, S., Solteszova, V., Groller, E., Hladuvka, J., Buhler, K., Yu, J., Dickson, B.: BrainGazer–visual queries for neurobiology research. IEEE Trans. Visualizat. Comput. Graph. 15(6), 1497–1504 (2009)CrossRefGoogle Scholar
  62. 62.
    Lin, C.-Y. Tsai, ,K.-L., Wang, S.-C., Hsieh, C.-H., Chang, H.-M., Chiang, A.-S.: The neuron navigator: exploring the information pathway through the neural maze. In: IEEE Pacific Visualization Symposium (PacificVis) 20, pp. 35–42 (2011)Google Scholar
  63. 63.
    von Kapri, A., Rick, T., Potjans, T.C., Diesmann, M., Kuhlen, T.: Towards the visualization of spiking neurons in virtual reality. Stud. Health Technol. Informat. 163, 685–87 (2011)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Neelava Sengupta
    • 1
  • Josafath Israel Espinosa Ramos
    • 1
  • Enmei Tu
    • 2
  • Stefan Marks
    • 3
  • Nathan Scott
    • 1
  • Jakub Weclawski
    • 4
  • Akshay Raj Gollahalli
    • 1
  • Maryam Gholami Doborjeh
    • 1
  • Zohreh Gholami Doborjeh
    • 1
  • Kaushalya Kumarasinghe
    • 1
  • Vivienne Breen
    • 1
  • Anne Abbott
    • 1
  1. 1.KEDRI, AUTAucklandNew Zealand
  2. 2.Rolls Royce@NTU-corporate LabNTUSingapore
  3. 3.Colab, AUTAucklandNew Zealand
  4. 4.Warsaw University of TechnologyWarsawPoland

Personalised recommendations