Smart: How to Simulate Huge Networks

Abstract

Artificial Neural Networks often rhyme with massive parallelism. We need parallelism to simulate neural nets and fully benefit of their promises but we do need an efficient way to implement it. So we propose a dedicated approach, general enough to encompass a large class of models. It consists in performing arithmetics oriented algorithms. That leads to the joint development of hardware and software tools: the SMART machine. SMART stands for Sparse Matrix Adaptive Recursive Transforms, the general framework we use to discribe ANN. In order to handle networks with a sparse or dynamic connections topology we propose an original architecture. Using high speed CMOS technology it will deliver up to 300 MFlops in a deck side format, supporting up to one million connections: from 1000 fully interconnected neuronlikes to 10 000 sparsely interconnected ones. With the user-friendly UNIX Workstation host and integrated high speed VME channels we get a Neurostation exhibiting a power to price ratio many times greater than a supercomputer one. In this paper, we discuss first the SMART approach and its appeal. Second, we shortly present an hardware architecture for SMART sparse calculus. Third, we address the programming issue and display early results. Finally we summarize the advantages of the proposed machine with performance forecastings.