Abstract
Neural networks are an attractive new technology for signal processing applications, due to their adaptive, self-organizing, fault tolerant, and non-linear capabilities. An example of such an application, which is used to illustrate the results of the paper, involves a use of a multilayer perception network with error back-propagation learning for underwater target detection by means of a sound spectrogram analysis. The paper presents a method of implementing neural networks on parallel, programmable computers, which can effectively address the computational requirements of such signal processing applications. The method is applicable to multilayer connectionist networks and two-dimensional, SIMD (single-instruction multiple data stream) processor arrays. A detailed description along with comparisons to previously proposed methods is provided for a mapping of a multilayer perceptron network with back-propagation learning algorithm. The mapping includes partitioning of inputs larger than the processor array. The performance of the method is evaluated using the Nettalk neural network and is compared to that of other methods. In particular, it is shown that the implementation of the method on the Systolic/Cellular machine of Hughes results in the processing rate equal to 100 MCPS.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ahalt, S. C., F. D. Garber, I. Jouny and A. K. Krishnamurthy. (1989). Performance Of Synthetic Neural Network Classification of Noisy Radar Signals. Advances in Neural Information Processing 1.
Blelloch, G. and C. R. Rosenberg. (1987). Network Learning on the Connection Machine. 10th Intern. Joint Conf. on Artificial Intelligence. 323–326.
Brown, J. R., M. M. Garber and S. F. Venable. (1988). Artificial Neural Network on a SIMP Architecture. Symposium on the Frontiers of Massively Parallel Computations. 43–47.
Deprit, E. (1989). “Implementing Recurrent Back-Propagation on the Connection Machine.” Neural Networks. 2: 295–314.
Fike, C. T. (1968). Computer Evaluation of Mathematical Functions. Prentice-Hall.
Forrest, B. M., D. Roweth, N. Stroud, D. J. Wallace and G. V. Wilson. (1987). “Implementing Neural Network Models on Parallel Computers.” The Comp. Jour. 30(5): 413–419.
Gaudiot, J.-L., C. v. d. Malsburg and S. Shams. (1988). A Data-Flow Implementation of a Neurocomputer for Pattern Recognition Applications. Areospace Applications of Artificial Intelligence.
Gorman, R. P. and T. J. Sejnowski. (1988). “Learned Classification of Sonar Targets Using a Massively Parallel Network.” IEEE Trans. Accou., Speech, and Sig. Proc. 36(7): 1135–1140.
Hillis, W. D. (1985). The Connection Machine. Cambridge MA, MIT Press.
Hopfield, J. J. and D. W. Tank. (1985). ““Neural” Computation of Decisions in Optimization Problems.” Biol. Cybern. 52: 141–152.
Iwata, A., Y. Nagasaka and N. Suzumura. (1989). A Digital Hoher Monitoring System with Paul 3 Layers Neural Networks. International Joint Conference on Neural Networks. 2: 69–74.
Kato, H., H. Yoshizawa, H. Iciki and K. Asakawa. (1990). A Parallel Neurocomputer Architecture Towards Billion Connection Updates Per Second. Inter. Joint Conf. on Neural Networks. 2: 47–50.
Kosko, B. (1988). “Bidirectional Associative Memories.” IEEE Trans. Syst., Man, Cybern. 18: 49–60.
Kung, S. Y. and J. N. Hwang. (1988). Systolic Architectures for Artificial Neural Nets. IEEE Inter. Conf. on Neural Networks.
Kung, S. Y. and J. N. Hwang. (1989). “A Unified Systolic Architecture for Artificial Neural Networks.” Journal of Parallel and Distributed Computing. 6: 358–387.
Kwan, H. K. and C. K. Lee. (1989). Pulse Radar Detection Using a Multi-laver Neural Network. Inter. Joint Conf. on Neural Networks. 2: 75–80.
Kwan, H. K. and P. C. Tsang. (1990). Systolic Implementation of Multi-Laver Feed-Forward Neural Network with Back-Propagation Learning Scheme. Inter. Joint Conf. on Neural Networks. 2: 155–158.
Little, M. J. and J. Grinberg. (1988). The 3-D Computer: An Integrated Stack of WSI Wafers. Wafer Scale Integration. Boston, Kluwer.
Luttrell, S. P. (1989). “Image Compression Using a Multilayer Neural Network.” Pat. Recog. 10: 1–7.
Malkoff, D. B. (1990). A Neural Network for Real-Time Signal Processing. Advances in Neural Information Processing 2. 248–255.
Murry, A. F., A. V. W. Smith and Z. F. Butler. (1989). Bit-Serial Neural Networks. Neural Information Processing Systems. 573–587.
Paris, B.-P., G. Orsak, M. Varanasi and B. Aazhang. (1989). Neural Net Receivers in Multiple-Access Communications. Advances in Neural Information Processing 1. 272–280.
Piazza, F., M. Marchesi, G. Orlandi and A. Unicini. (1990). Coarse-Grained Processor Array Implementing the Multilayer Neural Network Model. Inter. Symp. on Cir. & Sys. 4: 2963–2966.
Pomerleau, D. A., G. L. Gusciora, D. S. Touretzky and H. T. Kung. (1988). Neural Network Simulation at Warp Speed: How We Got 17 Million Connections Per Second. IEEE International Confer. on Neural Networks.
Przytula, K. W. (1989). Systolic/Cellular System.
Przytula, K. W. and J. G. Nash. (1987). A Special Purpose Coprocessor for Signal Processing. 21st Asilomar Conference on Signals, Systems and Computers.
Reddaway, S. F. (1973). DAP — a Distributed Array Processor. 1st Ann. Symp. on Computer Architecture. 61–65.
Roth, M. W. (1989). “Neural Networks for Extraction of Weak Targets in High Clutter Environments.” IEEE Trans. on Sys., Man, and Cyber. 19(5): 1210-12-17.
Rumelhart, D. E., G. E. Hinton and R. J. Williams. (1986). Learning Internal Representations by Error Propagation. Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge, MIT Press.
Sejnowski, T. J. and C. R. Rosenberg. (1987). “Parallel Networks that Learn to Pronounce English Text.” Comp. Sys. 1: 145–168.
Urick, R. J. (1983). Principles of Underwater Sound. McGraw-Hill.
Witbrock, M. and M. Zagha. (1989). An Implementation of Back-Propagation Learning on GF11. a Large SIMP Parallel Computer.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1991 Springer Science+Business Media New York
About this chapter
Cite this chapter
Shams, S., Przytula, K.W. (1991). Implementation of Multilayer Neural Networks on Parallel Programmable Digital Computers. In: Bayoumi, M.A. (eds) Parallel Algorithms and Architectures for DSP Applications. The Springer International Series in Engineering and Computer Science, vol 149. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-3996-4_9
Download citation
DOI: https://doi.org/10.1007/978-1-4615-3996-4_9
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4613-6786-4
Online ISBN: 978-1-4615-3996-4
eBook Packages: Springer Book Archive