Abstract
We will address here the simulation of large neural networks applied to real-world problems. In particular, we will consider the Multi Layer Perceptron (MLP) network and the back-propagation (BP) learning algorithm, showing that an efficient learning in MLP-BP networks depends on two factors: a fast BP algorithm and an efficient implementation respect to the particular target architecture. Unfortunately, as described here, the two objectives are mutually exclusive.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
SNAP — SIMD Numerical Array Processor. HNC, 5930 Cornerstone Court West, S.Diego, CA, (1994)
Adamo, J.M., Anguita, D.: Object Oriented Design of a BP Neural Network and Implementation on the Connection Machine (CM-5). ICSI Technical Report TR-94-046 (1994)
Anderson, E.C., Dongarra, J.: Perfomance of LAPACK: A Portable Library of Numerical Linear Algebra Routines. Proc. of the IEEE 81:8 (1993) 1094–1101
Anguita, D., Parodi, G., and Zunino, R.: An Efficient Implementation of BP on RISC-based Workstations. Neurocomputing 6 (1994) 57–65
Anguita, D., DaCanal, A., DaCanal, W., Falcone, A., Scapolla, A.M.: On the distributed implementation of the back-propagation. Proc. of the Int. Conf. on Artificial Neural Networks, Sorrento, Italy (1994) 1376–1379
Asanović, K., Beck, J., Callahan, T., Feldman, J., Irissou, B., Kingsbury, B., Kohn, P., Lazzaro, J., Morgan, N., Stoutamire, D., Wawrzynek, J.: CNS-1 Architecture Specification. ICSI Technical Report TR-93-021 (1993)
Asanović, K., Beck, J., Feldman, J., Morgan, N., Wawrzynek, J.: Designing a Connectionist Network Supercomputer. Int. J. of Neural Systems, 4:4 (1993) 317–326
Battiti, R.: First-and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method. Neural Computation 4 (1992) 141–166
Corana, A., Rolando, C., Ridella, S.: A Highly Efficient Implementation of Backpropagation Algorithm on SIMD Computers. High Performance Computing, J.-L.Delhaye and E.Gelenbe (Eds.) Elsevier (1989) 181–190
Corana, A., Rolando, C., Ridella, S.: Use of Level 3 BLAS Kernels in Neural Networks: The Back-propagation algorithm. Parallel Computing 89 (1990) 269–274
Dongarra,J.: Linear Algebra Library for High-Performance Computers. Frontiers of Supercomputing II. K.R.Ames and A.Brenner (Eds.) University of California Press (1994)
Grajski, K.A., Chinn, G., Chen, C., Kuszmaul, C., Tomboulian, S.: Neural Network Simulation on the MasPar MP-1 Massively Parallel Processor. Proc. of the Int. NN Conf. (1990) 673
Ienne, P.: Architectures for Neuro-Computers: Review and Performance Evaluation. Technical Report 93/21, Swiss Federal Institute of Technology, Lausanne (1993)
Jackson, D., Hammerstrom, D.: Distributing Back Propagation Networks Over the Intel iPSC/860 Hypercube. Proc. of the Int. Joint Conf. on NN, Seattle, WA, USA (1991) 1569–1574
Liu, X., Wilcox, G.L.: Benchmarking of the CM-5 and the Cray Machines with a Very Large Backpropagation Neural Network. Proc. of the IEEE Int. Conf. on NN, Orlando, FL, USA (1994) 22–27
Müller, U.A.: A High Performance Neural Net Simulation Environment. Proc. of the IEEE Int. Conf. on NN, Orlando, FL, USA (1994) 1–4
Müller, S.M.: A Performance Analysis of the CNS-1 on Large, Dense Backpropagation Networks. ICSI Technical Report TR-93-046 (1993)
Renals, S., Morgan, N.: Connectionist Probability Estimation in HMM Speech Recognition. ICSI Technical Report TR-92-081 (1992)
Sà nchez, E., Barro, S., Regueiro, C.V.: Artificial Neural Networks Implementation on Vectorial Supercomputers. Proc. of the IEEE Int. Conf. on NN, Orlando, FL, USA (1994) 3938–3943
Singer, A.: Exploiting the Inherent Parallelism of Artificial Neural Networks to Achieve 1300 Million Interconnets per Second. Proc. of the Int. NN Conf., Paris, France (1990) 656–660
Yamada, T., Yabuta, T.: Dynamic System Identification Using Neural Networks. IEEE Trans. on NN 23:1 (1993) 204–211
Zhang, X., Mckenna, M., Mesirov, J.P., Waltz, D.L.: An Efficient Implementation of the Back-Propagation Algorithm on the Connection Machine CM-2. Advances in Neural Information Processing Systems 2, D.S.Touretzky (Ed.), (1990) 801–809
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1995 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Anguita, D., Passaggio, F., Zunino, R. (1995). Learning in large neural networks. In: Hertzberger, B., Serazzi, G. (eds) High-Performance Computing and Networking. HPCN-Europe 1995. Lecture Notes in Computer Science, vol 919. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0046639
Download citation
DOI: https://doi.org/10.1007/BFb0046639
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-59393-5
Online ISBN: 978-3-540-49242-9
eBook Packages: Springer Book Archive