Massively Parallel Training of Multi Layer Perceptrons With Irregular Topologies

  • D. Koll
  • M. Riedmiller
  • H. Braun


In this paper we present an approach to the training of feed forward neural networks on massively parallel SIMD-architectures. In order to cover a wide field of applications we focus our attention on the flexibility of the load balancing routines. Our approach is characterized by three important properties: 1. All four types of parallelism inherent in the training phase are used. 2. In a preprocessing step neural networks are transformed into equivalent topologies, more suited for parallel computation. 3. Each learning task can be parallelized in a number of different ways, the best of which is chosen according to estimations of the computing efficiency.

Following these concepts we developed PINK2, a massively parallel simulator kernel for the MasPar MP1216. In contrast to most known approaches, efficient only for special topologies, it achieves good computing performance on a broad range of differing benchmark problems.


Load Balance Learning Problem Parallel Representation Connection Level Parallel Training 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. Singer: Parallel Computing 14, 305 (1990)CrossRefGoogle Scholar
  2. [2]
    X. Zhang, M. Mckenna, J.P. Mesirov, D.L. Waltz: NIPS 2, 801 (1990)Google Scholar
  3. [3]
    Wei-Ming Lin, V.K. Prasanna, K.W. Przytula: IEEE Transactions on Computers, Vol. 40, No. 12, 1390 (Dec 1991)CrossRefGoogle Scholar
  4. [4]
    N. Mache, Master thesis, University of Stuttgart, 1992Google Scholar
  5. [5]
    I. Pitas (ed), A. Petrowski, H. Paugam-Moisy: Parallel Algorithms pp 259–328 Wiley 1993Google Scholar
  6. [6]
    S.E. Fahlmann: CMU-CS-88-162 (1988)Google Scholar
  7. [7]
    M. Riedmiller, H. Braun: Proc. ICNN ’93, 379 (1993)Google Scholar
  8. [8]
    H. Braun, J. Weisbrod: Proc. ICANNGA ’93, 25 (1993)Google Scholar
  9. [9]
    J. Schaefer, H. Braun: Proc. ICANNGA ’95Google Scholar
  10. [10]
    H. Braun, P. Zagorski: Proc. 3rd PPSN, 444 (1994)Google Scholar
  11. [11]
    W. Butscher: Supercomputer ’91, 187 (1991)CrossRefGoogle Scholar
  12. [12]
    A. Zell: Simulation Neuronaler Netze, Addison Wesley 1994Google Scholar

Copyright information

© Springer-Verlag/Wien 1995

Authors and Affiliations

  • D. Koll
    • 1
  • M. Riedmiller
    • 1
  • H. Braun
    • 1
  1. 1.Institut für Logik, Komplexität und DeduktionssystemeUniversität KarlsruheKarlsruheGermany

Personalised recommendations