Abstract
The paper describes the implementation of a systolic array for a multilayer perceptron on ALTERA FLEX10KE FPGAs with a hardware-friendly learning algorithm. A pipelined adaptation of the on-line backpropagation algorithm is shown. It better exploits the parallelism because both the forward and backward phases can be performed simultaneously. As a result, a combined systolic array structure is proposed for both phases. Analytic expressions show that the pipelined version is more efficient than the non-pipelined version. The design is implemented and simulated using VHDL at different levels of abstraction and finally mapped on FPGAs.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
A. Singer, “Implementations of artificial neural networks on the connection machine”, Parallel Computing, vol. 14, 1990, pp. 305–315.
S. Shams, and J.L. Gaudiot, “Implementing Regularly Structured Neural Networks on the Dream Machine” IEEE transactions on neural networks, vol. 6, no.2, March 1995, pp. 408–421.
W-M Iin, V. K. Prasanna, and K. W. Przytula, “Algorithmic Mapping of Neural Network Models onto Parallel SIMD Machines”, IEEE Transactions on Computers, Vol. 40, no. 12, December 1991, pp. 1390–1401.
S.R. Jones, K.M. Sammut, and J. Hunter, “Learning in Linear Systolic Neural Network Engines: Analysis and Implementation”, Transactions on Neural Netwoks, Vol. 5, no. 4, July 1994, pp. 584–593.
D. Naylor, S. Jones, and D. Myers, “Backpropagation in Linear Arrays—A performance Analysis and Optimization”, IEEE Transactions on Neural Networks, Vol. 6, no. 3, May 1995, pp. 583–595.
P. Murtagh, A.C. Tsoi, and N. Bergmann, “Bit-serial array implementation af a multilayer perceptron”, IEEE Proceedings-E, Vol. 140, no. 5, September 1993, pp. 277–288.
X. Zhang, M. McKenna, J.P. Mesirov, and D Waltz, “An efficient implementation of the backpropagation algorithm on the conection Machine CM-2, Advances in Neural Information Porcessing Systems 2, D.S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 801–809.
C.R. Rosemberg, and G. Belloch, “An implementation of network learning on the Connection machine”, Connectionist Models an their Implications, D. Waltz and J Feldman, eds., Ablex, Norwood, NJ. 1988.
A. Petrowski, G. Dreyfus, and C. Girault, “Performance Analysis of a Pipelined Backpropagation Parallel Algorithm”, IEEE Transaction on Neural Networks, Vol.4, no. 6, November 1993, pp. 970–981.
R. Gadea, A. MocholĂ, “Systolic Implementation of a Pipelined On-Line Backpropagation”, Proc., April 1998.
D.E. Rumelhart, G.E. Hinton, and R.J. Williams, “Learning internal representations by error backpropagation, Parallel Distributed processing, Vol. 1, MIT Press, Cambridge, MA, 1986, pp. 318–362.
S.E. Falhman, “Faster learning variations on backpropagation: An empirical study”, Proc. 1988 Connectionist Models Summer School, 1988, pp. 38–50.
M.R. Zargham, Computer Architecture single and parallel systems, Prentice Hall International Inc. 1996.
Editor information
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gironés, R.G., Salcedo, A.M. (1999). Forward-backward parallelism in on-line backpropagation. In: Mira, J., Sánchez-Andrés, J.V. (eds) Engineering Applications of Bio-Inspired Artificial Neural Networks. IWANN 1999. Lecture Notes in Computer Science, vol 1607. Springer, Berlin, Heidelberg . https://doi.org/10.1007/BFb0100482
Download citation
DOI: https://doi.org/10.1007/BFb0100482
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-66068-2
Online ISBN: 978-3-540-48772-2
eBook Packages: Springer Book Archive