Abstract
This paper proposes a new way of digital hardware imple- mentation of nonlinear activation functions in feed-forward neural net- works. The basic idea of this new realization is that the nonlinear functions can be implemented using a matrix-vector multiplication. Recently a new approach was proposed for the realization of matrix-vector mul- tiplications which approach can also be applied for implementing the nonlinear functions if the nonlinear functions are approximated by sim- ple basis functions. The paper proposes to use B-spline basis functions to the approximate nonlinear sigmoidal functions, it shows that this ap proximation fulfills the general requirements on the activation functions, presents the details of the proposed hardware implementation, and gives a summary of an extensive study about the effects of B-spline nonlin- ear function realization on the size and the trainability of feed-forward neural networks.
This work was supported by the Hungarian Fund for Scientific Research (OTKA) under contract T023868
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
M. H. Hassoun, Fundamentals of artificial neural networks, MIT Press, 1995.
T. G. Clarkson and Y. Ding, RAM-Based Neural Networks, chapter Extracting directional information for the recognition of fingerprints by pRAM networks, pp. 174–185, World Scientific, 1998.
“White blood cell classification & counting with the ZISC, ” http://www.fr.ibm.com/france/cdlab/zblcell.htm, 2000.
Tamás Szabó, Béla Fehér, and Gäbor Horväth, “Neural network implementation using distributed arithmetic, ” in Proceedings of the International Conference on Knowledge-based Electronic Systems, Adelaide, Australia, 1998, vol. 3, pp. 511–520.
Tamäs Szabó, Lörinc Antoni, Gäbor Horváth, and Béla Fehér, “An efficient implementation for a matrix-vector multiplier structure, ” in Proceedings of IEEE International Joint Conference on Neural Networtks, IJCNN2000, 2000, vol. II, pp. 49–54.
Manferd Glesner and Werner Pöchmüller, Neurocomputers, an overview of neural networks in VLSI, Neural Computing. Chapman & Hall, 1994.
Franco Scarselli and Ah Chung Tsoi, “Universal approximation using feedforward neural networks: A survey of some existing methods, and some new results, ” Neural Networks, vol. 11, no. 1, pp. 15–37, 1998.
Věra Kurková, “Approximation of functions by neural networks, ” in Proceedings of NC’98, 1998, pp. 29–36.
M. B. Stinchcombe and H. White, “Approximating and learning unknown mappings using multilayer networks with bounded weights, ” in Proc. of Int. Joint Conference on Neural Networks, IJCNN’90. 1990, vol.III, pp. 7–16, IEEE Press.
Moshe Leshno, Vladimir Ya. Lin, Allan Pinkus, and Shimon Schocken, “Multilayer feedforward networks with a nonpolynomial activation function can approximate any function, ” Neural Networks, vol. 6, pp. 861–867, 1993.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Szabó, T., Horváth, G. (2001). An Efficient Hardware Implementation of Feed-Forward Neural Networks* . In: Monostori, L., Váncza, J., Ali, M. (eds) Engineering of Intelligent Systems. IEA/AIE 2001. Lecture Notes in Computer Science(), vol 2070. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45517-5_34
Download citation
DOI: https://doi.org/10.1007/3-540-45517-5_34
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42219-8
Online ISBN: 978-3-540-45517-2
eBook Packages: Springer Book Archive